Tunnel construction monitoring and analysis method and system based on AI video monitoring

Abstract

The present invention provides a tunnel construction monitoring and analysis method and system based on AI video surveillance, which relates to the field of artificial intelligence technology. The method obtains a set of original video stream data of tunnel construction through multi-view video acquisition equipment, generates a spatiotemporal feature sequence through spatiotemporal feature fusion processing, and extracts dynamic features of the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set. Based on this, construction personnel statistical processing is performed to generate statistical results of the number of personnel entering the tunnel, personnel distribution statistics of each construction site, and personnel identity matching results. The work efficiency is compared and analyzed based on the personnel statistical results and a preset construction progress plan, and a construction progress deviation index and a resource allocation optimization strategy are generated. When the progress deviation index exceeds a first threshold, an early warning is triggered and the strategy is output to a management terminal, thereby realizing effective monitoring and management of tunnel construction.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a tunnel construction monitoring and analysis method and system based on AI video monitoring. Background Art

[0002] In the field of tunnel construction, the ever-expanding scale and increasing complexity of projects are placing ever-more stringent demands on real-time monitoring and efficient management of the construction process. Traditional tunnel construction monitoring methods rely primarily on manual inspections, single-point video surveillance, and simple sensor data collection. However, these methods have numerous limitations.

[0003] Manual inspections are not only inefficient and difficult to achieve full, comprehensive coverage of construction sites, but are also easily affected by human factors, such as the experience and attention of inspectors, resulting in high subjectivity and uncertainty in monitoring results. While single-point video surveillance can provide images of a specific area, its single viewing angle prevents it from fully capturing dynamic changes within the construction scene. Numerous blind spots exist, making it difficult to accurately and comprehensively analyze the movements of construction workers, the operation of construction equipment, and the overall state of the construction surface. Simple sensor data collection methods, such as collecting only environmental parameters like temperature and humidity, or simply recording the operating time of construction equipment, provide limited data dimensions and are unable to reflect the complex spatiotemporal relationships and dynamic characteristics of the construction process, making it difficult to meet the demands for refined management of the construction process.

[0004] In terms of personnel management, traditional monitoring methods struggle to accurately count the number of personnel entering the tunnel, monitor the distribution of personnel at various construction sites in real time, and even fail to quickly and accurately match personnel identities. This makes it difficult to obtain accurate personnel information in a timely manner during safety incidents or emergencies, greatly complicating rescue and management efforts. Furthermore, due to a lack of effective data support, construction progress assessments rely primarily on the experience and subjective judgment of managers, making it difficult to accurately quantify progress deviations and dynamically adjust resource allocation based on real-time construction conditions. This results in low construction efficiency and significant resource waste. Summary of the Invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a tunnel construction monitoring and analysis method based on AI video monitoring, the method comprising:

[0006] The original video stream data set of the construction scene is acquired in real time by a multi-view video acquisition device deployed in the tunnel construction area, wherein the original video stream data set includes a plurality of spatially continuous frame sequences with time stamps aligned;

[0007] Performing spatiotemporal feature fusion processing on the original video stream data set to generate a spatiotemporal feature sequence, wherein each spatiotemporal feature unit in the spatiotemporal feature sequence is composed of pixel area motion trajectory features, personnel wear identification features, and construction equipment contour features of the corresponding frame sequence;

[0008] Performing dynamic feature extraction processing on the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set, wherein the dynamic spatiotemporal feature set includes construction personnel movement path features, construction equipment operation frequency features, and construction surface operation status features;

[0009] Performing construction personnel statistics processing based on the dynamic spatiotemporal feature set to generate statistics on the number of personnel entering the tunnel, personnel distribution statistics at each construction site, and personnel identity matching results, wherein the personnel identity matching results are determined by comparing identification features worn by the personnel with identification templates in a pre-registered identity database;

[0010] Based on the statistical results of the number of people entering the cave, the statistical results of the personnel distribution and the preset construction progress plan, a comparative analysis of work efficiency is performed to generate a construction progress deviation index and a resource allocation optimization strategy. When it is detected that the construction progress deviation index exceeds a first threshold, a construction early warning instruction is triggered and the resource allocation optimization strategy is output to the management terminal.

[0011] On the other hand, an embodiment of the present invention also provides a tunnel construction monitoring and analysis system based on AI video monitoring, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0012] Based on the above aspects, the present invention uses spatiotemporal feature fusion processing to transform spatially continuous frames scattered across multi-view video streams into a temporally correlated spatiotemporal feature sequence. This not only overcomes the field of view limitations of traditional single-view monitoring but also constructs a global semantic model of the construction scene by organically integrating pixel-region motion trajectories, personnel identification, and construction equipment contour features, enabling a comprehensive representation of the spatiotemporal evolution of construction activities. Furthermore, dynamic feature extraction further mines key dynamic information such as construction personnel movement paths, equipment operation frequency, and work status, providing multidimensional data support for construction process management and control. Through construction personnel statistics processing and work efficiency comparative analysis, the number, distribution, and identity matching results of personnel entering the tunnel are output in real time. Combined with the preset schedule, quantitative progress deviation indicators and resource allocation optimization strategies are generated, thereby triggering timely warnings and promoting targeted adjustment plans when construction progress is abnormal. This deep integration of spatiotemporal feature modeling, dynamic behavior recognition, and construction management decision-making not only enhances the intelligent level of tunnel construction monitoring but also significantly improves construction efficiency and safety through data-driven resource allocation optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the execution flow of the tunnel construction monitoring and analysis method based on AI video monitoring provided by an embodiment of the present invention.

[0014] Figure 2 Schematic diagram of exemplary hardware and software components of a tunnel construction monitoring and analysis system based on AI video surveillance provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This is a flow chart of a tunnel construction monitoring and analysis method based on AI video monitoring provided by an embodiment of the present invention. The tunnel construction monitoring and analysis method based on AI video monitoring is introduced in detail below.

[0016] Step S110: acquiring an original video stream data set of the construction scene in real time through a multi-view video acquisition device deployed in the tunnel construction area, wherein the original video stream data set includes a plurality of spatially continuous frame sequences aligned with time stamps.

[0017] In this embodiment, to comprehensively and accurately monitor the tunnel construction scene, multi-view video capture equipment can be deployed within the tunnel construction area. Specifically, cameras are installed at different key locations within the tunnel area. For example, a camera is installed at the tunnel entrance, 5 meters above the ground and tilted 30 degrees downward to clearly capture personnel and equipment entering the tunnel. Cameras are installed every 20 meters in the middle of the tunnel, 4 meters above the ground, with a horizontal viewing angle of 120 degrees to ensure comprehensive coverage of the construction area. A camera is also installed at the tunnel exit, at a similar height and angle as the entrance camera.

[0018] The video stream data captured by each camera can be transmitted to the data processing center in real time via the network. Because the startup times of different cameras may vary slightly, to ensure the accuracy of subsequent processing, the video stream data collected by each camera can be timestamped. Specifically, the data processing center adds timestamp information to each video stream data. Then, by comparing the timestamps of the different video streams, the timestamp of one camera is used as a benchmark to adjust the time of the video stream data of the other cameras. For example, assuming that the timestamp of the camera at the tunnel entrance is used as a benchmark, if the timestamp of the video stream data of a camera in the middle is found to be 0.1 second later than that of the camera at the entrance, the timestamps of all video frames of this camera are uniformly advanced by 0.1 second. After this timestamp alignment operation, a set of raw video stream data is obtained, consisting of multiple spatially continuous frame sequences. Each spatially continuous frame sequence represents a series of video frames from a specific perspective, and each spatially continuous frame sequence is temporally aligned.

[0019] Step S120: performing spatiotemporal feature fusion processing on the original video stream data set to generate a spatiotemporal feature sequence, wherein each spatiotemporal feature unit in the spatiotemporal feature sequence is composed of the pixel area motion trajectory features, personnel wear identification features and construction equipment contour features of the corresponding frame sequence.

[0020] In this embodiment, spatiotemporal feature fusion processing is performed on the original video stream data set, and this process includes multiple sub-steps.

[0021] Step S121: performing motion trajectory segmentation processing on each spatially continuous frame sequence in the original video stream data set to obtain pixel area motion trajectory features of each construction worker, wherein the pixel area motion trajectory features include independent motion trajectory segments and corresponding trajectory bounding box coordinates.

[0022] In this step, a series of processing is performed on each spatially continuous frame sequence to obtain the pixel area motion trajectory characteristics of the construction workers. The details are as follows:

[0023] Step S1211: grayscale processing is performed on each frame image in the spatial continuous frame sequence to convert the color image into a grayscale image, and a pixel brightness difference matrix between adjacent frames is calculated based on a preset adjacent time stamp interval.

[0024] For example, taking a spatially continuous frame sequence from a certain perspective, each color image in the sequence is converted to a grayscale image using a weighted average method. Specifically, for each pixel in the color image, its RGB values ​​are R, G, and B, respectively. The corresponding grayscale value is calculated using the formula Grayscale Value = 0.299*R + 0.587*G + 0.114*B. For example, if the RGB values ​​of a pixel are (200, 150, 100), its grayscale value is 0.299*200 + 0.587*150 + 0.114*100 = 153.55, which is rounded to 154. After this process, the entire color image is converted to a grayscale image.

[0025] Next, we calculate the pixel brightness difference matrix between adjacent frames based on the preset time stamp interval (assuming it's 0.033 seconds, corresponding to a frame rate of 30 frames per second). Specifically, for two adjacent grayscale frames, we calculate the brightness difference pixel by pixel. For example, if a pixel in the nth grayscale image has a grayscale value of 120, and in the n+1th grayscale image, the same pixel has a grayscale value of 130, then the brightness difference is 130 - 120 = 10. This yields a pixel brightness difference matrix with the same image size.

[0026] Step S1212: extracting a dynamic background model based on the pixel brightness difference matrix, separating the foreground motion target area through background difference method, performing morphological closing operation on the foreground motion target area, eliminating noise interference and filling the internal holes of the target, and generating a connected domain labeling map.

[0027] In this embodiment, a Gaussian mixture model (GMM) can be used for modeling. For each pixel, multiple Gaussian distributions are used to fit the changes in its brightness value. For example, for each pixel, three Gaussian distributions are used to fit the image, each with its own mean, variance, and weight. In the initial stage, these parameters are randomly initialized. Then, through continuous data updates, these parameters are adjusted to adapt to changes in pixel brightness values. After a period of training, a stable Gaussian mixture model is obtained, which is used as the dynamic background model.

[0028] Next, the background difference method is used to separate the foreground moving object region. For each frame of the grayscale image, it is compared with the dynamic background model and the brightness difference of each pixel is calculated. If the difference exceeds a preset threshold (assuming it is 20), the pixel is considered to belong to the foreground moving object region. For example, if a pixel has a grayscale value of 150 in the current frame and the corresponding mean value in the dynamic background model is 120, the difference is 30, which exceeds the threshold of 20, and the pixel is marked as part of the foreground moving object region.

[0029] Then, a morphological closing operation is performed on the separated foreground moving target region. For example, a dilation operation is first performed, using a 3x3 structuring element to dilate the foreground moving target region. Specifically, the center of the structuring element is aligned with each pixel in the foreground moving target region. If there is a pixel in the area covered by the structuring element that belongs to the foreground moving target region, the pixel corresponding to the center of the structuring element is marked as the foreground moving target region. Then, an erosion operation is performed, also using a 3x3 structuring element, and the center of the structuring element is aligned with each pixel in the foreground moving target region. If all pixels in the area covered by the structuring element belong to the foreground moving target region, the pixel corresponding to the center of the structuring element is retained as the foreground moving target region. Through the combination of dilation and erosion operations, noise interference is eliminated and the internal holes of the target are filled to generate a connected domain labeling map.

[0030] Step S1213: Fit the minimum bounding rectangle of each foreground moving target based on the connected domain label map, calculate the centroid coordinates and aspect ratio of each bounding rectangle, map the centroid coordinates to the global coordinate system, and record the pixel area of ​​each bounding rectangle.

[0031] In this embodiment, a rotating caliper algorithm can be used to calculate the minimum bounding rectangle. For a connected domain, its boundary points are first found. Then, by continuously rotating the tangent lines of the boundary points, a minimum rectangle that encloses the connected domain is found. For example, for an irregularly shaped connected domain, the rotating caliper algorithm can be used to find a rectangle that just encloses it, with all four sides of the rectangle being tangent to the boundary points of the connected domain.

[0032] Calculate the centroid coordinates of each bounding rectangle. This is done by adding the coordinates of the four vertices of the bounding rectangle and dividing by 4. For example, if the coordinates of the vertices of a bounding rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), the centroid coordinates are ((x1+x2+x3+x4) / 4, (y1+y2+y3+y4) / 4). Also, calculate the aspect ratio of the bounding rectangle: the length divided by the width.

[0033] Map the centroid coordinates to the global coordinate system. Because each camera is installed at a different position and angle, the image coordinate system it captures has a certain conversion relationship with the global coordinate system. By pre-measuring the camera's installation position and angle, as well as the actual size of the tunnel construction area, a transformation matrix is ​​established between the image coordinate system and the global coordinate system. For example, assuming the centroid coordinates of a bounding rectangle in the image coordinate system are (x_img, y_img), they can be converted to coordinates in the global coordinate system (x_global, y_global) using the transformation matrix.

[0034] Finally, record the pixel area of ​​each bounding rectangle, which is the length multiplied by the width. For example, if the bounding rectangle is 50 pixels long and 30 pixels wide, its pixel area is 50*30=1500 pixels.

[0035] Step S1214: Use a multi-target tracking algorithm to associate the centroid coordinates of consecutive time stamps with motion trajectories, and calculate the Euclidean distance between the centroid coordinates of the same target at adjacent time stamps. When the Euclidean distance is less than a preset distance threshold and the change in the aspect ratio is less than a preset ratio threshold, it is determined to be a motion trajectory segment of the same target.

[0036] In this embodiment, a multi-target tracking algorithm (such as a Kalman filter) can be used to correlate the centroid coordinates of consecutive timestamps. The Kalman filter tracks the target's trajectory through two steps: prediction and update. In the prediction step, the target's position at the next timestamp is predicted based on its historical motion information. In the update step, the prediction result is corrected based on the actual measured target position.

[0037] Specifically, for the centroid coordinates of adjacent timestamps, the Euclidean distance between them is calculated. The Euclidean distance is calculated by taking the square root of the sum of the squares of the differences between the horizontal and vertical coordinates of the two centroid coordinates. For example, if the centroid coordinates of an object at timestamp n are (x_n, y_n), and at timestamp n+1 are (x_n+1, y_n+1), then the Euclidean distance between them is the square root of ((x_n+1-x_n)^2+(y_n+1-y_n)^2).

[0038] At the same time, calculate the change in the aspect ratio of the target bounding rectangle between adjacent timestamps. Assume that at timestamp n, the aspect ratio of the target bounding rectangle is r_n, and at timestamp n+1, the aspect ratio is r_n+1. Then the change in aspect ratio is |r_n+1-r_n| / r_n.

[0039] When the Euclidean distance is less than a preset distance threshold (assuming 15 pixels) and the change in aspect ratio is less than a preset ratio threshold (assuming 0.1), the trajectory segments are determined to be the same target. For example, at the nth timestamp and the (n+1th) timestamp, if the Euclidean distance between the center of mass coordinates of an target is 10 pixels, less than the preset distance threshold of 15 pixels, and the change in aspect ratio is 0.05, less than the preset ratio threshold of 0.1, the targets at these two timestamps are considered to be the same target's trajectory segments.

[0040] Step S1215: Merge the motion trajectory segments of the same target at consecutive time stamps to generate independent motion trajectory segments, and calculate the trajectory bounding box coordinates according to the global coordinate sequence of the bounding rectangle. The trajectory bounding box coordinates are the minimum convex polygon vertex coordinate set containing all the vertices of the bounding rectangle.

[0041] In this embodiment, all motion trajectory segments belonging to the same target can be connected in chronological order. For example, if a target has a motion trajectory segment at timestamps 1-5 and another motion trajectory segment at timestamps 6-10, and if these two trajectory segments are determined to belong to the same target, they can be merged into a single independent motion trajectory segment.

[0042] The coordinates of the trajectory bounding box are calculated based on the global coordinate sequence of the bounding rectangle. First, the vertex coordinates of the bounding rectangle of the same target at all timestamps are collected, and then a convex hull algorithm (such as the Graham scan algorithm) is used to calculate the minimum convex polygon containing all these vertices. The basic idea of ​​the Graham scan algorithm is to first find the point with the smallest vertical coordinate among all vertices, then sort the other vertices according to the polar angle, and finally traverse the sorted vertices in turn, using the cross product to determine whether certain vertices need to be deleted, and finally obtain the minimum convex polygon. For example, for a target's bounding rectangle vertex coordinate set at multiple timestamps, the Graham scan algorithm can obtain a minimum convex polygon, and the vertex coordinates of this polygon are the coordinates of the trajectory bounding box.

[0043] Step S1216: Verify the integrity of the independent motion trajectory segment, count the coverage time of the trajectory bounding box coordinates on the time axis, and retain the independent motion trajectory segment and the corresponding trajectory bounding box coordinates when the coverage time exceeds a preset time threshold and the pixel area change rate is lower than a preset area change threshold.

[0044] In this embodiment, the coverage duration of the trajectory bounding box coordinates on the time axis is calculated, that is, the time difference from the start timestamp to the end timestamp of the independent motion trajectory segment. For example, if the start timestamp of an independent motion trajectory segment is 10 seconds and the end timestamp is 20 seconds, then its coverage duration is 20-10 = 10 seconds.

[0045] At the same time, calculate the pixel area change rate. For each timestamp in the independent motion trajectory segment, record the maximum and minimum values ​​of the bounding rectangle pixel area. The pixel area change rate is calculated as (maximum value - minimum value) / average value. For example, in an independent motion trajectory segment, the maximum value of the bounding rectangle pixel area is 2000 pixels, the minimum value is 1500 pixels, and the average value is (2000 + 1500) / 2 = 1750 pixels. The pixel area change rate is (2000 - 1500) / 1750 = 0.286.

[0046] When the coverage duration exceeds the preset duration threshold (assuming 8 seconds) and the pixel area change rate is lower than the preset area change threshold (assuming 0.3), the independent motion trajectory segment and the corresponding trajectory bounding box coordinates are retained. For example, if the coverage duration of an independent motion trajectory segment is 12 seconds, which exceeds the preset duration threshold of 8 seconds, and the pixel area change rate is 0.2, which is lower than the preset area change threshold of 0.3, the independent motion trajectory segment and the corresponding trajectory bounding box coordinates are retained.

[0047] Step S122: Extract a color space distribution histogram in the pixel area corresponding to the coordinates of the trajectory bounding box, and classify the color space distribution histogram based on a pre-trained wear identification classification model to obtain the personnel wear identification features. The wear identification classification model is trained using a training data set labeled with safety helmets, reflective clothing, and work type numbers.

[0048] In this embodiment, first, the image of the pixel area is converted to the HSV color space, because the HSV color space can better reflect the essential characteristics of color. Then, the three channels of H (hue), S (saturation), and V (brightness) are quantized separately. It is assumed that the H channel is quantized into 18 intervals, the S channel is quantized into 4 intervals, and the V channel is quantized into 3 intervals. For each pixel point in the pixel area, the interval to which it belongs is determined according to its HSV value, and then the number of pixels in each interval is counted to obtain a color space distribution histogram. For example, for a pixel point, its HSV value is (30, 0.5, 0.8), and after quantization of the H channel, it belongs to the third interval, after quantization of the S channel, it belongs to the second interval, and after quantization of the V channel, it belongs to the third interval, then one count is added in the corresponding interval.

[0049] The color space distribution histogram is then classified based on a pretrained wear identification classification model. This wear identification classification model is trained using a training dataset labeled with safety helmets, reflective clothing, and work type numbers. The training dataset contains a large number of image samples, each labeled with information such as whether it contains a safety helmet, reflective clothing, and work type number. During the training process, a convolutional neural network (CNN) is used as the model architecture. The backpropagation algorithm continuously adjusts the CNN parameters, enabling the CNN to accurately identify wear identification in images.

[0050] Next, the extracted color space distribution histogram is fed into a pre-trained wear identification classification model. This model then outputs the wear identification features of the person within that pixel, such as whether they are wearing a hard hat, reflective clothing, and their job number. For example, the wear identification classification model outputs that the person within that pixel is wearing a red hard hat, yellow reflective clothing, and their job number is 007.

[0051] Step S123: Perform edge enhancement processing on the outline of the construction equipment in the spatial continuous frame sequence, extract the polygon vertex coordinate set of the enhanced outline, perform shape matching based on the polygon vertex coordinate set and the pre-stored equipment template library, and determine the type of construction equipment and the corresponding operating status.

[0052] In this embodiment, edge enhancement processing is performed on the outline of the construction equipment in the spatial continuous frame sequence. The specific steps are as follows:

[0053] Step S1231: converting each frame image of the spatial continuous frame sequence into a grayscale image, and performing binarization segmentation on the grayscale image based on a preset fixed grayscale threshold to obtain an initial binarized image.

[0054] Each color image in the spatially continuous frame sequence is converted to a grayscale image using the same method as in step S1211, namely, a weighted average method. For example, for a color image, the RGB value of each pixel is converted to a grayscale value using the formula grayscale value = 0.299*R + 0.587*G + 0.114*B to obtain a grayscale image.

[0055] The grayscale image is binarized based on a preset fixed grayscale threshold (assuming it's 128). For each pixel in the grayscale image, if its grayscale value is greater than or equal to 128, its value is set to 255 (white); if its grayscale value is less than 128, its value is set to 0 (black). This process results in the initial binary image.

[0056] Step S1232: Gaussian filtering is performed on the initial binary image to eliminate high-frequency noise interference to generate a smoothed binary image, and the Canny edge detection algorithm is used to extract edge pixels from the smoothed binary image to generate an initial edge pixel set.

[0057] Perform a Gaussian filter on the initial binary image to remove high-frequency noise. Gaussian filtering is a linear smoothing filter implemented by convolving the image with a two-dimensional Gaussian kernel. Assume a 3x3 Gaussian kernel with coefficients of [[1, 2, 1], [2, 4, 2], [1, 2, 1]] / 16 is used. This Gaussian kernel is convolved with each pixel of the initial binary image to produce a smoothed binary image.

[0058] The Canny edge detection algorithm is used to extract edge pixels from the smoothed binary image. The Canny edge detection algorithm includes the following steps: first, Gaussian smoothing is performed. Since Gaussian filtering has been performed before, it can be omitted here; then the gradient magnitude and direction of the image are calculated. The gradient of the image in the x and y directions are calculated respectively by the Sobel operator. The gradient magnitude is then calculated according to the square root of the formula gradient magnitude = (x-direction gradient squared + y-direction gradient squared), and the gradient direction is calculated according to the formula gradient direction = arctan (y-direction gradient / x-direction gradient); then non-maximum suppression is performed. In the gradient direction, the gradient magnitude of each pixel is compared with the gradient magnitude of the adjacent pixels. If the gradient magnitude of the pixel is not a local maximum, its value is set to 0; finally, dual threshold detection and edge connection are performed. Two thresholds are set (assuming the low threshold is 50 and the high threshold is 150). If the gradient magnitude of a pixel is greater than the high threshold, it is marked as a strong edge point; if the gradient magnitude of a pixel is greater than the low threshold and is adjacent to a strong edge point, it is marked as a weak edge point. The initial edge pixel set is obtained by connecting the strong edge points and the weak edge points. For example, in a smoothed binary image, if the gradient amplitude of a pixel is 180, which is greater than the high threshold of 150, it will be marked as a strong edge point; if the gradient amplitude of another pixel is 80, which is greater than the low threshold of 50 and is adjacent to a strong edge point, it will be marked as a weak edge point.

[0059] Step S1233: performing a morphological gradient operation on the initial edge pixel set, enhancing edge continuity by calculating the pixel difference between the dilation operation and the erosion operation, and generating an edge-enhanced image.

[0060] In this embodiment, a dilation operation is first performed, using a 3x3 structuring element to dilate the initial edge pixel set. The center of the structuring element is sequentially aligned with each pixel in the initial edge pixel set. If a pixel within the area covered by the structuring element belongs to the initial edge pixel set, the pixel corresponding to the center of the structuring element is marked as an edge pixel. An erosion operation is then performed, also using the 3x3 structuring element, with the center of the structuring element sequentially aligned with each pixel in the initial edge pixel set. If all pixels within the area covered by the structuring element belong to the initial edge pixel set, the pixel corresponding to the center of the structuring element is retained as an edge pixel. The difference between the pixel set obtained by the dilation operation and the pixel set obtained by the erosion operation is the morphological gradient. This method enhances edge continuity and generates an edge-enhanced image. For example, in the dilation operation, a pixel that was not originally in the edge pixel set is marked as an edge pixel after the dilation operation because its neighboring pixels do belong to the edge pixel set. In contrast, in the erosion operation, some edge pixels are removed because their neighboring pixels do not all belong to the edge pixel set. By calculating the difference between the two, the edge is made more continuous.

[0061] Step S1234: performing contour tracking processing based on the edge-enhanced image, traversing adjacent edge pixels in a clockwise direction to generate a closed contour point set, and eliminating non-closed contour point sets whose pixel number is less than a preset contour point number threshold.

[0062] In this embodiment, an unvisited edge pixel point can be selected from the edge-enhanced image as the starting point, and then its adjacent edge pixels can be traversed in a clockwise direction. For example, starting from the starting point, the pixel point above it is first checked. If the pixel point is an edge pixel point and has not been visited, it is used as the next point to continue traversal; if the pixel point above is not an edge pixel point, the pixel point to the upper right is checked, and so on. By continuously traversing adjacent edge pixels until returning to the starting point, a closed contour point set is formed.

[0063] During the traversal process, the number of pixels in each closed contour point set is recorded. A threshold for the number of contour points is preset (assuming it is 20). If a closed contour point set has fewer than 20 pixels, it is removed. For example, if a closed contour point set contains only 15 pixels, which is less than the preset threshold of 20, the closed contour point set is removed from the result.

[0064] Step S1235: Execute the Douglas-Peucker polygon approximation algorithm on the retained closed contour point set, calculate the maximum vertical distance error between each closed contour point set and the approximation polygon, and when the maximum vertical distance error is less than a preset shape error threshold, extract the vertex coordinates of the approximation polygon as a candidate vertex coordinate set.

[0065] In this embodiment, the basic idea of ​​the Douglas-Peucker polygon approximation algorithm is to first connect the starting and ending points of the closed contour point set to form a straight line. The algorithm then calculates the perpendicular distances from all points in the closed contour point set to this straight line and finds the point with the largest perpendicular distance. If this maximum perpendicular distance is greater than a preset shape error threshold (assuming it's 3 pixels), this point is used as a new segmentation point, and the closed contour point set is divided into two parts. The above process is repeated for each part. If the maximum perpendicular distance is less than or equal to the preset shape error threshold, the closed contour point set in that part is approximated by the straight line connecting the starting and ending points.

[0066] For example, for a closed contour point set, the starting point is A and the end point is B. The vertical distances from all points in the point set to the straight line AB are calculated. Assuming that the point corresponding to the maximum vertical distance is C, and the vertical distance is 4 pixels, which is greater than the preset shape error threshold of 3 pixels, point C is used as the segmentation point, and the closed contour point set is divided into AC segment and CB segment, and the two segments are processed separately.

[0067] Through continuous iteration, the approximate polygon is finally obtained. The maximum vertical distance error between each closed contour point set and the approximate polygon is calculated. If the error is less than the preset shape error threshold, the vertex coordinates of the approximate polygon are extracted as the candidate vertex coordinate set.

[0068] Step S1236: Calculate the vertex sampling point spacing based on the candidate vertex coordinate set, count the pixel spacing between adjacent vertices in the image coordinate system, convert the pixel spacing into a proportional distance based on the actual construction equipment size, eliminate redundant vertices whose proportional distance is less than the preset minimum spacing threshold, and generate an optimized vertex coordinate set.

[0069] In this embodiment, for adjacent vertices in the candidate vertex coordinate set, the pixel distance between them in the image coordinate system is calculated. For example, if the coordinates of adjacent vertex A are (x1, y1) and the coordinates of vertex B are (x2, y2), then their pixel distance is the square root of ((x2-x1)^2+(y2-y1)^2).

[0070] Convert pixel spacing to a proportional distance based on the actual size of the construction equipment. Measure the actual size of the construction equipment in advance. For example, if the actual length of an excavator is 5 meters and the pixel length corresponding to the excavator in the image is 500 pixels, the ratio of pixels to actual size is 5 meters / 500 pixels = 0.01 meters / pixel. Multiply the pixel spacing between adjacent vertices by this ratio to obtain the proportional distance based on the actual size of the construction equipment.

[0071] A minimum spacing threshold is preset (assuming it is 0.1 meters). If the proportional distance between adjacent vertices is less than 0.1 meters, the vertex is considered redundant and is removed. For example, if the pixel spacing between adjacent vertices is 8 pixels, the corresponding proportional distance is 8 pixels * 0.01 meters / pixel = 0.08 meters. If it is less than the preset minimum spacing threshold of 0.1 meters, the vertex is removed from the candidate vertex coordinate set and the optimized vertex coordinate set is generated.

[0072] Step S1237: Normalize the optimized vertex coordinate set, convert the vertex coordinates into a normalized coordinate set based on the actual space coordinate system according to the ratio coefficient between the image resolution and the actual size of the construction equipment, and generate a polygon vertex coordinate set of the enhanced contour.

[0073] In this embodiment, the conversion can be performed based on the scale factor between the image resolution and the actual size of the construction equipment. Assuming the image resolution is 1920x1080 pixels, the actual length of the construction equipment is 5 meters, and the width is 2 meters. In the image, the corresponding pixel length of the construction equipment is 500 pixels, and the width is 200 pixels. Therefore, the scale factor in the length direction is 5 meters / 500 pixels = 0.01 meters / pixel, and the scale factor in the width direction is 2 meters / 200 pixels = 0.01 meters / pixel.

[0074] For each vertex coordinate in the optimized vertex coordinate set, multiply its horizontal coordinate by the scale factor in the length direction and its vertical coordinate by the scale factor in the width direction to convert the vertex coordinate to a coordinate based on the real-world coordinate system. For example, if the image coordinates of a vertex are (200, 150) pixels, the converted real-world coordinates are (200 pixels * 0.01 meters / pixel, 150 pixels * 0.01 meters / pixel) = (2 meters, 1.5 meters).

[0075] Through such normalization processing, the vertex coordinates are converted into a normalized coordinate set based on the actual space coordinate system, and a polygon vertex coordinate set of the enhanced contour is generated.

[0076] Shape matching is performed based on the polygon vertex coordinate set of the generated enhanced contour and the pre-stored equipment template library. The pre-stored equipment template library contains standard polygon vertex coordinate sets of various construction equipment, as well as the corresponding equipment type and operation status information. A shape matching algorithm (such as a matching algorithm based on Hausdorff distance) is used to calculate the similarity between the polygon vertex coordinate set of the enhanced contour and each template in the template library. Hausdorff distance refers to the maximum one-way distance between two point sets, that is, the maximum value of the minimum distance from each point in one point set to another point set. For example, the Hausdorff distance between the polygon vertex coordinate set A of the enhanced contour and template B in the template library is calculated. If the distance is less than the preset similarity threshold (assuming it is 0.2 meters), the two are considered to match, and the type of construction equipment and the corresponding operation status are determined based on the information corresponding to template B.

[0077] Step S124: performing feature splicing on the pixel area motion trajectory features, the personnel wear identification features, and the construction equipment type at the same timestamp to generate a spatiotemporal feature unit, and arranging all the spatiotemporal feature units in chronological order to obtain a spatiotemporal feature sequence.

[0078] In this embodiment, pixel-area motion trajectory features include independent motion trajectory segments and their corresponding trajectory bounding box coordinates. Personnel clothing identification features include information such as whether they are wearing a helmet, reflective clothing, and their job type. Construction equipment types include specific types such as excavators and loaders. These features are then concatenated in a predefined order, for example, concatenating the trajectory bounding box coordinates first, then the personnel clothing identification features, and finally the construction equipment type.

[0079] For example, at a certain moment, the coordinates of a construction worker's trajectory bounding box are [(100, 200), (150, 250), (200, 200), (150, 150)], the worker's wearing identification features are "red safety helmet, yellow reflective clothing, work type number 007", and the construction equipment type is "excavator". These features are spliced ​​together to form a spatiotemporal feature unit.

[0080] Arrange all spatiotemporal feature units in chronological order, and arrange the spatiotemporal feature units corresponding to each timestamp in sequence to obtain a spatiotemporal feature sequence. For example, arrange the spatiotemporal feature units at different times in order from early to late to form a sequence containing multiple spatiotemporal feature units.

[0081] Step S130: performing dynamic feature extraction processing on the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set, wherein the dynamic spatiotemporal feature set includes construction personnel movement path features, construction equipment operation frequency features, and construction surface operation status features.

[0082] Construct a dynamic feature extraction model that includes a convolutional neural network, a recurrent neural network, and a graph convolutional network. Input the spatiotemporal feature sequence into the dynamic feature extraction model for dynamic feature extraction processing. The specific steps are as follows:

[0083] Step S131: construct a dynamic feature extraction model including a convolutional neural network, a recurrent neural network and a graph convolutional network, and input the spatiotemporal feature sequence into the dynamic feature extraction model.

[0084] In this embodiment, the dynamic feature extraction model consists of a convolutional neural network (CNN), a recurrent neural network (RNN), and a graph convolutional network (GCN). The CNN is used to extract local features of each spatiotemporal feature unit in the spatiotemporal feature sequence, the RNN is used to capture the feature dependencies between consecutive timestamps, and the GCN is used to model the spatial relationships between the motion trajectory features of the pixel area.

[0085] The spatiotemporal feature sequence is input into the dynamic feature extraction model to start the feature extraction process. For example, the spatiotemporal feature sequence containing multiple spatiotemporal feature units is sequentially input into the input layer of the convolutional neural network.

[0086] Step S132: Perform local feature convolution operation on each spatiotemporal feature unit through the convolutional neural network to extract the texture change gradient features and instrument operation action segment features of the construction surface working area.

[0087] In this embodiment, the convolutional neural network includes multiple convolutional layers and pooling layers. In the convolutional layers, a convolution kernel is used to perform a local feature convolution operation on each spatiotemporal feature unit. The convolution kernel is a small matrix, such as a 3x3 matrix. The convolution kernel is slid over the spatiotemporal feature unit, and a weighted sum is taken of the area covered by the convolution kernel to obtain a new eigenvalue. By continuously sliding the convolution kernel, a new set of feature maps is obtained.

[0088] For example, for a spatiotemporal feature unit, a 3x3 convolution kernel is used for convolution, and the coefficients of the convolution kernel are [[1, 0, -1], [2, 0, -2], [1, 0, -1]]. The center of the convolution kernel is aligned with each pixel of the spatiotemporal feature unit in turn, and a weighted sum is taken of the area covered by the convolution kernel. Assuming that the pixel values ​​of the area covered by the convolution kernel are [[10, 20, 30], [40, 50, 60], [70, 80, 90]], the convolution result is 1*10+0*20+(-1)*30+2*40+0*50+(-2)*60+1*70+0*80+(-1)*90=-10.

[0089] By alternating multiple convolutional and pooling layers, we extract the texture gradient features of the construction work area and the characteristics of the machine operation action segments. Pooling layers are used to reduce the dimensionality of feature maps. Common pooling methods include max pooling and average pooling. For example, using a 2x2 max pooling kernel, we divide the feature map into multiple 2x2 regions. The maximum value of each region is taken as the new feature value to obtain the pooled feature map.

[0090] Step S133: The texture change gradient features of the continuous timestamps and the equipment operation action segment features are subjected to temporal dependency modeling by the recurrent neural network to generate construction equipment operation frequency features and construction surface operation status features.

[0091] In this embodiment, a recurrent neural network (such as a bidirectional long short-term memory unit, Bi-LSTM) can capture the feature dependencies of consecutive timestamps.

[0092] Bidirectional long short-term memory (LSTM) units consist of memory units in both the forward and backward directions. At each time step, the forward memory unit updates the current hidden state based on the current input and the hidden state from the previous time step. The backward memory unit updates the current hidden state based on the current input and the hidden state from the next time step. By concatenating the forward and backward hidden states, a richer feature representation is obtained.

[0093] For example, at time step t, the forward memory unit updates the current hidden state based on the input texture change gradient features, the instrument operation action segment features, and the hidden state at time step t-1. The backward memory unit updates the current hidden state based on the input and the hidden state at time step t+1. After concatenating the forward and backward hidden states, they pass through a fully connected layer and activation function to output the predicted features of the instrument operation frequency and the construction surface operation status at that time step.

[0094] By processing the entire time series, the construction equipment operation frequency characteristics and construction surface operation status characteristics are generated. The construction equipment operation frequency characteristics can be obtained by counting the equipment operation frequency prediction characteristics at each time step, and the construction surface operation status characteristics can be obtained by classifying the construction surface operation status prediction characteristics at each time step.

[0095] Step S134: The spatial relationship of the motion trajectory features of the pixel area is modeled through the graph convolutional network, and a construction movement graph structure is constructed with the centroid of the trajectory bounding box as the node and the distance between adjacent trajectories as the edge weight. A speed-direction joint analysis is performed on the construction movement graph structure, and the displacement vector modulus of each construction worker per unit time and the angular covariance of the adjacent worker trajectories are calculated to generate the construction worker movement path features.

[0096] In this embodiment, the construction movement graph structure is first constructed by taking the centroids of the trajectory bounding boxes as nodes and calculating the distances between the centroids of adjacent trajectory bounding boxes as edge weights. For example, for the centroids of two trajectory bounding boxes A and B, the Euclidean distance between them is calculated as the edge weight.

[0097] A joint speed-direction analysis is performed on the construction movement graph structure. The displacement vector modulus per unit time is calculated for each construction worker. The displacement vector modulus can be obtained by taking the square root of the sum of the squared differences in the centroid coordinates of the trajectory bounding box at adjacent timestamps. For example, at timestamp t, the centroid coordinates of a construction worker's trajectory bounding box are (x1, y1), and at timestamp t+1, the centroid coordinates are (x2, y2). Therefore, the displacement vector modulus is the square root of ((x2-x1)^2+(y2-y1)^2).

[0098] Calculate the angular covariance of adjacent worker trajectories. Angular covariance reflects the directional correlation between adjacent worker trajectories. This is measured by calculating the cosine of the angle between the displacement vectors of adjacent workers. For example, for two adjacent construction workers, whose displacement vectors are vector A and vector B, the cosine of the angle is calculated by dividing the dot product of vectors A and B by the product of their moduli.

[0099] By analyzing the structure of the construction movement graph, the construction worker movement path features are generated. The construction worker movement path features include the displacement vector modulus of each construction worker and the angular covariance of adjacent worker trajectories.

[0100] Step S135: Aggregate the construction personnel movement path features, the construction equipment operation frequency features, and the construction surface operation status features to generate a dynamic spatiotemporal feature set.

[0101] In this embodiment, these features can be combined by splicing to form a higher-dimensional feature vector. For example, the construction worker movement path feature is a vector containing 10 elements, the construction equipment operation frequency feature is a vector containing 5 elements, and the construction surface operation status feature is a vector containing 3 elements. These can be spliced ​​together to form a dynamic spatiotemporal feature set containing 10 + 5 + 3 = 18 elements.

[0102] Step S140: Execute construction personnel statistics processing based on the dynamic spatiotemporal feature set to generate statistics on the number of personnel entering the cave, personnel distribution statistics at each construction site, and personnel identity matching results. The personnel identity matching results are determined by comparing the identification features worn by the personnel with identification templates in the pre-registered identity database.

[0103] Execute construction personnel statistics processing based on dynamic spatiotemporal feature sets. The specific steps are as follows:

[0104] Step S141: Determine the movement direction of the construction workers based on the displacement vector modulus in the construction workers' movement path characteristics and the angular covariance of the adjacent personnel trajectories. When the displacement vector modulus is greater than the second threshold and the movement direction corresponding to the angular covariance points to the tunnel entrance, it is marked as a tunnel entry behavior. The number of tunnel entry behaviors per unit time is accumulated to generate a statistical result of the number of people entering the tunnel.

[0105] In this embodiment, the displacement vector modulus represents the distance traveled by the construction workers in unit time, and the angular covariance reflects the directional correlation between the trajectories of adjacent workers.

[0106] A second threshold (assuming it's 2 m / s) is preset. When the displacement vector modulus is greater than 2 m / s and the angular covariance indicates a direction of movement toward the tunnel entrance, the activity is marked as a tunnel entry. For example, if a construction worker's displacement vector modulus is 2.5 m / s and, based on analysis of the angular covariance of adjacent worker trajectories, their movement is determined to be toward the tunnel entrance, the worker's movement is marked as a tunnel entry.

[0107] The number of cave entries is accumulated within a unit of time (assuming 1 hour) to generate a count of the number of people entering the cave. For example, if 10 cave entries are counted within 1 hour, the count would be 10 people.

[0108] Step S142: Based on a predefined three-dimensional coordinate model of the tunnel construction site, the coordinates of the trajectory bounding box of each construction worker are mapped to the three-dimensional coordinate model, the frequency of occurrence of the trajectory bounding box within the coordinate interval of each construction site is counted, and the personnel distribution statistics of each construction site are generated.

[0109] The predefined 3D coordinate model of tunnel construction sites is built based on the actual tunnel structure and construction plan. It precisely defines the coordinate ranges of each construction site in 3D space. For example, the tunnel is divided into different sections, such as the portal construction area, the face construction area, and the lining construction area, each of which has a clear 3D coordinate range.

[0110] For each construction worker's trajectory bounding box coordinates, it is necessary to map them from the image coordinate system to the three-dimensional coordinate system of the tunnel construction site three-dimensional coordinate model. This mapping process is based on the conversion relationship between the image coordinate system and the global coordinate system established in step S1213, and is further converted in combination with the coordinate reference of the three-dimensional coordinate model of the tunnel construction site. Assuming that the centroid coordinates of a construction worker's trajectory bounding box in the image coordinate system are (x_img, y_img), they are first converted to coordinates in the global coordinate system (x_global, y_global) through the transformation matrix. Then, based on the origin position and coordinate axis direction of the three-dimensional coordinate model of the tunnel construction site, the global coordinates are converted to coordinates in the three-dimensional coordinate model (x_3d, y_3d, z_3d).

[0111] Next, count the frequency of occurrence of the trajectory bounding box within the coordinate interval of each construction site. For each construction worker's trajectory bounding box, determine whether the three-dimensional coordinates of its center of mass fall within the coordinate interval of a certain construction site. If it falls within a certain interval, the frequency of occurrence of the trajectory bounding box of the construction site is increased by 1. For example, within a period of time, the center of mass coordinates of the trajectory bounding boxes of 5 construction workers fall within the coordinate interval of the face construction area, then the frequency of occurrence of the trajectory bounding box of the face construction area is 5 times. By performing such statistics on the trajectory bounding boxes of all construction workers, the personnel distribution statistics of each construction site can be generated. For example, the personnel distribution statistics of the portal construction area are 3 people, 5 people in the face construction area, 2 people in the lining construction area, etc.

[0112] Step S143: perform optical character recognition on the job number area in the identification feature worn by the person, extract the job number string, perform fuzzy matching on the job number string with the pre-registered identity database, and generate a person identity matching result when the matching similarity exceeds a third threshold.

[0113] In this embodiment, the identification features of the wearer's clothing include information about the work type number area. Optical character recognition (OCR) is performed on this area to extract the work type number string. First, the image portion containing the work type number area undergoes preprocessing, including grayscale conversion, binarization, and denoising, to improve character recognition accuracy. For example, a grayscale conversion method similar to that in step S1211 is used to convert the color image into a grayscale image. Binarization is then performed using a preset fixed grayscale threshold to remove noise interference from the image.

[0114] Then, a mature OCR algorithm is used to perform character recognition on the preprocessed image. For example, a deep learning-based OCR model, trained on a large number of character samples, can accurately recognize characters in images. For example, if the work number on a wearable identification badge is "007," this character string can be successfully extracted after OCR processing.

[0115] The extracted work type number string is fuzzy matched against the pre-registered identity database. The pre-registered identity database stores the identity information of all construction workers, including work type number, name, position, etc. Fuzzy matching is used to address possible recognition errors, such as individual character errors in OCR recognition results. A string similarity algorithm (such as the edit distance algorithm) is used to calculate the similarity between the extracted work type number string and each work type number in the database. The edit distance is the minimum number of editing operations (insertion, deletion, substitution) required to convert one string into another. Similarity can be calculated by subtracting the ratio of the edit distance to the length of the longer string from 1.

[0116] A third threshold (assuming it's 0.8) is preset. When the match similarity exceeds 0.8, a matching person identity is considered found. For example, if the extracted job number string is "007," and a record with that same job number is found in the pre-registered identity database, the match similarity is 1, exceeding the third threshold of 0.8. Based on this record, a person identity match result is generated, including the construction worker's name, position, and other detailed information.

[0117] Step S150: Perform work efficiency comparison analysis based on the statistical results of the number of people entering the cave, the statistical results of the personnel distribution and the preset construction progress plan, generate a construction progress deviation index and a resource allocation optimization strategy, and trigger a construction early warning instruction when it is detected that the construction progress deviation index exceeds a first threshold and output the resource allocation optimization strategy to the management terminal.

[0118] Based on the statistical results of the number of personnel entering the tunnel, the statistical results of personnel distribution, and the preset construction schedule, the work efficiency comparison analysis is carried out. The specific steps are as follows:

[0119] Step S151: Obtain a target staffing table for the current construction phase from the preset construction schedule, wherein the target staffing table includes theoretical staffing numbers and work type combination requirements for each construction site.

[0120] In this embodiment, the preset construction schedule is a detailed plan developed based on the overall planning and time schedule of tunnel construction, which includes the specific requirements for each construction phase. The target staffing table for the current construction phase is obtained from this plan. For example, if the current construction phase is the initial support phase of the tunnel, the target staffing table stipulates that the theoretical number of personnel in the portal construction area is 4, and the work type combination requirements are 2 blasters, 1 support worker, and 1 safety officer; the theoretical number of personnel in the face construction area is 6, and the work type combination requirements are 3 excavators, 2 slag workers, and 1 surveyor, etc.

[0121] Step S152: Calculate the absolute difference between the actual number of personnel in the personnel distribution statistics of each construction site and the theoretical number of personnel to generate a personnel number deviation index.

[0122] In this embodiment, for each construction site, the absolute difference between the actual number of personnel and the theoretical number of personnel is calculated. For example, the actual number of personnel in the tunnel construction area is 3 people, and the theoretical number of personnel is 4 people. Then the personnel number deviation of this site is |3-4|=1 person; the actual number of personnel in the face construction area is 5 people, and the theoretical number of personnel is 6 people. The personnel number deviation is |5-6|=1 person. The personnel number deviation of each construction site is summarized to obtain the personnel number deviation index. Assuming that there are 3 construction sites in the tunnel, and the personnel number deviations are 1 person, 1 person, and 0 people respectively, then the personnel number deviation index is 1+1+0=2 people.

[0123] Step S153: Compare the job type combinations in the actual number of personnel with the job type combination requirements, count missing job types and redundant job types, and generate a job type matching deviation index.

[0124] In this embodiment, the job type combinations in the actual number of personnel at each construction site are compared with the job type combination requirements in the target personnel configuration table. For each construction site, the actual job type and number are checked to see if they meet the requirements. For example, in the tunnel entrance construction area, the target job type combination requires 2 blasters, 1 support worker, and 1 safety officer, but the actual personnel only have 1 blaster, 1 support worker, and 1 safety officer. In this case, the missing job type is 1 blaster; if the actual personnel include 3 blasters, 1 support worker, and 1 safety officer, then the redundant job type is 1 blaster.

[0125] Count the missing and redundant work types across all construction sites to generate a work type matching deviation index. The number of missing and redundant work types can be summed up. For example, if there are 2 missing work types and 1 redundant work type in the entire tunnel construction, the work type matching deviation index is 2 + 1 = 3.

[0126] Step S154: Calculate the construction progress deviation index based on the personnel quantity deviation index and the work type matching deviation index. The calculation formula of the construction progress deviation index is: deviation index = α×personnel quantity deviation + β×work type matching deviation, where α and β are weight coefficients dynamically adjusted according to the construction stage.

[0127] In this embodiment, the construction progress deviation index is calculated based on the personnel quantity deviation index and the work type matching deviation index. The weight coefficients α and β are dynamically adjusted according to the construction stage. Different construction stages have different requirements for personnel quantity and work type matching, so the weight coefficients will also change accordingly. For example, in the initial tunnel support stage, the adequacy of personnel quantity has a significant impact on the construction progress. In this case, α can be set to 0.6 and β to 0.4. In the later stage of tunnel decoration, the rationality of the work type matching is more critical. In this case, α can be set to 0.4 and β to 0.6.

[0128] Assuming that in the current construction stage α=0.6, β=0.4, the personnel quantity deviation index is 2 people, and the work type matching deviation index is 3, then the construction progress deviation index = 0.6×2+0.4×3=2.4.

[0129] Step S155: When the construction progress deviation index exceeds a first threshold, a resource allocation optimization strategy including personnel transfer suggestions and recruitment needs is generated based on the missing job types and the redundant job types.

[0130] A threshold of 2 is preset. When the construction progress deviation index exceeds 2, it indicates that there may be a significant deviation in the construction progress, and resource allocation optimization is needed. Based on the statistically obtained missing and redundant job types, a resource allocation optimization strategy is generated, including personnel transfer recommendations and recruitment needs.

[0131] For example, if the statistical results show that in a certain construction phase, the face construction area is short of two tunneling workers, while the portal construction area has three redundant support workers, then the resource allocation optimization strategy can recommend transferring the two support workers from the portal construction area to the face construction area for tunneling training before taking up their posts; if the missing jobs cannot be allocated among the existing personnel, recruitment needs will be proposed based on the type and number of missing jobs, such as recruiting two tunneling workers.

[0132] When the construction progress deviation indicator is detected to exceed a first threshold, a construction warning command is triggered. This warning command can be an audible or visual alarm signal or an automatic message notification sent by the system. Simultaneously, the generated resource allocation optimization strategy is output to a management terminal, such as a computer or mobile phone, allowing management personnel to promptly monitor construction progress deviations and take appropriate measures. For example, after receiving the construction warning command and resource allocation optimization strategy, management personnel can make staff transfer arrangements and formulate recruitment plans based on the strategy to ensure that tunnel construction proceeds smoothly according to the planned schedule.

[0133] Step S210: The training method of the dynamic feature extraction model includes obtaining a training data set containing a spatiotemporal feature sequence and a corresponding annotation data set, wherein the annotation data set includes construction equipment operation frequency annotations, construction surface operation status annotations, and construction personnel movement path annotations.

[0134] In this embodiment, the training dataset can be obtained by collecting video data from multiple tunnel construction scenes and performing a processing similar to steps S110-S120 to obtain a spatiotemporal feature sequence. For example, multiple cameras can be installed at different tunnel construction sites to collect video data, and then the collected video data can be subjected to spatiotemporal feature fusion processing to generate a spatiotemporal feature sequence.

[0135] The corresponding labeled data sets need to be manually labeled. For the labeling of the operation frequency of construction equipment, it is necessary to count the number of times each construction equipment is operated in the video and record it. For example, in a video, the excavator is operated 10 times and the loader is operated 8 times. For the labeling of the construction surface operation status, classification labeling is performed according to the actual operation conditions of the construction surface, such as normal operation, shutdown for maintenance, waiting for materials, etc. For the labeling of the movement path of construction personnel, the movement trajectory of each construction personnel is manually tracked and their location information at different timestamps is recorded.

[0136] Step S220: Perform multi-channel feature normalization processing on the spatiotemporal feature sequence, convert the pixel area motion trajectory features, personnel wear identification features and construction equipment contour features into normalized feature vectors with zero mean and unit variance, ensuring that the dimensional units of different feature dimensions are consistent.

[0137] In this embodiment, standardization operations are performed on the pixel region motion trajectory features, personnel wear identification features, and construction equipment outline features. Taking the pixel region motion trajectory feature as an example, the mean and standard deviation of this feature in the training data set are first calculated. Assuming that the pixel region motion trajectory feature is a vector containing multiple coordinate values, the mean and standard deviation of the coordinate values ​​corresponding to all vectors are calculated. For example, for the horizontal coordinate of the trajectory bounding box, the mean and standard deviation of the horizontal coordinates of all trajectory bounding boxes are calculated.

[0138] Next, the mean of each coordinate value in each vector is subtracted and divided by the standard deviation to obtain a normalized feature vector with zero mean and unit variance. The same method is used to normalize the features of personnel identification and the outline of construction equipment. For example, a personnel identification feature might be a vector containing information such as the color of a helmet or the type of reflective clothing, while a construction equipment outline feature might be a vector containing the coordinates of polygon vertices. These features are standardized separately to ensure that the units of the different feature dimensions are consistent, allowing the model to better learn these features.

[0139] Step S230: Initialize the convolution kernel weight matrix of the convolutional neural network, the hidden state transfer matrix of the recurrent neural network, and the adjacency matrix update function of the graph convolutional network, wherein the convolution kernel size matches the spatial size of the pixel area of ​​the spatiotemporal feature unit, the hidden state dimension matches the time step of the instrument operation action fragment feature, and the edge weight dimension of the adjacency matrix matches the actual physical distance unit of the center of mass spacing of the trajectory bounding box.

[0140] In this embodiment, for the convolutional neural network, the convolution kernel size needs to match the spatial size of the pixel area of ​​the spatiotemporal feature unit. For example, if the pixel area of ​​the spatiotemporal feature unit is a 32x32 image area, the convolution kernel size can be set to an appropriate size such as 3x3 or 5x5 to ensure that local features can be effectively extracted. The initial value of the convolution kernel weight matrix can be randomly initialized, for example, by random sampling from a normal distribution.

[0141] For recurrent neural networks, the hidden state dimension needs to match the time step length of the machine operation action segment features. Assuming the time step length of the machine operation action segment features is 10, the hidden state dimension can be set to 100 to capture sufficient temporal dependency information. The initial value of the hidden state transition matrix can also be randomly initialized.

[0142] For graph convolutional networks, the edge weight dimension of the adjacency matrix needs to match the actual physical distance units between the center-of-mass distances of the trajectory bounding boxes. For example, if the actual physical distance units between the center-of-mass distances of the trajectory bounding boxes are meters, then the edge weights of the adjacency matrix should also be initialized in meters. The adjacency matrix update function can take the form of some common functions, such as linear functions or nonlinear activation functions, and the initial parameters can be randomly initialized.

[0143] Step S240: Slice the standardized spatiotemporal feature sequence according to the time window to generate a time segment sequence, each time segment sequence containing a fixed number of continuous spatiotemporal feature units.

[0144] In this embodiment, a preset time window size (assuming it's 5) means that each time segment sequence contains 5 consecutive spatiotemporal feature units. Starting from the start of the spatiotemporal feature sequence, 5 consecutive spatiotemporal feature units are sequentially selected to form a time segment sequence. Then, moving backward one time step, 5 consecutive spatiotemporal feature units are selected to form the next time segment sequence, and this continues until the entire spatiotemporal feature sequence has been traversed. For example, if a spatiotemporal feature sequence contains 100 spatiotemporal feature units and is sliced ​​using a time window size of 5, 100 - 5 + 1 = 96 time segment sequences can be obtained.

[0145] Step S250: Input the time segment sequence into the convolutional neural network, and extract the local texture feature map through the alternating operation of multiple convolutional layers and pooling layers. The local texture feature map includes the texture change gradient characteristics of the construction surface working area and the equipment operation action segment characteristics.

[0146] In this embodiment, the convolution layer uses an initialized convolution kernel weight matrix to perform a convolution operation on each spatiotemporal feature unit in the time segment sequence. For example, a 3x3 convolution kernel is used to convolve the spatiotemporal feature unit. The convolution kernel coefficients are continuously weighted and summed with the pixel values ​​of the spatiotemporal feature unit to obtain new feature values. Through the operation of multiple convolution layers, higher-level local features are gradually extracted.

[0147] In the pooling layer, maximum pooling or average pooling is used to reduce the dimensionality of the feature map output by the convolutional layer. For example, a 2x2 maximum pooling kernel is used to divide the feature map into multiple 2x2 regions. The maximum value of each region is taken as the new feature value to obtain the pooled feature map. By alternating multiple convolutional and pooling layers, a local texture feature map is extracted. This feature map contains the texture gradient characteristics of the construction surface working area and the characteristics of the machine operation action segments.

[0148] Step S260: Input the local texture feature map into the recurrent neural network in chronological order, capture the feature dependency of adjacent time slices through the bidirectional long short-term memory unit, and output the equipment operation frequency prediction feature and construction surface operation status prediction feature of each time slice.

[0149] In this embodiment, local texture feature maps are input into the bidirectional long short-term memory (Bi-LSTM) unit of a recurrent neural network in chronological order. At each time step, the forward memory unit of the Bi-LSTM updates the current hidden state based on the current input local texture feature map and the hidden state of the previous time step. The backward memory unit updates the current hidden state based on the current input and the hidden state of the next time step. By concatenating the forward and backward hidden states, a richer feature representation is obtained.

[0150] After processing through the fully connected layer and activation function, the output is the predicted features for the machine operation frequency and the construction surface working status for each time slice. For example, at time step t, the Bi-LSTM outputs a vector, one part of which represents the predicted features for the machine operation frequency, and the other part represents the predicted features for the construction surface working status. By processing the entire time series, the predicted features for each time slice are obtained.

[0151] Step S270: Input the pixel area motion trajectory features in the spatiotemporal feature sequence into the graph convolutional network, construct a dynamic adjacency matrix based on the centroid coordinates of the trajectory bounding box, and generate the motion path prediction features with enhanced spatial relationships through node feature aggregation and edge weight update.

[0152] In this embodiment, a dynamic adjacency matrix is ​​constructed based on the coordinates of the trajectory bounding box centroids. The elements of the adjacency matrix represent the connection relationship between nodes (trajectory bounding box centroids) and edge weights (the distance between trajectory bounding box centroids). For example, for two trajectory bounding box centroids A and B, if the distance between them is less than a certain threshold, they are considered to be connected, and the element value at the corresponding position in the adjacency matrix is ​​their distance; if the distance is greater than the threshold, they are considered to be unconnected, and the element value at the corresponding position in the adjacency matrix is ​​0.

[0153] In graph convolutional networks, spatially enhanced mobility path prediction features are generated through node feature aggregation and edge weight updating. Node feature aggregation involves weighted summing of features of adjacent nodes, while edge weight updating involves adjusting edge weights based on node features and connectivity. For example, a node's features can be updated using the features of its adjacent nodes and edge weights. This can be expressed as: new node features = weighted sum (adjacent node features * edge weights). By continuously aggregating node features and updating edge weights, spatially enhanced mobility path prediction features are generated.

[0154] Step S280: combining the equipment operation frequency prediction feature, the construction surface operation state prediction feature, and the movement path prediction feature to generate a dynamic spatiotemporal feature prediction set.

[0155] In this example, for example, the machine operation frequency prediction feature is a vector containing 5 elements, the construction surface work status prediction feature is a vector containing 3 elements, and the movement path prediction feature is a vector containing 10 elements. These are sequentially spliced ​​together to form a dynamic spatiotemporal feature prediction set containing 5 + 3 + 10 = 18 elements. This splicing method can integrate prediction features from different aspects, providing more comprehensive information for subsequent loss calculation and model optimization.

[0156] Step S290: Calculate the multi-task loss function between the dynamic spatiotemporal feature prediction set and the labeled data set, wherein the multi-task loss function includes the weighted sum of the mean square error of the equipment operation frequency, the cross entropy loss of the construction surface operation state, and the cosine similarity loss of the moving path direction.

[0157] Calculate the multi-task loss function between the dynamic spatiotemporal feature prediction set and the labeled data set, which specifically includes the following parts:

[0158] Step S291: Perform time dimension average pooling on the equipment operation frequency prediction features to generate a global operation frequency scalar for each construction equipment, and calculate the mean square error between the global operation frequency scalar and the actual operation frequency of the corresponding equipment in the labeled data.

[0159] In this embodiment, the machine operation frequency prediction feature is a sequence with multiple values ​​in the time dimension. For example, there are 10 time steps in a time segment sequence, and each time step has a machine operation frequency prediction value. These 10 values ​​are average pooled, that is, they are added together and divided by the number of time steps to obtain a global operation frequency scalar for each construction machine. For example, for the machine operation frequency prediction feature of an excavator, the predicted values ​​in 10 time steps are 3, 4, 3, 5, 4, 3, 4, 5, 4, and 3, respectively. These values ​​are added together to get 38, and then divided by 10 to obtain a global operation frequency scalar of 3.8.

[0160] Calculate the mean squared error (MSE) between the global operation frequency scalar and the actual operation frequency of the corresponding machine in the labeled data. Assuming the actual operation frequency of the excavator in the labeled data is 4, the MSE is calculated by first taking the difference between the two values ​​(3.8 - 4 = -0.2), then squaring the difference to obtain (-0.2)^2 = 0.04. By performing this calculation for all construction machines, we obtain the MSE of the machine operation frequency.

[0161] Step S292: Perform multi-category probability normalization on the construction surface operation state prediction features to generate the operation state probability distribution of each time slice, and calculate the cross entropy loss between the operation state probability distribution and the corresponding state label in the labeled data.

[0162] In this embodiment, the construction surface operation state prediction feature is typically a vector whose elements represent the prediction scores for different operation states. These scores are normalized using the softmax function and converted into a probability distribution. For example, the construction surface operation state has three states: normal operation, shutdown for maintenance, and waiting for materials. The construction surface operation state prediction feature vector is [2, 1, 3]. The probability distribution calculated by the softmax function is [0.2, 0.1, 0.7], indicating that the probability of normal operation is 0.2, the probability of shutdown for maintenance is 0.1, and the probability of waiting for materials is 0.7.

[0163] Calculate the cross entropy loss between the probability distribution of the work status and the corresponding state label in the annotated data. Assume that the work status label for this time slice in the annotated data is "waiting for materials," and the corresponding probability distribution is [0, 0, 1]. The cross entropy loss is calculated by multiplying the corresponding elements of the probability distribution of the annotated data with the predicted probability distribution, taking the negative logarithm, and summing them. That is, -(0*log(0.2)+0*log(0.1)+1*log(0.7))=-log(0.7). By performing this calculation for all time slices, we obtain the cross entropy loss for the work status of the construction surface.

[0164] Step S293: Decomposing the movement path prediction feature into a unit time displacement vector, extracting the displacement direction angle value, and calculating the cosine similarity between the displacement direction angle value and the corresponding path direction angle value in the annotation data.

[0165] In this embodiment, the movement path prediction feature can be represented as a series of coordinate points, and the unit time displacement vector is obtained by calculating the difference between adjacent coordinate points. For example, if the coordinates at time step t are (x1, y1) and the coordinates at time step t+1 are (x2, y2), then the unit time displacement vector is (x2-x1, y2-y1).

[0166] Extract the displacement direction angle by calculating the angle between the displacement vector and the positive x-axis. For example, if the displacement vector is (3, 4), you can use the inverse tangent function to calculate the angle, i.e., arctan(4 / 3), to obtain the displacement direction angle.

[0167] Calculate the cosine similarity between the displacement direction angle and the corresponding path direction angle in the annotated data. Assume the predicted displacement direction angle is 60 degrees, and the corresponding path direction angle in the annotated data is 50 degrees. Convert the angle value to a vector, and then calculate the cosine similarity by dividing the dot product of the two vectors by the product of their moduli. Perform this calculation for all construction workers' movement paths to obtain the cosine similarity loss for the movement path direction.

[0168] Step S294: Set the equipment operation frequency loss weight, operation status loss weight, and movement path similarity weight according to the construction monitoring task priority, and add the weighted sub-losses to generate a multi-task loss function.

[0169] In this embodiment, for example, in certain construction stages, the frequency of equipment operation has a greater impact on the construction progress, so the equipment operation frequency loss weight can be set to 0.5; accurate judgment of the operation status is also very important, and the operation status loss weight can be set to 0.3; the movement path similarity is relatively less important, and the movement path similarity weight can be set to 0.2.

[0170] The weighted sub-losses are summed to generate the multi-task loss function. Specifically, the multi-task loss function = weighted machine operation frequency loss * mean squared error of machine operation frequency + weighted operation state loss * cross entropy loss of construction surface operation state + weighted movement path similarity * direction cosine similarity loss of movement path. For example, if the mean squared error of machine operation frequency is 0.04, the cross entropy loss of construction surface operation state is 0.1, and the direction cosine similarity loss of movement path is 0.05, then the multi-task loss function calculated based on the above weights is 0.5 * 0.04 + 0.3 * 0.1 + 0.2 * 0.05 = 0.02 + 0.03 + 0.01 = 0.06.

[0171] Step S295: During the back propagation process, the local texture feature map gradient of the convolutional neural network, the state transition gradient of the recurrent neural network, and the adjacency matrix gradient of the graph convolutional network are chain-derived according to the multi-task loss function so that the dimension of each sub-loss gradient matches the corresponding network parameter update step size.

[0172] In this embodiment, chain derivation is based on the composite function derivation rule, starting from the multi-task loss function, and gradually calculating the gradient of each network layer.

[0173] For convolutional neural networks, the gradient of the local texture feature map is calculated using a multi-task loss function, and then the gradient of the convolution kernel weight matrix is ​​obtained through chain derivation. For example, if the gradient of the local texture feature map output by the multi-task loss function is [0.1, 0.2, 0.3], the gradient of the convolution kernel weight matrix of this convolution layer can be calculated through chain derivation, allowing the convolution kernel weight matrix to be updated based on the gradient.

[0174] For recurrent neural networks, the state transition gradient is calculated based on the multi-task loss function. Specifically, the gradient of the multi-task loss function with respect to the hidden state transfer matrix is ​​calculated. For example, the gradient of the multi-task loss function with respect to the hidden state at a certain time step is calculated through backpropagation. Then, based on the relationship between the hidden state and the hidden state transfer matrix, the gradient of the hidden state transfer matrix is ​​calculated.

[0175] For graph convolutional networks, the adjacency matrix gradient is calculated based on the multi-task loss function. The gradient of the adjacency matrix is ​​calculated by analyzing the relationship between the multi-task loss function and the adjacency matrix update function. For example, the adjacency matrix update function is a complex function, and chain derivation can be used to obtain the gradient of the multi-task loss function with respect to the elements of the adjacency matrix.

[0176] Through this chained derivation process, the dimensions of each sub-loss gradient match the corresponding network parameter update step size, ensuring that the model can accurately update parameters based on the gradient, thereby continuously optimizing model performance. After each backpropagation, an optimization algorithm (such as stochastic gradient descent) is used to update the convolutional neural network's convolutional kernel weight matrix, the recurrent neural network's hidden state transfer matrix, and the graph convolutional network's adjacency matrix update function parameters based on the calculated gradient until the loss function converges, meaning that the value of the multi-task loss function no longer shows a significant downward trend. At this point, model training is complete.

[0177] Figure 2 A schematic diagram illustrates exemplary hardware and software components of an AI-based video surveillance tunnel construction monitoring and analysis system 100, which can implement the concepts of the present invention, according to some embodiments of the present invention. For example, processor 120 can be used in AI-based video surveillance tunnel construction monitoring and analysis system 100 to perform the functions of the present invention.

[0178] The AI-powered video surveillance-based tunnel construction monitoring and analysis system 100 can be a general-purpose server or a special-purpose server, both of which can be used to implement the AI-powered video surveillance-based tunnel construction monitoring and analysis method of the present invention. Although only one server is shown in this invention, for convenience, the functions described in this invention can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0179] For example, the tunnel construction monitoring and analysis system 100 based on AI video surveillance may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. For example, the tunnel construction monitoring and analysis system 100 based on AI video surveillance may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present invention may be implemented based on these program instructions. The tunnel construction monitoring and analysis system 100 based on AI video surveillance may also include an I / O interface 150 between the computer and other input and output devices.

[0180] For ease of explanation, only one processor is described in the tunnel construction monitoring and analysis system 100 based on AI video surveillance. However, it should be noted that the tunnel construction monitoring and analysis system 100 based on AI video surveillance in the present invention may also include multiple processors, so the steps performed by one processor described in the present invention may also be performed jointly or individually by multiple processors. For example, if the processor of the tunnel construction monitoring and analysis system 100 based on AI video surveillance executes step A and step B, it should be understood that step A and step B may also be executed jointly by two different processors or individually in one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor execute steps A and B together.

[0181] In addition, an embodiment of the present invention also provides a readable storage medium, in which computer-executable instructions are preset. When the processor executes the computer-executable instructions, the above-mentioned tunnel construction monitoring and analysis method based on AI video monitoring is implemented.

[0182] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, multiple features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A tunnel construction monitoring and analysis method based on AI video monitoring, characterized in that: The method comprises: The original video stream data set of the construction scene is acquired in real time by a multi-view video acquisition device deployed in the tunnel construction area, wherein the original video stream data set includes a plurality of spatially continuous frame sequences with time stamps aligned; Performing spatiotemporal feature fusion processing on the original video stream data set to generate a spatiotemporal feature sequence, wherein each spatiotemporal feature unit in the spatiotemporal feature sequence is composed of pixel area motion trajectory features, personnel wear identification features, and construction equipment contour features of the corresponding frame sequence; Performing dynamic feature extraction processing on the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set, wherein the dynamic spatiotemporal feature set includes construction personnel movement path features, construction equipment operation frequency features, and construction surface operation status features; Performing construction personnel statistics processing based on the dynamic spatiotemporal feature set to generate statistics on the number of personnel entering the tunnel, personnel distribution statistics at each construction site, and personnel identity matching results, wherein the personnel identity matching results are determined by comparing identification features worn by the personnel with identification templates in a pre-registered identity database; A comparative analysis of work efficiency is performed based on the statistical results of the number of personnel entering the tunnel, the statistical results of the personnel distribution, and the preset construction progress plan, generating a construction progress deviation index and a resource allocation optimization strategy; when it is detected that the construction progress deviation index exceeds a first threshold, a construction early warning instruction is triggered and the resource allocation optimization strategy is output to the management terminal; The performing spatiotemporal feature fusion processing on the original video stream data set to generate a spatiotemporal feature sequence includes: Performing motion trajectory segmentation processing on each spatially continuous frame sequence in the original video stream data set to obtain pixel region motion trajectory features of each construction worker, wherein the pixel region motion trajectory features include independent motion trajectory segments and corresponding trajectory bounding box coordinates; Extracting a color space distribution histogram within the pixel area corresponding to the coordinates of the trajectory bounding box, and classifying the color space distribution histogram based on a pre-trained wear identification classification model to obtain wear identification features of the person, wherein the wear identification classification model is trained using a training data set labeled with safety helmets, reflective clothing, and work type numbers; Performing edge enhancement processing on the construction machine outline in the spatial continuous frame sequence, extracting a polygon vertex coordinate set of the enhanced outline, performing shape matching based on the polygon vertex coordinate set and a pre-stored machine template library to determine the type of construction machine and the corresponding operating state; The motion trajectory features of the pixel area, the identification features of the personnel wearing and the type of the construction equipment at the same time stamp are spliced ​​to generate a spatiotemporal feature unit, and all the spatiotemporal feature units are arranged in chronological order to obtain a spatiotemporal feature sequence; The step of performing dynamic feature extraction on the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set includes: Constructing a dynamic feature extraction model comprising a convolutional neural network, a recurrent neural network, and a graph convolutional network, and inputting the spatiotemporal feature sequence into the dynamic feature extraction model; Performing a local feature convolution operation on each spatiotemporal feature unit through the convolutional neural network to extract the texture change gradient features of the construction surface working area and the equipment operation action segment features; The texture change gradient features of the continuous time stamps and the equipment operation action fragment features are modeled by the recurrent neural network to generate construction equipment operation frequency features and construction surface operation status features; The spatial relationship modeling of the motion trajectory features of the pixel area is performed using the graph convolutional network, and a construction movement graph structure is constructed with the centroid of the trajectory bounding box as the node and the distance between adjacent trajectories as the edge weight. A speed-direction joint analysis is performed on the construction movement graph structure to calculate the displacement vector modulus of each construction worker per unit time and the angular covariance of the trajectories of adjacent workers, thereby generating the movement path features of the construction workers; Aggregating the construction personnel movement path features, the construction equipment operation frequency features, and the construction surface operation status features to generate a dynamic spatiotemporal feature set; The training method of the dynamic feature extraction model includes: Obtaining a training data set containing a spatiotemporal feature sequence and a corresponding annotated data set, wherein the annotated data set includes annotated construction equipment operation frequency, construction surface operation status, and construction personnel movement path; Performing multi-channel feature normalization on the spatiotemporal feature sequence, converting the pixel area motion trajectory features, personnel wear identification features, and construction equipment contour features into normalized feature vectors with zero mean and unit variance, ensuring that the dimensional units of different feature dimensions are consistent; Initializing the convolution kernel weight matrix of the convolutional neural network, the hidden state transfer matrix of the recurrent neural network, and the adjacency matrix update function of the graph convolutional network, wherein the convolution kernel size matches the pixel region spatial size of the spatiotemporal feature unit, the hidden state dimension matches the time step of the instrument operation action segment feature, and the edge weight dimension of the adjacency matrix matches the actual physical distance unit between the center of mass of the trajectory bounding box; Slice the normalized spatiotemporal feature sequence by time window to generate a time segment sequence, each of which contains a fixed number of continuous spatiotemporal feature units; Inputting the time segment sequence into the convolutional neural network, extracting a local texture feature map through alternating operations of multiple convolutional layers and pooling layers, wherein the local texture feature map includes texture change gradient features of the construction surface working area and instrument operation action segment features; The local texture feature map is input into the recurrent neural network in chronological order, and the feature dependency relationship of adjacent time slices is captured through a bidirectional long short-term memory unit, and the prediction feature of the machine operation frequency and the construction surface operation status of each time slice is output; Inputting the pixel area motion trajectory features in the spatiotemporal feature sequence into the graph convolutional network, constructing a dynamic adjacency matrix based on the centroid coordinates of the trajectory bounding box, and generating a motion path prediction feature with enhanced spatial relationship through node feature aggregation and edge weight update; Splicing the equipment operation frequency prediction feature, the construction surface operation state prediction feature, and the movement path prediction feature to generate a dynamic spatiotemporal feature prediction set; Calculating a multi-task loss function between the dynamic spatiotemporal feature prediction set and the labeled data set, the multi-task loss function including a weighted sum of the mean square error of the instrument operation frequency, the cross entropy loss of the construction surface operation state, and the directional cosine similarity loss of the movement path; Backpropagation optimization is performed based on the multi-task loss function, and the convolution kernel weight matrix of the convolutional neural network, the hidden state transfer matrix of the recurrent neural network, and the adjacency matrix update function of the graph convolutional network are synchronously updated until the loss function converges.

2. The tunnel construction monitoring and analysis method based on AI video monitoring according to claim 1 is characterized in that: The step of performing motion trajectory segmentation processing on each spatially continuous frame sequence in the original video stream data set to obtain an independent motion trajectory segment of each construction worker and the corresponding trajectory bounding box coordinates includes: Performing grayscale processing on each frame image in the spatially continuous frame sequence to convert the color image into a grayscale image, and calculating a pixel brightness difference matrix between adjacent frames based on a preset adjacent time stamp interval; Extracting a dynamic background model based on the pixel brightness difference matrix, separating a foreground moving target region by a background difference method, performing a morphological closing operation on the foreground moving target region to eliminate noise interference and fill holes inside the target, thereby generating a connected domain labeling map; Fitting a minimum bounding rectangle of each foreground moving object based on the connected domain label graph, calculating the centroid coordinates and aspect ratio of each bounding rectangle, mapping the centroid coordinates to the global coordinate system, and recording the pixel area of ​​each bounding rectangle; A multi-target tracking algorithm is used to associate the centroid coordinates of consecutive time stamps with motion trajectories. The Euclidean distance between the centroid coordinates of the same target at adjacent time stamps is calculated. When the Euclidean distance is less than a preset distance threshold and the change in the aspect ratio is less than a preset ratio threshold, it is determined to be a motion trajectory segment of the same target. Merge the motion trajectory segments of the same target at consecutive timestamps to generate independent motion trajectory segments, and calculate the trajectory bounding box coordinates based on the global coordinate sequence of the bounding rectangle. The trajectory bounding box coordinates are the minimum convex polygon vertex coordinate set containing all the vertices of the bounding rectangle; The independent motion trajectory segment is verified for trajectory integrity, and the coverage duration of the trajectory bounding box coordinates on the time axis is counted. When the coverage duration exceeds a preset duration threshold and the pixel area change rate is lower than a preset area change threshold, the independent motion trajectory segment and the corresponding trajectory bounding box coordinates are retained.

3. The tunnel construction monitoring and analysis method based on AI video monitoring according to claim 1 is characterized in that: The edge enhancement processing is performed on the construction equipment outline in the spatial continuous frame sequence, and the polygon vertex coordinate set of the enhanced outline is extracted, including: Converting each frame image of the spatial continuous frame sequence into a grayscale image, and performing binarization segmentation on the grayscale image based on a preset fixed grayscale threshold to obtain an initial binarized image; Performing Gaussian filtering on the initial binary image to eliminate high-frequency noise interference to generate a smoothed binary image, and using the Canny edge detection algorithm to extract edge pixels from the smoothed binary image to generate an initial edge pixel set; Performing a morphological gradient operation on the initial edge pixel set, enhancing edge continuity by calculating pixel differences between a dilation operation and an erosion operation, and generating an edge-enhanced image; Performing contour tracking processing based on the edge-enhanced image, traversing adjacent edge pixels in a clockwise direction to generate a closed contour point set, and eliminating non-closed contour point sets whose pixel number is less than a preset contour point number threshold; Execute the Douglas-Peucker polygon approximation algorithm on the retained closed contour point set, calculate the maximum vertical distance error between each closed contour point set and the approximation polygon, and when the maximum vertical distance error is less than a preset shape error threshold, extract the vertex coordinates of the approximation polygon as a candidate vertex coordinate set; Calculating the spacing between vertex sampling points based on the candidate vertex coordinate set, counting the pixel spacing between adjacent vertices in the image coordinate system, converting the pixel spacing into a proportional distance based on the actual size of the construction equipment, eliminating redundant vertices whose proportional distance is less than a preset minimum spacing threshold, and generating an optimized vertex coordinate set; The optimized vertex coordinate set is normalized, and the vertex coordinates are converted into a normalized coordinate set based on the actual space coordinate system according to the ratio coefficient between the image resolution and the actual size of the construction equipment, thereby generating a polygon vertex coordinate set of the enhanced contour.

4. The tunnel construction monitoring and analysis method based on AI video monitoring according to claim 1 is characterized in that: The calculating of the multi-task loss function between the dynamic spatiotemporal feature prediction set and the labeled data set includes: Performing time-dimensional average pooling on the predicted features of the machine operation frequency to generate a global operation frequency scalar for each construction machine, and calculating the mean square error between the global operation frequency scalar and the actual operation frequency of the corresponding machine in the labeled data; Performing multi-category probability normalization on the construction surface operation state prediction features to generate an operation state probability distribution for each time slice, and calculating the cross entropy loss between the operation state probability distribution and the corresponding state label in the annotated data; Decomposing the movement path prediction feature by a unit time displacement vector, extracting a displacement direction angle value, and calculating a cosine similarity between the displacement direction angle value and a corresponding path direction angle value in the annotation data; According to the priority of the construction monitoring task, the weight of the equipment operation frequency loss, the weight of the operation status loss, and the weight of the movement path similarity are set, and the weighted sub-losses are added together to generate a multi-task loss function; During the back-propagation process, the local texture feature map gradient of the convolutional neural network, the state transition gradient of the recurrent neural network, and the adjacency matrix gradient of the graph convolutional network are chain-derived according to the multi-task loss function so that the dimension of each sub-loss gradient matches the corresponding network parameter update step size.

5. The tunnel construction monitoring and analysis method based on AI video monitoring according to claim 1 is characterized in that: The construction personnel statistics processing is performed based on the dynamic spatiotemporal feature set to generate the statistical results of the number of personnel entering the tunnel, the statistical results of the personnel distribution at each construction site, and the personnel identity matching results, including: Determine the movement direction of the construction workers based on the displacement vector modulus in the construction workers' movement path characteristics and the angular covariance of the trajectories of adjacent workers. When the displacement vector modulus is greater than a second threshold and the movement direction corresponding to the angular covariance points to the tunnel entrance, mark it as a tunnel entry behavior. Accumulate the number of tunnel entry behaviors per unit time to generate a statistical result of the number of tunnel entry workers. Based on a predefined three-dimensional coordinate model of the tunnel construction site, the coordinates of the trajectory bounding box of each construction worker are mapped to the three-dimensional coordinate model, and the frequency of occurrence of the trajectory bounding box within the coordinate interval of each construction site is counted to generate the personnel distribution statistics of each construction site; Optical character recognition is performed on the job number area in the identification feature worn by the person, the job number string is extracted, the job number string is fuzzy matched with the pre-registered identity database, and a person identity matching result is generated when the matching similarity exceeds a third threshold.

6. The tunnel construction monitoring and analysis method based on AI video monitoring according to claim 1 is characterized in that: The method of performing work efficiency comparison analysis based on the statistical results of the number of personnel entering the cave, the statistical results of the personnel distribution, and the preset construction progress plan to generate a construction progress deviation index and a resource allocation optimization strategy includes: Obtaining a target staffing table for the current construction phase from the preset construction schedule, wherein the target staffing table includes theoretical staffing numbers and work type combination requirements for each construction site; Calculating the absolute difference between the actual number of personnel in the personnel distribution statistics of each construction site and the theoretical number of personnel to generate a personnel number deviation index; Comparing the job type combinations in the actual number of personnel with the job type combination requirements, counting missing job types and redundant job types, and generating a job type matching deviation index; The construction progress deviation index is calculated based on the personnel quantity deviation index and the work type matching deviation index. The calculation formula of the construction progress deviation index is: deviation index = α × personnel quantity deviation + β × work type matching deviation, where α and β are weight coefficients dynamically adjusted according to the construction stage; When the construction progress deviation index exceeds a first threshold, a resource allocation optimization strategy including personnel transfer suggestions and recruitment needs is generated based on the missing job types and the redundant job types.

7. A tunnel construction monitoring and analysis system based on AI video monitoring, characterized in that: It includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the tunnel construction monitoring and analysis method based on AI video monitoring as described in any one of claims 1 to 6 above.

Images