Component hoisting progress interference intelligent monitoring method based on dense optical flow tracking

Abstract

The invention discloses a dense optical flow tracking-based component hoisting progress interference intelligent monitoring method, which comprises the following steps of S1, instance segmentation model establishment, S2, component shielding judgment, S3, leak detection compensation, and estimation of a pixel-level dense optical flow field between adjacent frames by using dense optical flow. The method is based on a dense optical flow estimation method, the problem of missing detection possibly occurring in the instance segmentation process can be effectively corrected, and the overall detection precision and operation stability of the system are remarkably improved. Besides, a robust identity matching mechanism is designed for the situation that the prefabricated part temporarily disappears and then reappears due to shielding in the hoisting process, the identity recognition problem generated when the prefabricated part reappears due to shielding is effectively solved, the accuracy and robustness of target tracking of the prefabricated part under the complex working condition are greatly improved, and the hoisting efficiency is improved. And the interference of shielding on progress judgment is reduced.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring and control technology, specifically relating to an intelligent monitoring method for interference in component hoisting progress based on dense optical flow tracking. Background Technology

[0002] Prefabricated construction is a building method that involves assembling prefabricated components such as beams, columns, walls, and slabs from a factory to the construction site, fundamentally different from traditional on-site casting. However, during the construction of prefabricated buildings, due to the high degree of coordination required for the hoisting of prefabricated components, even minor schedule disruptions can gradually accumulate and amplify, ultimately leading to project delays and cost overruns. Therefore, effectively identifying and controlling disruptions to the hoisting schedule of prefabricated components is crucial to solving this problem.

[0003] In the prior art, Chinese patent literature discloses an intelligent construction method for tracking the hoisting progress of prefabricated components based on radio frequency tags (RFID) (CN202411589361.4). This method relies on manual scanning of RFID tags on prefabricated components within a limited area using a handheld scanning device to obtain their location and progress status information. Although RFID technology has high recognition accuracy, the need for manual intervention may lead to a long monitoring process, making it difficult to achieve real-time and dynamic monitoring of construction progress. Chinese patent literature discloses a pedestrian multi-target tracking method based on joint instance segmentation (CN202310306219.3) and a vision-based deep learning instance segmentation tracking method for off-site construction (CN202210565509.5). In both of these methods, no inter-coupling mechanism is set between instance segmentation and multi-target tracking. Instance segmentation errors will directly cause multi-target tracking to be interrupted, and normal tracking cannot be resumed after the instance segmentation is restored. In addition, both methods lack occlusion handling mechanisms and cannot determine whether a newly added object is a target that was previously occluded and has reappeared.

[0004] The paper (Yan Xuzhong, Zhu Yiqiao, Zhang Hong, Chen Yiyuan (2025). Intelligent perception and assessment of interference from precast component hoisting delay. Journal of Civil Engineering and Management) proposed a co-location mechanism for precast component instance segmentation and multi-target tracking. However, this mechanism relies on the matching of sparse feature points (such as corner points and edge points) between images. In cases where the background is simple, the texture is not rich, or the component outline is blurred, the number of key points is small and the coverage area is limited, which makes target tracking prone to drift or loss, affecting the subsequent segmentation accuracy. Sparse feature points only provide motion information of local discrete points and lack the ability to model the continuous flow field of the entire image or target region. It is difficult to support pixel-level instance segmentation and fine mask propagation between consecutive frames, especially when the target is deformed or partially occluded. In addition, in sparse feature point tracking, the segmentation results cannot make full use of the global motion information between consecutive frames, which makes it difficult for the segmentation and tracking tasks to form a tight coupling, limiting the improvement of the overall system performance. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides an intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking, in order to solve the technical problem that existing intelligent detection technologies cannot support pixel-level instance segmentation and fine mask propagation between consecutive frames.

[0006] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0007] A method for intelligent monitoring of component hoisting progress interference based on dense optical flow tracking includes the following steps:

[0008] Step S1: Instance segmentation model construction. The scanned image frames are processed. Each prefabricated component in the image frame is defined as a multiple target instance, and each target instance is segmented to form an instance segmentation model, generating multiple instance segmentation targets. Each consecutive frame is analyzed and detected. If the detection result is correct, the image data is output to step S4. If there is a missed detection, proceed to step S3. If there is occlusion or disappearance, proceed to step S2.

[0009] Step S2, component occlusion judgment: Analyze and judge the newly added prefabricated components detected by the instance segmentation model, and determine whether the scanned prefabricated component is an occluded prefabricated component. If the system judges it to be yes, assign it the original component ID and proceed to step S4; if the system judges it to be no, assign it a new component ID and proceed to step S4.

[0010] Step S3, Missed Detection Compensation: Pixel-level dense optical flow fields between adjacent frames are estimated using dense optical flow. When a target is detected as being missed in the current frame, its position and shape in the current frame are inferred based on its mask region and corresponding optical flow vector from the previous frame. A compensatory segmentation mask is generated, and the process proceeds to step S4.

[0011] Step S4: Multi-target tracking of prefabricated components. Cross-frame association is performed on the instance segmentation results in consecutive frames to achieve continuous tracking of prefabricated components in the video sequence.

[0012] Furthermore, in step 1, instance segmentation is based on a deep neural network model, and an activation function with a gating mechanism is used to perform nonlinear feature transformation, thereby achieving instance-level recognition and pixel-level segmentation of multiple prefabricated components in the image.

[0013] Furthermore, the deep neural network model adopts an improved model based on the YOLOv11 series architecture, and the gating activation function is the sigmoid linear unit activation function, the mathematical expression of which is shown in Equation 1:

[0014] Y= X ⊙ 1 / (1 + e^(−X)) Formula 1

[0015] Where X is the input feature matrix, Y is the output feature matrix, ⊙ is the Hadamard product, and e is the natural logarithm.

[0016] Furthermore, the specific processing steps for the image frame in step S1 are as follows:

[0017] Step S1-1: Read the video of the precast component hoisting;

[0018] Step S1-2: Read each frame of the prefabricated component hoisting video and extract the multi-scale features of each frame;

[0019] Steps S1-3: Fuse multi-scale features and output the class probability of each target and the pixel-level segmentation mask;

[0020] Steps S1-4: Set the number of frames after the prefabricated component appears. If the category probability and the number of consecutively detected frames are both greater than the set value, output image data to step S4. If the number of frames after the prefabricated component disappears is less than the preset value, it is determined that there is a missed detection and proceed to step S3. If the disappearance of the prefabricated component continues for the preset value, it is determined that the prefabricated component is occluded or has disappeared from the video and proceed to step S2.

[0021] Furthermore, in step 2, when a prefabricated component is obscured, its information is cached and stored in the cache set. In subsequent video frames, when the system detects a new component, it compares it to determine whether it is a previously cached prefabricated component. If the system determines that it is, it assigns it the original component ID and proceeds to step S4; otherwise, it assigns it a new component ID and proceeds to step S4.

[0022] Furthermore, the occlusion processing step S2 is as follows:

[0023] Step S2-1: When the system determines that a precast component is obscured by other personnel, equipment or components at the construction site, the algorithm will automatically cache the region of interest image features of the interrupted component, assign it a virtual ID, and store this information in the cache set.

[0024] In step S2-2, when the instance segmentation model detects a new component, the system calculates the feature similarity between the new component and each component in the cache set using a lightweight convolutional neural network. If the similarity exceeds a preset threshold, the component ID with the highest similarity is assigned to the new component, and the result is output to step S4. If the similarity does not reach the threshold, a new component ID is assigned to the new component, and the result is output to step S4.

[0025] Furthermore, the specific steps of the data association algorithm in step S3 are as follows:

[0026] Step S3-1: Input a continuous sequence of image frames. ,in The image representing time point t.

[0027] Step S3-2, for two adjacent frames of images and Calculate the dense optical flow field to obtain the value of each pixel. The motion vector from frame t-1 to frame t is shown in Formula 2:

[0028] Formula 2

[0029] in, and These represent the horizontal and vertical motion components of a pixel between frame t-1 and frame t, respectively.

[0030] Step S3-3: When a missed detection occurs in frame t+1, let the set of prefabricated component masks generated by instance segmentation in frame t (the current frame) be... ,in, It is a binary mask for object k at time t. Utilizing optical flow field... , mask Mapping to frame t+1, we obtain the prediction mask, calculated as shown in Formula 3:

[0031] Formula 3

[0032] The prediction mask This indicates the pixel position of the object in the next frame, inferred from motion estimation.

[0033] Step S3-4: When a missed detection occurs during instance segmentation, optical flow prediction mask is used. Compensation will be provided.

[0034] Step S3-5, remove the missed components. The component ID is fused with the target ID, and a stable and continuous target tracking trajectory and segmentation mask are output to step S4.

[0035] Furthermore, the specific steps of multi-target tracking in step S4 are as follows:

[0036] Step S4-1: Initialize the motion state of each instance segmented target, and use Kalman filtering to predict the target's position in the next frame in order to estimate the target's motion trajectory;

[0037] Step S4-2: Based on the predicted instance segmentation target location and the instance segmentation target location detected in the current frame, calculate the similarity matrix between targets; use feature vectors to calculate the overlap between target contours for comprehensive matching; use the Hungarian algorithm to perform optimal matching of targets and solve the association problem between targets and contours;

[0038] Step S4-3, Instance Segmentation Target Trajectory Management: For successfully matched targets, update their motion state and feature vector; for unmatched targets...

[0039] Step S4-4: Output the contour position and component ID of each prefabricated component in the multi-target tracking of prefabricated components.

[0040] Furthermore, the method also includes step S5, which, based on the instance segmentation and multi-target tracking results in step S4, determines the hoisting or installation status of the prefabricated components in the video and calculates the time from the appearance of each component to its installation.

[0041] Furthermore, the method also includes step S6, which calculates the estimated overall hoisting completion time based on the real-time monitoring of the prefabricated component hoisting progress in step S5, and compares it with the planned completion time to determine whether the deviation between the two exceeds the predetermined allowable range.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] 1. This invention, based on a dense optical flow estimation method, effectively corrects potential missed detections during instance segmentation, significantly improving the overall detection accuracy and operational stability of the system. Furthermore, addressing the issue of prefabricated components temporarily disappearing and then reappearing due to occlusion during hoisting, this invention designs a robust identity matching mechanism. This effectively solves the problem of identity recognition when prefabricated components reappear due to occlusion, greatly improving the accuracy and robustness of target tracking for prefabricated components under complex working conditions and reducing the interference of occlusion on progress judgment. These technological innovations not only significantly enhance the perception and decision-making capabilities of the component hoisting interference monitoring system but also provide reliable technical support for the refined and intelligent management of prefabricated building construction progress.

[0044] 2. This invention employs a non-invasive visual perception method, eliminating the need for additional sensors or tags on prefabricated components. It is convenient to deploy, highly compatible, and easily applicable to various construction site environments. Through intelligent monitoring and control algorithms, the system can analyze dynamic visual data of the hoisting process collected by cameras in real time without interfering with normal construction procedures, achieving accurate identification and effective control of interference factors such as hoisting delays. Furthermore, the system introduces an incremental learning mechanism, enhancing the model's transferability and system scalability across different project scenarios. By deeply integrating advanced intelligent visual perception technology with the Critical Path Method (CPM) in prefabricated building construction management, this invention achieves automated, intelligent, and real-time monitoring and control of prefabricated component hoisting progress interference, demonstrating promising engineering application prospects and significant promotional value. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0046] In the description of this invention, it should be understood that the terms "one end", "the other end", "outer side", "upper side", "inner side", "horizontal", "coaxial", "center", "end", "length", "outer end", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0047] The invention will now be further described with reference to the accompanying drawings.

[0048] like Figure 1 As shown, a method for intelligent monitoring of component hoisting progress interference based on dense optical flow tracking is described, with the following specific steps:

[0049] Step S1, prefabricated component instance segmentation: Based on a deep neural network model, an activation function with a gating mechanism is used to perform nonlinear feature transformation, thereby realizing instance-level recognition and pixel-level segmentation of multiple prefabricated components in the image, forming an instance segmentation model, and generating multiple instance segmentation targets.

[0050] In this embodiment, the deep neural network model adopts an improved model of the YOLOv11 series architecture, and the gate activation function is the sigmoid linear unit (SiLU) activation function, the mathematical expression of which is shown in Equation 1:

[0051] Y= X ⊙ 1 / (1 + e^(−X)) Formula 1

[0052] Where X is the input feature matrix, Y is the output feature matrix, ⊙ is the Hadamard product (element-by-element multiplication), and e is the natural logarithm.

[0053] Specifically, the image frame processing steps in step S1 are as follows:

[0054] Step S1-1: Read the video of the prefabricated component hoisting.

[0055] Steps S1-2 involve reading each frame of the prefabricated component hoisting video in real time using an improved YOLOv11 network configured with SiLU-type gated activation functions, extracting multi-scale features from each frame, and enhancing the representation capability of prefabricated components of different sizes (especially small targets).

[0056] Steps S1-3 fuse multi-scale features to output the class probability of each target and the pixel-level segmentation mask, thereby achieving instance-level segmentation of each prefabricated component.

[0057] In steps S1-4, if a prefabricated component appears for the first time and its category probability is greater than 0.5, it is considered a correct detection result after being continuously detected for 10 frames. A structured result containing the category label and pixel-level segmentation mask for each prefabricated component is output to step S4 for subsequent component tracking.

[0058] If the number of objects detected in the current frame is less than the number of objects detected in the previous frame, and this state lasts for less than 10 frames, it indicates that there is a missed detection in instance segmentation, and proceeds to step S3; if this state lasts for more than 10 frames, it is determined that the object is occluded or has disappeared from the video, and proceeds to step S2.

[0059] Step S2: Prefabricated component occlusion processing. When a prefabricated component is occluded, its information, namely the mask, category, and component ID from the last appearing frame, is cached and stored in the cache set. In subsequent video frames, when the system detects a new component, it compares it with a previously cached prefabricated component. If the system determines that it is, it means that the new component is a previously appearing prefabricated component, assigns it its original component ID, and proceeds to step S4; if the system determines that it is not, it assigns it a new component ID and proceeds to step S4.

[0060] Specifically, the occlusion processing steps in step S2 are as follows:

[0061] Step S2-1: When the system determines that the precast component is obscured by other personnel, equipment or components on the construction site, the algorithm will automatically cache the Region of Interest (RoI) image features of the interrupted component and assign it a virtual ID, which is the original component ID before the precast component was obscured or disappeared, and store this information in the cache set.

[0062] Step S2-2: When the instance segmentation model detects a new component, the system calculates the feature similarity between the new component and each component in the cache set using a lightweight convolutional neural network.

[0063] By comparing these similarities, the system can determine whether a newly added component matches a component in the cache set. If the similarity between the newly added component and a component in the cache set exceeds a preset threshold (e.g., 90%), the newly added component is determined to be a component that was previously occluded and has reappeared. The ID of the component with the highest similarity in the cache set is assigned to the newly added component, and the result is output to step S4. If the similarity does not reach the threshold, a new component ID is assigned to the newly added component, and the result is output to step S4.

[0064] Furthermore, the specific calculation method for the lightweight convolutional neural network in step S2-2 is as follows: the input image size is defined as 128×64 pixels. The entire network consists of two convolutional layers, one max-pooling layer, three residual blocks, and one fully connected layer. Each convolutional layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 1, and uses padding of 1 for the input feature map. The input and output sizes of each convolutional layer remain unchanged. The pooling layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 2, and padding of 1. Residual blocks 1 to 3 have 32, 64, and 128 channels, respectively. The fully connected layer generates a global feature map. Subsequently, a cosine similarity measurement method is used to determine whether a newly added component originates from the cache set. When the cache set is full, the oldest cached record is automatically deleted.

[0065] Step S3: Compensation for missed detections in prefabricated component instance segmentation. During instance segmentation and multi-target tracking in consecutive frames, dense optical flow is used to estimate the pixel-level dense optical flow field between adjacent frames. When a target is detected as being missed in segmentation in the current frame, its position and shape in the current frame are inferred based on its mask region and corresponding optical flow vector from the previous frame, and a compensatory segmentation mask is generated to maintain the continuity of the target trajectory.

[0066] Furthermore, the specific steps of the data association algorithm in step S3 are as follows:

[0067] Step S3-1, Image sequence input: Input a continuous sequence of image frames. ,in The image representing time point t.

[0068] Step S3-2, for two adjacent frames of images and Calculate the dense optical flow field to obtain the value of each pixel. The motion vector from frame t-1 to frame t is shown in Formula 2:

[0069] Formula 2

[0070] in, and These represent the horizontal and vertical motion components of a pixel between frame t-1 and frame t, respectively.

[0071] Step S3-3: When a missed detection occurs in frame t+1, let the set of prefabricated component masks generated by instance segmentation in frame t (the current frame) be... ,in, It is a binary mask for object k at time t. Utilizing optical flow field... , mask Mapping to the position of frame t+1 (the next frame), the prediction mask is obtained, and the specific calculation is shown in Formula 3:

[0072] Formula 3

[0073] The prediction mask This indicates the pixel position of the object in the next frame, inferred from motion estimation.

[0074] Step S3-4: When a missed detection occurs during instance segmentation, i.e., the corresponding mask... If the mask is missing or too small, use optical flow to predict it. Compensation is performed to ensure the continuity of multi-target tracking.

[0075] Step S3-5, remove the missed components. The component ID is merged, and a stable and continuous target tracking trajectory and segmentation mask are output to step S4 for subsequent hoisting progress analysis.

[0076] Step S4: Multi-target tracking of prefabricated components. Cross-frame association is performed on the instance segmentation results in consecutive frames to achieve continuous tracking of prefabricated components in the video sequence.

[0077] The specific steps of multi-target tracking in step S4 are as follows:

[0078] Step S4-1, target motion trajectory prediction, that is, initializing the motion state (including position and velocity information) of each instance segmented target, and using Kalman filtering to predict the position of the target in the next frame in order to estimate the target's motion trajectory.

[0079] Step S4-2, data association and matching, that is, based on the predicted instance segmentation target position and the instance segmentation target position detected in the current frame, calculate the similarity matrix between targets; use feature vectors to calculate the intersection over union (IoU) between target contours for comprehensive matching; use the Hungarian algorithm to perform optimal matching of targets and solve the association problem between targets and contours.

[0080] Step S4-3, instance segmentation target trajectory management, that is, for the successfully matched target, update its motion state and feature vector; for the unmatched target, there are three cases: (1) if the target is missed and the duration is less than 10 frames, return to step S3 to compensate for it; (2) if the target is a new target, return to step S2 to determine whether the new target is a previously occluded target; (3) if the target is occluded or disappears during the matching process, it is automatically included in the cache set of the occlusion component.

[0081] Step S4-4: Output the multi-target tracking results of the precast components to step S5, namely the contour position and component ID of each precast component.

[0082] Step S5 involves detecting the hoisting progress of prefabricated components. Based on the instance segmentation and multi-target tracking results in Step S4, the hoisting or installation status of the prefabricated components in the video is determined, and the time from the appearance of each component to its installation is calculated. The average hoisting time is obtained using the kernel function distribution estimation method, multiplied by the number of components, to estimate the overall hoisting completion time, thus achieving accurate progress prediction.

[0083] Specifically, the precast component hoisting progress detection steps in step S5 are as follows:

[0084] Step S5-1: Let T* be the planned completion time for hoisting the current floor, T0 be the time when the first component is detected, t1 be the time when the previous component is in place, n be the total number of components detected, and N be the total number of components required for the floor.

[0085] Step S5-2: Execute steps S1 to S4 to obtain the current component outline position and component ID.

[0086] Step S5-3: Determine whether the intersection ratio of the component outline position in the two consecutive frames is greater than 0.8 and lasts for 50 frames. Execute the corresponding steps based on the determination result: If the determination result is True, the component is in the hoisting state, and return to step S5-2; if the determination result is False, the component has been installed in place, record this moment as T2, and execute step S5-4.

[0087] Step S5-4: Based on the existing data, calculate the estimated hoisting time for the current floor as T = (T2 - T0) + (T2 - T1)(Nn).

[0088] Step S5-5: Determine the relationship between the estimated completion time T and the planned completion time T*. If T > T*, the system detects a delay interference in the hoisting of precast components, with a delay duration of T. Delay =TT *, otherwise if there is no delay or interference in the hoisting of precast components, wait for the next component to be hoisted, and return to step S5-1.

[0089] Step S6, Precast Component Hoisting Delay and Interference Management: Based on the real-time monitoring of the precast component hoisting progress in Step S5, calculate the estimated overall hoisting completion time and compare it with the planned completion time to determine if the deviation exceeds the predetermined allowable range. Once the deviation exceeds the threshold, corresponding corrective measures are initiated to ensure the project progresses smoothly as planned.

[0090] The specific evaluation method is as follows: Evaluate the monitored T Delay Whether the interference exceeds the allowable range of the prefabricated construction schedule includes three situations: 1) If the hoisting delay interference appears on the critical path of the prefabricated construction schedule, then it is determined that the hoisting delay interference will cause a delay in the entire construction schedule; 2) If the hoisting delay interference does not appear on the critical path of the construction schedule, but T Delay If the delay exceeds the total float of the hoisting procedure in the construction schedule, then the hoisting delay will also cause a delay in the entire construction schedule; 3) The hoisting delay does not occur on the critical path of the construction schedule and T Delay Less than the total float of the hoisting procedure in the construction schedule, but T Delay If the delay exceeds the free float of the hoisting procedure in the construction schedule, it is determined that the hoisting delay will cause delays in subsequent procedures. If any of these three situations occur, the initial schedule will become invalid. In this case, management personnel need to take appropriate corrective measures based on the resource conditions at the construction site to bring the disrupted schedule back on track.

[0091] This invention provides an intelligent monitoring system for component hoisting progress interference based on dense optical flow tracking, used to perform the above steps.

[0092] The system includes a precast component instance segmentation module, a precast component instance segmentation omission error compensation module, a precast component occlusion handling module, a precast component multi-target tracking module, a progress monitoring module, and a progress deviation management module.

[0093] The acquired images are input into the precast component instance segmentation module. This module, based on a deep neural network model, employs a gating activation function for nonlinear feature transformation, enabling instance-level recognition and pixel-level segmentation of multiple precast components within the image.

[0094] If the number of objects detected in the current frame is less than the number of objects detected in the previous frame for more than 10 consecutive frames, it is determined that the object is occluded or has disappeared from the video. This information is then input into the prefabricated component occlusion processing module for judgment, determining whether the scanned prefabricated component is an occluded component. When the system determines that a scanned prefabricated component is occluded, the system automatically caches the region of interest image features of the interrupted component, assigns it a virtual ID, and stores this information in the cache set.

[0095] If the number of objects identified in the current frame is less than the number of objects identified in the previous frame, and this state lasts for less than 10 frames, it indicates that there is a missed detection in instance segmentation. This information is then input into the prefabricated component instance segmentation missed detection error compensation module.

[0096] During the segmentation of prefabricated component instances, when a prefabricated component appears for the first time and its class probability is greater than 0.5, it is considered a correct detection result after being continuously detected for 10 frames. A structured result containing the class label and pixel-level segmentation mask for each prefabricated component is output to the prefabricated component multi-target tracking module for subsequent component tracking.

[0097] The multi-target tracking module initializes the motion state of each instance segmented target, uses Kalman filtering to predict the target's position in the next frame to estimate the target's trajectory, and calculates the similarity matrix between targets based on the predicted instance segmented target position and the instance segmented target position detected in the current frame; it uses feature vectors to calculate the overlap between target contours for comprehensive matching; and it uses the Hungarian algorithm to perform optimal matching of targets.

[0098] The multi-target tracking module receives feedback information from the precast component instance segmentation omission error compensation module and the precast component occlusion processing module, and outputs it to the progress deviation management module to monitor the hoisting or installation status of the precast components in continuous video frames, so as to make a judgment on the actual hoisting progress.

[0099] If a hoisting delay occurs on the critical path of the prefabricated construction schedule, the schedule deviation management module determines that the hoisting delay will cause a delay in the entire construction schedule. If the hoisting delay does not occur on the critical path of the construction schedule, but the delay duration is greater than the total float of the hoisting procedure in the construction schedule, the schedule deviation management module determines that the hoisting delay will also cause a delay in the entire construction schedule. If the hoisting delay does not occur on the critical path of the construction schedule and the delay duration is less than the total float of the hoisting procedure in the construction schedule, but the delay duration is greater than the free float of the hoisting procedure in the construction schedule, the schedule deviation management module determines that the hoisting delay will cause a delay in its subsequent procedures.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent monitoring of component hoisting progress interference based on dense optical flow tracking, characterized in that: The specific steps are as follows: Step S1: Process the scanned image frames; define each prefabricated component in the image frame as multiple target instances, segment each target instance to form an instance segmentation model, generate multiple instance segmentation targets, and analyze and detect each consecutive frame. If the detection result is correct, output the image data to step S4; if there is a missed detection, proceed to step S3; if there is occlusion or disappearance, proceed to step S2. Step S2, component occlusion judgment: Analyze and judge the newly added prefabricated components detected by the instance segmentation model, and determine whether the scanned prefabricated component is an occluded prefabricated component. If the system judges it to be yes, assign it the original component ID and proceed to step S4; if the system judges it to no, assign it a new component ID and proceed to step S4. Step S3, Missed detection compensation, using dense optical flow to estimate the pixel-level dense optical flow field between adjacent frames; When a target is detected to be missed during segmentation in the current frame, its position and shape in the current frame are inferred based on its mask region and corresponding optical flow vector in the previous frame, a compensating segmentation mask is generated, and the process proceeds to step S4. Step S4: Multi-target tracking of prefabricated components. Cross-frame association is performed on the instance segmentation results in consecutive frames to achieve continuous tracking of prefabricated components in the video sequence.

2. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: In step 1, instance segmentation is based on a deep neural network model and uses an activation function with a gating mechanism to perform nonlinear feature transformation, thereby achieving instance-level recognition and pixel-level segmentation of multiple prefabricated components in the image.

3. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 2, characterized in that: The deep neural network model adopts an improved model of the YOLOv11 series architecture, and the gating activation function is the sigmoid linear unit activation function, the mathematical expression of which is shown in Equation 1: Y= X ⊙ 1 / (1 + e^(−X)) Formula 1 Where X is the input feature matrix, Y is the output feature matrix, ⊙ is the Hadamard product, and e is the natural logarithm.

4. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: The specific processing steps for the image frame in step S1 are as follows: Step S1-1: Read the video of the precast component hoisting; Step S1-2: Read each frame of the prefabricated component hoisting video and extract the multi-scale features of each frame; Steps S1-3: Fuse multi-scale features and output the class probability of each target and the pixel-level segmentation mask; Steps S1-4: Set the number of frames after the prefabricated component appears. If the category probability and the number of consecutively detected frames are both greater than the set value, output image data to step S4. If the number of frames after the prefabricated component disappears is less than the preset value, it is determined that there is a missed detection and proceed to step S3. If the disappearance of the prefabricated component continues for the preset value, it is determined that the prefabricated component is occluded or has disappeared from the video and proceed to step S2.

5. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: In step 2, when a prefabricated component is obscured, its information is cached and stored in the cache set. In subsequent video frames, when the system detects a new component, it compares and determines whether it is a previously cached prefabricated component. If the system determines that it is, it assigns the original component ID and proceeds to step S4; otherwise, it assigns a new component ID and proceeds to step S4.

6. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: The occlusion processing steps in step S2 are as follows: Step S2-1: When the system determines that a precast component is obscured by other personnel, equipment or components at the construction site, the algorithm will automatically cache the region of interest image features of the interrupted component, assign it a virtual ID, and store this information in the cache set. Step S2-2: When the instance segmentation model detects a new component, the system calculates the feature similarity between the new component and each component in the cache set by using a lightweight convolutional neural network. If the similarity exceeds the preset threshold, the ID of the component with the highest similarity is assigned to the newly added component, and the result is output to step S4. If the similarity does not reach the threshold, a new component ID is assigned to the newly added component, and the result is output to step S4.

7. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: The specific steps of the data association algorithm in step S3 are as follows: Step S3-1: Input a continuous sequence of image frames. ,in A graph representing time point t; Step S3-2, for two adjacent frames of images and Calculate the dense optical flow field to obtain the value of each pixel. The motion vector from frame t-1 to frame t is shown in Formula 2: Formula 2 in, and These represent the horizontal and vertical motion components of a pixel between frame t-1 and frame t, respectively. Step S3-3: When a missed detection occurs in frame t+1, let the set of prefabricated component masks generated by instance segmentation in frame t (the current frame) be... ,in, It is a binary mask of object k at time t; utilizing optical flow field , mask Mapping to frame t+1, we obtain the prediction mask, calculated as shown in Formula 3: Formula 3 The prediction mask This indicates the pixel position of the object in the next frame, inferred from motion estimation. Step S3-4: When a missed detection occurs during instance segmentation, optical flow prediction mask is used. Provide compensation; Step S3-5, remove the missed components. The component ID is fused with the target ID, and a stable and continuous target tracking trajectory and segmentation mask are output to step S4.

8. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: The specific steps of multi-target tracking in step S4 are as follows: Step S4-1: Initialize the motion state of each instance segmented target, and use Kalman filtering to predict the target's position in the next frame in order to estimate the target's motion trajectory; Step S4-2: Calculate the similarity matrix between targets based on the predicted instance segmentation target location and the instance segmentation target location detected in the current frame; The overlap between target contours is calculated using feature vectors for comprehensive matching; the Hungarian algorithm is used to perform optimal matching of targets, thus solving the correlation problem between targets and contours. Step S4-3, Instance Segmentation Target Trajectory Management: For successfully matched targets, update their motion state and feature vector; for unmatched targets; Step S4-4: Output the contour position and component ID of each prefabricated component in the multi-target tracking of prefabricated components.

9. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: It also includes step S5, which, based on the instance segmentation and multi-target tracking results in step S4, determines the hoisting or installation status of the prefabricated components in the video and calculates the time from the appearance of each component to its installation.

10. The intelligent monitoring method for component hoisting progress interference based on dense optical flow tracking according to claim 1, characterized in that: It also includes step S6, which calculates the estimated overall hoisting completion time based on the real-time monitoring of the prefabricated component hoisting progress in step S5, and compares it with the planned completion time to determine whether the deviation between the two exceeds the predetermined allowable range.

Images