Intelligent monitoring method for hoisting progress interference of prefabricated building prefabricated parts

Abstract

The invention discloses a prefabricated building prefabricated component hoisting progress interference intelligent monitoring method, which comprises the following steps of S1, instance segmentation model building, S2, component shielding judgment, S3, component shielding processing, S4, multi-target tracking, S5, mutual coupling of instance segmentation and multi-target tracking, S6, hoisting progress detection, and based on component identification and tracking data in the S5, carrying out intelligent monitoring on hoisting progress interference of a prefabricated component. And judging the hoisting or mounting state of the prefabricated part in the continuous video frames so as to monitor the actual hoisting progress. The adopted non-intrusive camera does not need to be in direct contact with the prefabricated part and is widely applied to a construction site, the adopted intelligent monitoring and control technology can analyze prefabricated part hoisting visual data which are captured by the camera and contain rich dynamic information in real time on the premise that normal construction is not interfered, and the construction efficiency is improved. And the hoisting delay interference of the prefabricated part can be effectively monitored and controlled.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent monitoring and control, and particularly relates to an intelligent monitoring method for the hoisting progress interference of precast components in prefabricated buildings. Background Technique

[0002] In the prior art, a Chinese patent with the publication number CN119514975A discloses an intelligent construction method for tracking the hoisting progress of precast components based on radio frequency identification tags. This method relies on manual use of a handheld scanning device to manually scan the radio frequency identification tags on precast components within a limited area to obtain their position and progress status information. Although radio frequency identification technology has high identification accuracy, due to the need for manual intervention, the monitoring process may take a long time, making it difficult to achieve real-time and dynamic monitoring of the construction progress. Summary of the Invention

[0003] In order to make up for the deficiencies of the prior art, the present invention provides an intelligent monitoring method for the hoisting progress interference of precast components in prefabricated buildings to solve the technical problem that the monitoring process takes a long time and it is difficult to achieve real-time and dynamic monitoring of the construction progress.

[0004] To achieve the above object, the specific technical solution of the present invention is as follows:

[0005] An intelligent monitoring method for the hoisting progress interference of precast components in prefabricated buildings, the specific steps are as follows:

[0006] S1. Build an instance segmentation model, and use a convolutional neural network algorithm to process the scanned image frames. Each precast component in the image frame is defined as multiple target instances. Collect and process the feature data of each target instance to generate a feature map containing multi-scale information. Based on the feature map of multi-scale information, perform precise pixel-level instance segmentation on each target instance to form an instance segmentation model and generate multiple instance segmentation targets.

[0007] S2. Determine component occlusion. When the instance segmentation model detects a newly added precast component, use a lightweight convolutional neural network to analyze and judge the newly added component to determine whether the precast component detected by scanning is an occluded precast component. If so, proceed to step 3. Otherwise, directly proceed to step S4.

[0008] S3. Process component occlusion. When the system determines that a precast component may be occluded by other personnel, equipment, or components at the construction site, the algorithm will automatically cache the image features of the interested area of the interrupted component, assign a virtual ID, and store this information in the cached component set.

[0009] S4. Multi-object tracking. The multi-object tracking algorithm continuously monitors the change of the contour position information of the identified components, assigns a component ID to each component, inputs the monitored information into step S5 for analysis and calculation, and receives the contour positions and component IDs of each precast component analyzed in step S5, and centrally outputs the received information to step S6.

[0010] S5. Instance segmentation and multi-object tracking are coupled. Receive the precast component information correctly detected by the instance segmentation model in the image frame in step S4, perform gray processing on the contour area and generate up to 200 corner points within the gray contour area. When the instance segmentation model fails to detect in a certain frame, the corner point positions of the current frame are estimated based on the sparse optical flow between the current frame and the previous frame. If there is no undetected situation, the result of the instance segmentation model is directly output to step S4; if there is an undetected situation, the result of the sparse optical flow estimation is output to step S4.

[0011] S6. Lifting progress detection. Based on the data of component recognition and tracking in step S4, that is, the contour position of the component in the image frame and its component ID, judge the hoisting or installation state of the precast component in consecutive video frames to monitor the actual lifting progress, that is, the transportation or installation state of the precast component.

[0012] Furthermore, the specific processing steps of the image frame in step S1 are as follows:

[0013] S1-1. Perform three preprocessing operations on the input image frame: normalization, color space conversion, and data augmentation;

[0014] S1-2. Input the image frame into a series of convolutional layers, and scan the entire image frame through a set of filters to capture its local features;

[0015] S1-3. Apply the rectified linear unit function as the activation function after each convolutional operation;

[0016] S1-4. Use the max pooling method to reduce the size of the feature map via the pooling layer while ensuring that key information is retained;

[0017] S1-5. Generate a feature map containing multi-scale information by adjusting the configuration of the convolutional layer and the pooling layer;

[0018] S1-6. Based on the generated multi-scale feature map, use a segmentation network to achieve accurate pixel-level instance segmentation of each target instance.

[0019] Furthermore, the specific judgment method in step S2 is that when there is an interruption phenomenon in multiple consecutive frames during the detection of the precast component, the system will determine that the precast component may be blocked by other personnel, equipment, or components at the construction site.

[0020] Further, in step S3, when the instance segmentation model detects a newly added prefabricated component, the system calculates the feature similarity between the newly added prefabricated component and each component in the cached component set by using a lightweight convolutional neural network. If it matches a certain prefabricated component in the cached component set, it is determined that the newly added prefabricated component is a prefabricated component that reappeared after being occluded before, and the virtual ID with the highest similarity in the cache set is assigned to the newly added component. Then it enters step S4. Otherwise, a new virtual ID is assigned to the newly added prefabricated component.

[0021] Further, the specific calculation method of the lightweight convolutional neural network in step S3 is as follows. The lightweight convolutional neural network defines the size of the input image as 128×64 pixels. The entire network includes five convolutional layers, one max pooling layer, four residual blocks, and one fully connected layer. Each convolutional layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 1, and uses padding of 1 to process the input feature map. The input and output sizes of each convolutional layer remain unchanged. The pooling layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 2, and padding of 1. Residual blocks 1 to 4 have 32, 64, 64, and 128 channels respectively. The fully connected layer generates a global feature map. Subsequently, the cosine similarity measurement method is used to determine whether the newly added prefabricated component is from the cached component set.

[0022] Further, the tracking method of the multi-object tracking algorithm in step S4 is as follows.

[0023] S4-1. Prediction of the target motion trajectory, that is, initializing the motion state of each instance segmentation target, and using the Kalman filter to predict the position of the target in the next frame to estimate the target motion trajectory.

[0024] S4-2. Data association and matching, that is, based on the predicted instance segmentation target position and the instance segmentation target position detected in the current frame, the detected data is input into step S5 for analysis and calculation to calculate the similarity matrix between the targets; the overlap degree between the target contours is calculated using the feature vector for comprehensive matching; the Hungarian algorithm is used to perform optimal matching on the targets.

[0025] S4-3. Management of the instance segmentation target trajectory, that is, for the instance segmentation target with successful matching, updating its motion state and feature vector; for the instance segmentation target with unsuccessful matching, if the instance segmentation target is a newly added target, return to step S3 to determine whether the newly added target is a previously occluded target. If the target disappears during the matching process, it is automatically classified into the cache set of occluded components.

[0026] S4-4. Output the multi-object tracking result of the prefabricated component to step S6, that is, the contour position and component ID of each prefabricated component.

[0027] Further, the specific calculation method for estimating the corner position of the current frame in step S5 is as follows: Let the pixel intensity inside the contour be I mask , translate the filter w(x, y), then the sparse optical flow between the current frame and the previous frame can be represented by the change in pixel density of the filter movement , as shown in Equation (1), and regard the significantly changed area as the estimated position of the corner of the current frame:

[0028] (1)

[0029] Use Taylor expansion approximation , and let and be the partial derivatives of , can be written in matrix form as shown in Equation (2). Then, locate the corner features inside the target by larger in all directions.

[0030] (2)

[0031] In the formula refers to the pixel intensity of the (x, y) coordinates; refers to the pixel intensity after the (x, y) coordinates are translated ; The translation filter refers to a small area centered on the corner generated within the gray-scale contour area. By calculating the change in pixel density within this small area, sparse optical flow tracking of the corner inside the contour can be achieved.

[0032] Next, calculate the affine transformation matrix between the homogeneous coordinates of the corners in the current frame and the previous frame, and use this matrix to map the vertex coordinates of the contour in the previous frame to the current frame. Next, calculate the affine transformation matrix between the homogeneous coordinates of the corners in the current frame and the previous frame, and use this matrix to map the vertex coordinates (x i , y i ) in the previous frame to the current frame, as shown in Equation (3):

[0033] (3)

[0034] In the formula s 11 、s 12 、s 21 and s 22 refer to the scaling, shearing, and rotation parameters; t 1 and t 2 refer to the translation parameters;(x frame , y frame ) and ([[]] x last_frame , y last_frame ) respectively refer to the vertex coordinates of the contour in the current frame and the previous frame. When the instance segmentation model fails to detect an object in a certain frame, ([[]] x frame , y frame ) is used for automatic correction.

[0035] Furthermore, the specific monitoring steps in step S6 are as follows:

[0036] S6-1. The planned hoisting completion time for the current floor is T*, the moment when the component is first detected is T0, the moment when the previous component is in place is T1, the total number of detected components is n, and the total number of components required for the floor is N;

[0037] S6-2. Execute the four modules S1~S5 to obtain the current component contour position and ID;

[0038] S6-3. Determine whether the intersection over union (IoU) of the component contour position in the previous and current frames is greater than 0.8 and lasts for 50 frames. Execute corresponding steps according to the judgment result: If the judgment result is True, the component is in the hoisting state, and at this time, return to step S6-2; If the judgment result is False, the component has been installed in place. Record this moment as T2 and continue to execute the next step S6-4;

[0039] S6-4. According to the existing data, calculate the estimated hoisting completion time for the current floor as T = (T2 - T0) + (T2 - T1)(N - n);

[0040] S6-5. Judge the magnitude relationship between the estimated hoisting completion time T and the planned hoisting completion time T*. If T > T*, the system detects a precast component hoisting delay interference, and the delay duration is T Delay = T - T*. Otherwise, there is no precast component hoisting delay interference. Wait for the next component to start hoisting and return to step S6-1.

[0041] Furthermore, it also includes step S7, hoisting delay interference control. According to the evaluated T Delay judge whether it exceeds the interference tolerance range of the prefabricated construction progress plan.

[0042] If the hoisting delay interference appears on the critical path of the prefabricated construction progress plan, it is determined that this hoisting delay interference will cause the delay of the entire construction progress plan; If this hoisting delay interference does not appear on the critical path of the construction progress plan, but T DelayIf it is greater than the total float of the hoisting process in the construction schedule, it is determined that this hoisting delay interference will also cause delays in the entire construction schedule; if this hoisting delay interference does not appear on the critical path of the construction schedule and T Delay is less than the total float of the hoisting process in the construction schedule, but T Delay is greater than the free float of the hoisting process in the construction schedule, it is determined that this hoisting delay interference will cause delays in its subsequent processes.

[0043] Furthermore, when the cache component set is full in step S3, the earliest cache record will be automatically deleted.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] 1. The non-intrusive camera adopted by the present invention does not need to directly contact the precast components and can be widely applied to the construction site. The intelligent monitoring and control technology adopted can, without disturbing normal construction, analyze in real time the visual data of precast component hoisting containing rich dynamic information captured by the camera, and realize effective monitoring and control of the hoisting delay interference of precast components. The system adopts an incremental learning method, ensuring the transferability and scalability of the interference monitoring and control system. By effectively combining advanced intelligent perception technology with the critical path method in the construction schedule management of prefabricated buildings, the present invention realizes the automation, intelligence and real-time of the monitoring and control of hoisting delay interference of precast components, showing good engineering application prospects.

[0046] 2. In the intelligent monitoring and control system for hoisting delay interference of precast components, the present invention uses a convolutional neural network algorithm for instance segmentation and multi-object tracking fusion technology, which can accurately identify and track the contour position and its identity ID of the precast component in the hoisting process in the image frame, and maintain stable tracking of the target between consecutive frames. Compared with the prior art, the present invention enhances the coupling degree between the two modules through the mutual coupling algorithm of instance segmentation and multi-object tracking, enabling instance segmentation to automatically correct the errors accumulated in the target tracking process, while target tracking can also correct the missed detection errors in instance segmentation, improving the accuracy and stability of the overall system. In addition, the present invention also effectively solves the problem of identity recognition when precast components reappear due to occlusion through the occlusion processing module constructed by a lightweight convolutional neural network, thereby improving the accuracy and robustness of instance segmentation and target tracking, and reducing the impact of occlusion on hoisting progress monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by terms such as "one end", "the other end", "outer side", "upper", "inner side", "horizontal", "coaxial", "center", "end", "length", "outer end", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0049] The present invention will be further described below with reference to the accompanying drawings.

[0050] As Figure 1 shown, an intelligent monitoring and control method for the hoisting progress interference of prefabricated components in prefabricated buildings includes the following steps:

[0051] S1. Build an instance segmentation model, and use a convolutional neural network algorithm to process the scanned image frames. Each prefabricated component in the image frame is defined as multiple target instances. Collect and process the feature data of each target instance to generate a feature map containing multi-scale information. Based on the feature map of multi-scale information, perform precise pixel-level instance segmentation on each target instance to form an instance segmentation model and generate multiple instance segmentation targets.

[0052] Specifically, S1-1. Perform three preprocessing operations on the input image frame: normalization, color space conversion, and data augmentation;

[0053] S1-2. Input the image frame into a series of convolutional layers, and scan the entire image frame through a set of filters to capture its local features;

[0054] S1-3. Apply the rectified linear unit (ReLU) as the activation function after each convolutional operation;

[0055] S1-4. Use the max pooling method to reduce the size of the feature map via the pooling layer while ensuring that key information is retained;

[0056] S1-5. Generate a feature map containing multi-scale information by adjusting the configuration of the convolutional layer and the pooling layer;

[0057] S1-6. Based on the generated multi-scale feature map, use a segmentation network to achieve precise pixel-level instance segmentation of each target instance.

[0058] S2. Component occlusion judgment. When the instance segmentation model detects a newly added prefabricated component, use a lightweight convolutional neural network to analyze and judge the newly added component to determine whether the prefabricated component detected by scanning is an occluded prefabricated component. If so, enter step 3. Otherwise, directly enter step S4.

[0059] The specific judgment method is that when there are interruption phenomena in multiple consecutive frames during the detection of precast components, the system will determine that the precast component may be blocked by other personnel, equipment or components at the construction site.

[0060] S3. Component occlusion processing. When the system determines that the precast component may be blocked by other personnel, equipment or components at the construction site, the algorithm will automatically cache the image features of the Region of Interest (RoI) of the interrupted component, assign a virtual ID, and store this information in the cached component set.

[0061] When the instance segmentation model detects a newly added precast component, the system will calculate the feature similarity between the newly added precast component and each component in the cached component set by using a lightweight convolutional neural network. By comparing these similarities, the system can determine whether the newly added precast component matches a certain precast component in the cached set. If the similarity between the newly added precast component and a certain precast component in the cached set exceeds a preset threshold (e.g., 90%), it is determined that the newly added precast component is the precast component that reappears after being blocked before, and the virtual ID with the highest similarity in the cached set is assigned to the newly added component. Subsequently, it enters step S4. Conversely, when the similarity does not reach the threshold, a new virtual ID is assigned to the newly added precast component.

[0062] Furthermore, in step S3, the lightweight convolutional neural network defines the size of the input image as 128×64 pixels. The entire network includes five convolutional layers, one max-pooling layer, four residual blocks and one fully connected layer. Each convolutional layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 1, and uses a padding of 1 to process the input feature map. The input and output sizes of each convolutional layer remain unchanged. The pooling layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 2, and a padding of 1. Residual blocks 1 to 4 have 32, 64, 64 and 128 channels respectively. The fully connected layer generates a global feature map. Subsequently, the cosine similarity measurement method is used to determine whether the newly added precast component originates from the cached component set.

[0063] In this embodiment, when the cached component set is full, the earliest cached record will be automatically deleted.

[0064] S4. Multi-object tracking. The multi-object tracking algorithm continuously monitors the change of the contour position information of the identified components and assigns a component ID to each component.

[0065] Specifically, S4-1. Target motion trajectory prediction, that is, initialize the motion state of each instance segmentation target, and use the Kalman filter to predict the position of the target in the next frame to estimate the motion trajectory of the target.

[0066] S4-2. Data association and matching: Based on the predicted instance segmentation target positions and the instance segmentation target positions detected in the current frame, the detected data is input into step S5 for analysis and calculation to compute the similarity matrix between the targets; the overlap degree between the target contours is calculated using feature vectors for comprehensive matching; the targets are optimally matched through the Hungarian algorithm to solve the association problem between the targets and the contours.

[0067] S4-3. Instance segmentation target trajectory management: For the instance segmentation targets with successful matching, update their motion states and feature vectors; for the instance segmentation targets with unsuccessful matching, if the instance segmentation target is a newly added target, return to step S3 to determine whether the newly added target is a previously occluded target. If the target disappears during the matching process, it is automatically included in the cache set of the occluded components.

[0068] S4-4. Output the multi-target tracking results of the precast components to step S6, that is, the contour positions and component IDs of each precast component.

[0069] S5. Mutual coupling of instance segmentation and multi-target tracking. This step receives the instance segmentation target position data from S4-2. When the instance segmentation model correctly detects the precast components in the image frame, the contour area is grayscale processed and up to 200 corner points are generated within the grayscale contour area. When the instance segmentation model misses a detection in a certain frame, the corner point positions in the current frame are estimated based on the sparse optical flow between the current frame and the previous frame. If there is no missed detection, the results of the instance segmentation model are directly output to step S4-4. If there is a missed detection, the sparse optical flow estimation results are output to step S4-4.

[0070] Specifically, let the pixel intensity within the contour be I mask , and the translation filter w(x, y). Then, the sparse optical flow between the current frame and the previous frame can be represented by the change in pixel density of the filter movement , as shown in Equation (1), and the significantly changed area is regarded as the estimated position of the corner points in the current frame:

[0071] (1)

[0072] Using Taylor expansion approximation , and let and be the partial derivatives of , can be written in the matrix form shown in Equation (2). Then, the corner point features within the target are located by the larger in all directions.

[0073] (2)

[0074] In the formula The pixel intensity at the (x, y) coordinates; Refers to the translation of the (x, y) coordinates of the pixel intensity after translation; translation filter Refers to a small area centered on the corner points generated within the grayscale contour region. By calculating the change in pixel density within this small area, sparse optical flow tracking of the corner points within the contour can be achieved.

[0075] Next, calculate the affine transformation matrix between the homogeneous coordinates of the corner points in the current frame and the previous frame, and use this matrix to map the vertex coordinates of the contour in the previous frame to the current frame. Next, calculate the affine transformation matrix between the homogeneous coordinates of the corner points in the current frame and the previous frame, and use this matrix to map the vertex coordinates (x i , y i ) in the previous frame to the current frame, as shown in Equation (3):

[0076] (3)

[0077] In the formula s 11 、s 12 、s 21 and s 22 refer to the scaling, shearing, and rotation parameters; t 1 and t 2 refer to the translation parameters; ( x frame , y frame ) and ( x last_frame , y last_frame ) respectively refer to the vertex coordinates of the contour in the current frame and the previous frame. When the instance segmentation model has a missed detection error in a certain frame, ([[]] x frame , y frame ) is used for automatic correction.

[0078] Through the above-mentioned mutual coupling algorithm of instance segmentation and multi-object tracking, the coupling degree between the precast component instance segmentation module and the multi-object tracking module can be enhanced, enabling instance segmentation and multi-object tracking to mutually correct errors, that is, instance segmentation automatically corrects the cumulative error of object tracking, and object tracking automatically corrects the missed detection error of instance segmentation.

[0079] S6. Lifting progress detection. Based on the data of component recognition and tracking in step S4-4, that is, the contour position of the component in the image frame and its component ID, determine the hoisting or installation state of the precast component in consecutive video frames to monitor the actual lifting progress, that is, the transportation or installation state of the precast component.

[0080] Specifically, in S6-1, the planned completion time for hoisting on the current floor is T*, the moment when the component is first detected is T0, the moment when the previous component is in place is T1, the total number of detected components is n, and the total number of components required on the floor is N.

[0081] In S6-2, execute the four modules of S1 to S5 to obtain the contour position and ID of the current component.

[0082] In S6-3, determine whether the intersection over union of the component contour positions in two consecutive frames is greater than 0.8 and lasts for 50 frames, and execute corresponding steps according to the judgment result: If the judgment result is True, the component is in the hoisting state, and at this time, return to step S6-2; if the judgment result is False, the component has been installed in place, record this moment as T2 and continue to execute the next step S6-4.

[0083] In S6-4, according to the existing data, calculate the estimated completion time for hoisting on the current floor as T = (T2 - T0) + (T2 - T1)(N - n).

[0084] In S6-5, judge the magnitude relationship between the estimated completion time for hoisting T and the planned completion time for hoisting T*. If T > T*, the system detects interference in the hoisting delay of the precast component, and the delay duration is T Delay = T - T*. Otherwise, there is no interference in the hoisting delay of the precast component. Wait for the next component to start hoisting and return to step S6-1.

[0085] S7. Hoisting delay interference control. Evaluate the monitored T Delay Whether it exceeds the interference tolerance range of the prefabricated construction progress plan, including three cases: 1) If the hoisting delay interference appears in the critical path of the prefabricated construction progress plan, it is determined that this hoisting delay interference will cause delays in the entire construction progress plan; 2) If this hoisting delay interference does not appear in the critical path of the construction progress plan, but T Delay is greater than the total float of the hoisting process in the construction progress plan, it is determined that this hoisting delay interference will also cause delays in the entire construction progress plan; 3) If the hoisting delay interference does not appear in the critical path of the construction progress plan and T Delay is less than the total float of the hoisting process in the construction progress plan, but T DelayIf it is greater than the free float of the hoisting process in the construction schedule, it is determined that the hoisting delay interference will cause delays in its subsequent processes. Once any of the above three situations occurs, the initial schedule will become invalid. At this time, the management personnel need to take corresponding corrective measures to deal with the interference according to the resource conditions at the construction site, so that the schedule that has deviated from the track due to interference can return to the right track according to the plan.

[0086] The present invention provides an intelligent monitoring system for the hoisting progress interference of prefabricated components of an assembled building, which is used to execute the above steps.

[0087] The system includes a precast component instance segmentation module, an occlusion processing module, a multi-target tracking module, a mutual coupling module, a progress detection module, and a progress deviation management module.

[0088] The collected image is input into the precast component instance segmentation module. The precast component instance segmentation module first performs three preprocessing operations on the input image frame: normalization, color space conversion, and data augmentation. Secondly, the image frame is input into a series of convolutional layers, and a set of filters scans the entire image frame to capture its local features. After each convolutional operation, the rectified linear unit function is applied as the activation function, and the max-pooling method is used to reduce the size of the feature map via the pooling layer while ensuring that key information is retained.

[0089] By adjusting the configuration of the convolutional layer and the pooling layer, a feature map containing multi-scale information is generated. Based on the generated multi-scale feature map, a segmentation network is used to achieve precise pixel-level instance segmentation of each target instance, and multiple instance segmentation targets are generated while constructing the instance segmentation model. This process ensures the ability to effectively analyze the image frame and accurately segment the target object.

[0090] When the precast component instance segmentation module encounters a newly added precast component during the image processing, it will enter the occlusion processing module for judgment to determine whether the scanned and detected precast component is an occluded precast component. When the scanned and detected precast component is determined by the system to be occluded, the system automatically caches the image features of the region of interest of the interrupted component, assigns a virtual ID, and stores this information in the cached component set.

[0091] If there is no occlusion phenomenon for the newly added precast component, the information data obtained by the precast component instance segmentation module is input into the multi-object tracking module. The multi-object tracking module initializes the motion state of each instance segmentation target, uses Kalman filtering to predict the position of the target in the next frame to estimate the motion trajectory of the target, and calculates the similarity matrix between the targets based on the predicted position of the instance segmentation target and the position of the instance segmentation target detected in the current frame; uses eigenvectors to calculate the overlap degree between the target contours for comprehensive matching; and performs optimal matching on the targets through the Hungarian algorithm. For the instance segmentation targets with successful matching, the detected data is input into the mutual coupling module for further analysis and calculation.

[0092] The mutual coupling module receives the contour position information output from the multi-object tracking module, performs grayscale processing on the contour area and generates up to 200 corner points within the grayscale contour area; when the instance segmentation model has a missed detection in a certain frame, the corner point position of the current frame is estimated based on the sparse optical flow between the current frame and the previous frame; if there is no missed detection, the result of the instance segmentation model is directly fed back to the multi-object tracking module, and if there is a missed detection, the sparse optical flow estimation result is fed back to the multi-object tracking module.

[0093] The multi-object tracking module receives the feedback information output by the mutual coupling module and outputs it to the progress deviation management module to judge the hoisting or installation state of the precast component in consecutive video frames to monitor the actual hoisting progress.

[0094] If the hoisting delay interference appears in the critical path of the precast construction progress plan, the progress deviation management module determines that this hoisting delay interference will cause a delay in the entire construction progress plan; if this hoisting delay interference does not appear in the critical path of the construction progress plan, but the delay duration is greater than the total float of the hoisting process in the construction progress plan, the progress deviation management module determines that this hoisting delay interference will also cause a delay in the entire construction progress plan; if this hoisting delay interference does not appear in the critical path of the construction progress plan and the delay duration is less than the total float of the hoisting process in the construction progress plan, but the delay duration is greater than the free float of the hoisting process in the construction progress plan, the progress deviation management module determines that this hoisting delay interference will cause a delay in its subsequent processes.

[0095] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent monitoring method for the hoisting progress interference of prefabricated components in prefabricated buildings, characterized in that: The specific steps are as follows: S1. Build an instance segmentation model to process the scanned image frames; define each prefabricated component in the image frame as multiple target instances, perform instance segmentation on each target instance to form an instance segmentation model, and generate multiple instance segmentation targets; S2. Component occlusion judgment: Analyze and judge the newly detected prefabricated components by the instance segmentation model to determine whether the prefabricated components detected by scanning are occluded prefabricated components. If so, go to step 3; otherwise, directly go to step S4; S3. Component occlusion processing: When the system determines that there is occlusion of the prefabricated component, the system automatically caches the image features of the region of interest of the interrupted component, assigns a virtual ID, and stores this information in the cached component set; S4. Multi-target tracking: Continuously monitor the change of the contour position information of the identified components, assign a component ID to each component, input the monitored information into step S5 for analysis and calculation, and receive the contour position and component ID information of each prefabricated component analyzed in step S5, and centrally output the received information to step S6; S5. Coupling of instance segmentation and multi-target tracking: Receive the contour position information of the identified components in step S4, perform gray processing on the contour area and generate multiple corner points within the gray contour area; When the instance segmentation model fails to detect in a certain frame, estimate the corner point position of the current frame based on the sparse optical flow between the current frame and the previous frame; if there is no missed detection, directly output the result of the instance segmentation model to step S4. If there is a missed detection, output the sparse optical flow estimation result to step S4; S6. Lifting progress detection: Based on the data of component identification and tracking in step S4, judge the hoisting or installation state of the prefabricated components in consecutive video frames to monitor the actual lifting progress.

2. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 1, characterized in that: The specific processing steps of the image frame in step S1 are as follows: S1-1. Perform three preprocessing operations on the input image frame: normalization, color space conversion, and data augmentation; S1-2. Input the image frame into a series of convolutional layers, and scan the entire image frame through a set of filters to capture its local features; S1-3. Apply a rectified linear unit function as the activation function after each convolutional operation; S1-4. Use the max pooling method to reduce the size of the feature map via the pooling layer while ensuring that key information is retained; S1-5. Generate a feature map containing multi-scale information by adjusting the configuration of the convolutional layer and the pooling layer; S1-6. Based on the generated multi-scale feature map, use a segmentation network to achieve accurate pixel-level instance segmentation of each target instance.

3. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 1, characterized in that: The specific judgment method in step S2 is that when there is an interruption phenomenon in multiple consecutive frames during the detection of the prefabricated component, the system will determine that the prefabricated component may be occluded by other personnel, equipment, or components at the construction site.

4. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 1, characterized in that: In step S3, when the instance segmentation model detects a newly added precast component, the system calculates the feature similarity between the newly added precast component and each component in the cached component set by using a lightweight convolutional neural network. If it matches a certain precast component in the cached component set, it is determined that the newly added precast component is a precast component that reappears after being occluded before, and the virtual ID with the highest similarity in the cache set is assigned to the newly added component; Subsequently, it enters step S4; Otherwise, a new virtual ID is assigned to the newly added precast component.

5. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 4, characterized in that: The specific calculation method of the lightweight convolutional neural network in step S3 is as follows. The lightweight convolutional neural network defines the size of the input image as 128×64 pixels; the entire network includes five convolutional layers, one max-pooling layer, four residual blocks, and one fully connected layer; each convolutional layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 1, and uses padding of 1 to process the input feature map; the input and output sizes of each convolutional layer remain unchanged; the pooling layer contains 32 channels, uses a 3×3 convolutional kernel, has a stride of 2, and padding of 1; residual blocks 1 to 4 have 32, 64, 64, and 128 channels respectively; the fully connected layer generates a global feature map; Subsequently, the cosine similarity measurement method is used to determine whether the newly added precast component is from the cached component set.

6. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 4, characterized in that: The tracking method of the multi-object tracking algorithm in step S4 is as follows: S4-1. Prediction of the target motion trajectory, that is, initializing the motion state of each instance segmentation target, and using the Kalman filter to predict the position of the target in the next frame to estimate the target motion trajectory; S4-2. Data association and matching, that is, based on the predicted instance segmentation target position and the instance segmentation target position detected in the current frame, calculating the similarity matrix between the targets; Using the feature vector to calculate the overlap degree between the target contours for comprehensive matching; using the Hungarian algorithm to perform optimal matching on the targets to solve the association problem between the targets and the contours; S4-3. Management of the instance segmentation target trajectory, that is, for the instance segmentation targets that are successfully matched, input the detected data into step S5 for analysis and calculation to update their motion state and feature vectors; for the instance segmentation targets that are not successfully matched, if the instance segmentation target is a newly added target, return to step S3 to determine whether the newly added target is a previously occluded target. If the target disappears during the matching process, it is automatically classified into the cache set of occluded components; S4-4. Output the multi-object tracking results of the precast components to step S6, that is, the contour positions and component IDs of each precast component.

7. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 1, characterized in that: The specific calculation method for estimating the corner position of the current frame in step S5 is as follows: Let the pixel intensity within the contour be I mask , translate the filter w(x, y), then the sparse optical flow between the current frame and the previous frame can be represented by the change in pixel density of the filter movement , as shown in Equation (1), and regard the significantly changed area as the estimated position of the corner of the current frame: (1) Using Taylor expansion approximation , and let and be partial derivatives of can be written in matrix form as shown in Equation (2); then, by the larger in all directions to locate the corner features within the target; (2) wherein denotes the pixel intensity of the (x, y) coordinates; denotes the pixel intensity after the translation of the (x, y) coordinates ; the translation filter denotes a small area centered on the corner points generated within the gray-scale contour region. By calculating the change in pixel density within this small area, sparse optical flow tracking of the corner points within the contour can be achieved; Next, calculate the affine transformation matrix between the homogeneous coordinates of the corner points in the current frame and the previous frame, and use this matrix to map the vertex coordinates of the contour in the previous frame to the current frame. Next, calculate the affine transformation matrix between the homogeneous coordinates of the corner points in the current frame and the previous frame, and use this matrix to map the vertex coordinates (x i , y i ) of the contour in the previous frame to the current frame, as shown in Equation (3): (3) where s 11 、s 12 、s 21 and s 22 refer to the scaling, shearing and rotation parameters; t 1 and t 2 refer to the translation parameters;( x frame , y frame ) and ( x last_frame , y last_frame ) refer to the vertex coordinates of the contour in the current frame and the previous frame respectively; When the instance segmentation model has a missed detection error in a certain frame, it is automatically corrected using ( x frame , y frame ).

8. The intelligent monitoring method for the hoisting progress interference of prefabricated components of an assembled building according to claim 6, characterized in that: The specific monitoring steps in step S6 are as follows: S6-1. The planned completion hoisting time for the current floor is T*, the moment when the component is first detected is T0, the moment when the previous component is in place is T1, the total number of detected components is n, and the total number of components required for the floor is N; S6-2. Execute the four modules S1~S5 to obtain the current component contour position and ID; S6-3. Determine whether the intersection over union of the component contour position in the previous and current frames is greater than 0.8 and lasts for 50 frames, and execute corresponding steps according to the judgment result: If the judgment result is True, the component is in the hoisting state, and at this time, return to step S6-2; If the judgment result is False, the component has been installed in place, record this moment as T2 and continue to execute the next step S6-4; S6-4. According to the existing data, calculate that the estimated hoisting completion time of the current floor is T = (T2 - T0) + (T2 - T1)(N - n); S6-5. Determine the magnitude relationship between the estimated hoisting completion time T and the planned hoisting completion time T*. If T > T*, the system detects interference in the hoisting delay of precast components, and the delay duration is T Delay = T - T*. Otherwise, there is no interference in the hoisting delay of precast components. Wait for the start of hoisting of the next component and return to step S6-1.

9. The intelligent monitoring method for the hoisting progress interference of prefabricated components of an assembled building according to claim 8, characterized in that: It also includes step S7, hoisting delay interference control; according to the monitored T evaluated Delay judge whether it exceeds the interference tolerance range of the prefabricated construction schedule; If the hoisting delay interference appears in the critical path of the prefabricated construction schedule, it is determined that this hoisting delay interference will cause delays in the entire construction schedule; if this hoisting delay interference does not appear in the critical path of the construction schedule, but T Delay is greater than the total float of the hoisting operation in the construction schedule, it is determined that this hoisting delay interference will also cause delays in the entire construction schedule; if this hoisting delay interference does not appear in the critical path of the construction schedule and T Delay is less than the total float of the hoisting operation in the construction schedule, but T Delay is greater than the free float of the hoisting operation in the construction schedule, it is determined that this hoisting delay interference will cause delays in its subsequent operations.

10. The intelligent monitoring method for the hoisting progress interference of prefabricated components of prefabricated buildings according to claim 1, characterized in that: In step S3, when the cache component set is full, the earliest cache record will be automatically deleted.

Images