Intelligent identification system and method for tower crane installation and disassembly process
Through the intelligent identification system of the tower crane installation and dismantling process, and using technologies such as neural networks and Kalman filters, real-time monitoring and intelligent identification of the tower crane installation and dismantling construction site is achieved, solving the problem that traditional monitoring methods cannot achieve real-time and accurate monitoring. Improve construction safety and efficiency.
Patent Information
- Application Number
- CN202410292208.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-14
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-03-14
Smart Images

Figure CN118397496B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of installation and disassembly of building hoisting machinery, and in particular to an intelligent recognition system and method for tower crane installation and disassembly procedures. Background Art
[0002] Tower crane installation and dismantling is an important part of engineering construction, but the construction site environment is complex, the operation is difficult, and safety accidents are prone to occur. Traditional monitoring methods mainly rely on manual observation and simple sensors, which cannot achieve real-time, accurate monitoring and intelligent identification. Summary of the invention
[0003] The purpose of the present invention is to provide a tower crane installation and disassembly process intelligent identification system and method, so as to improve the safety and efficiency of tower crane installation and disassembly construction.
[0004] To achieve this purpose, the tower crane installation and disassembly process intelligent identification system designed by the present invention includes: a data acquisition and preprocessing module for acquiring real-time video stream data of tower crane installation and disassembly, and preprocessing the real-time video stream data of tower crane installation and disassembly;
[0005] The tower crane basic component detection module uses a backbone neural network to extract feature maps of each tower crane basic component of different scales from the preprocessed video stream data frame. Then, a regression operation is performed on the feature maps of different scales, and non-maximum suppression is performed on the regression candidate frames of each tower crane basic component obtained by the regression operation, so as to obtain the final detection frame corresponding to each tower crane basic component, and then determine the category and position of each tower crane basic component;
[0006] The tower crane basic component tracking module is used to take the initial frame of the preprocessed video stream data frame as the starting point of target tracking, use the tower crane basic component detection module to detect the initial frame, determine the target category and position of each tower crane basic component in the initial frame, mark the tower crane basic component target in the initial frame, predict the possible position of the tower crane basic component in the initial frame on the next frame through the Kalman filter, use the tower crane basic component detection module to detect the next frame, obtain the corresponding tower crane basic component target and mark it, apply the Hungarian matching algorithm to calculate the correlation between the predicted position information of each basic component of the tower crane in the same frame and the actual target position information detected by the corresponding tower crane basic component detection module. If the correlation between the predicted position information of a basic component and the actual target position information detected by the corresponding tower crane basic component detection module exceeds the corresponding threshold, the tower crane basic component matching targets of the previous and next frames share the same number, and the numbering order of different tower crane basic components is determined according to the installation order. Each frame of the preprocessed video stream data frame is processed in a loop in this way to complete the tracking and detection of each tower crane basic component.
[0007] The tower crane installation and disassembly judgment module uses the target position change measurement function to judge the movement status of each tower crane basic component tracked in the entire video stream data frame, thereby judging the installation and disassembly process status of each tower crane basic component.
[0008] Through the technical solution of the present invention, real-time monitoring and intelligent identification of tower crane installation and disassembly construction sites can be achieved, thereby improving construction safety and efficiency. Specifically, the beneficial effects of the present invention include:
[0009] 1. The algorithm adopted by the present invention has low complexity, which realizes the rapid and accurate identification and tracking of basic components of tower crane installation and disassembly under complex construction environments. The improved algorithm has higher recognition accuracy and greatly reduces the occurrence of missed detection and false detection;
[0010] 2. Based on the actual tower crane installation and dismantling operations, an effective tower crane installation and dismantling process determination algorithm was designed. The effectiveness was verified through engineering application tests, providing a reliable scientific tool for the information monitoring of tower crane installation and dismantling;
[0011] 3. The method and system of the present invention can be widely used in various types of tower crane installation and disassembly construction sites, providing strong protection for engineering safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 It is a structural schematic diagram of the present invention;
[0013] Figure 2 It is a logic block diagram of the present invention. DETAILED DESCRIPTION
[0014] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0015] Example 1
[0016] like Figures 1-2 The tower crane installation and disassembly process intelligent identification system shown includes: a data acquisition and preprocessing module for acquiring real-time video stream data of tower crane installation and disassembly, and preprocessing the real-time video stream data of tower crane installation and disassembly;
[0017] The tower crane basic component detection module uses a backbone neural network to extract feature maps of each tower crane basic component of different scales from the preprocessed video stream data frame. Then, regression operations are performed on feature maps of different scales (the pixels on the feature map are directly mapped to the original image through the convolution corresponding receptive field for classification and positioning), and non-maximum suppression is performed on the regression candidate frames of each tower crane basic component obtained by the regression operation (each target will generate multiple detection frames through regression, resulting in duplication. It is necessary to remove redundant detection frames through non-maximum suppression and retain the detection frame with the best effect), so as to obtain the final detection frame corresponding to each tower crane basic component, and use the above final detection frame to determine the category and position of each tower crane basic component. The above process extracts feature maps of different scales, which allows the network to retain coarse and fine-grained features when learning features and optimize the learning effect of the algorithm; direct regression operations greatly reduce the complexity of the algorithm and improve the speed of the algorithm;
[0018] The tower crane basic component tracking module is used to take the initial frame of the preprocessed video stream data frame as the starting point of target tracking, use the tower crane basic component detection module to detect the initial frame, determine the target category and position of each tower crane basic component in the initial frame, mark the category, position and ID of the tower crane basic component target in the initial frame, predict the possible position of the tower crane basic component in the initial frame on the next frame through the Kalman filter, use the tower crane basic component detection module to detect the next frame, obtain the corresponding tower crane basic component target and mark the category, position and ID, and apply the Hungarian matching algorithm to match the predicted position information of each basic component of the tower crane in the same frame with the corresponding tower crane basic component The actual target position information obtained by the detection module is subjected to correlation calculation. If the correlation between the predicted position information of a basic component and the actual target position information detected by the corresponding tower crane basic component detection module exceeds the corresponding threshold, the matching targets of the tower crane basic components of the previous and next frames share the same number. The numbering sequence of different tower crane basic components is determined according to the installation sequence. Each frame of the pre-processed video stream data frame is processed in this way in a loop to complete the tracking detection of each tower crane basic component (although ordinary target detection can detect video streams, it performs target detection once per frame, and the targets detected in the previous and next frames have no logical connection. Tracking detection will continuously detect the same target to ensure that the detected target is the same component);
[0019] The tower crane installation and disassembly judgment module uses the target position change measurement function to judge the movement status of each tower crane basic component tracked in the entire video stream data frame, thereby judging the installation and disassembly process status of each tower crane basic component.
[0020] In the above technical solution, the Kalman filter uses the state transfer matrix to predict the current state from the state at the previous moment to obtain a priori estimates. The Kalman filter uses the state transfer matrix, the covariance of the predicted value, the observation matrix, and the covariance of the observed value to calculate the Kalman gain (used to decide whether to trust the predicted value or the observed value more), and combines the current predicted value and the current observed value to update the predicted value and obtain a posteriori estimate. At the same time, the Kalman filter will also update the error covariance for the next iteration.
[0021] In the above technical scheme, the specific method for preprocessing the real-time video stream data of tower crane installation and disassembly is: the real-time video stream data of tower crane installation and disassembly is firstly grayed using the weighted average method to reduce the parameter amount of the data to be processed, and then based on the Gaussian filter, the noise in the video stream frame is removed and the video stream frame is smoothed, and then the spatial transformation and channel transformation are used to enhance the features of the video stream frame, wherein the specific operations of the spatial transformation include random scaling, cropping and splicing, rotation and translation, and the channel transformation includes grayscale histogram equalization and channel grayscale level reduction.
[0022] In the above technical solution, the backbone neural network is a ResNet50 neural network, which fuses the feature maps of each tower crane basic component of different scales before and after each convolution as the input of the next convolution, so that the feature maps of each tower crane basic component of different scales will be fused with the upper feature map. The scale is the size of the feature map, and the image becomes a feature map after convolution of the convolutional neural network.
[0023] In the present invention, it is first necessary to perform target detection on the basic components of the tower crane, and the actual video stream data is a high-resolution, large-scale, global tower crane installation and disassembly video, so its components are all small target features, and the features are all fine-grained features. The original SSD (Single Shot MultiBox Detector) model cannot meet the actual needs of the method of the present invention. Therefore, the present invention performs two algorithm optimizations on SSD to improve its detection accuracy without reducing its detection efficiency.
[0024] The first optimization: backbone neural network optimization. The original feature extraction backbone network of SSD is VGG-16. Although the network can extract feature maps of different scales through different convolution operations, there is no good relationship coupling between the generation of low-resolution feature maps and high-resolution feature maps. The loss of features is hereditary and cannot be compensated. Therefore, the present invention introduces the idea of residual neural network and replaces VGG-16 with ResNet50 (the feature map will generate a smaller scale feature map after the feature extraction through convolution. ResNet50 designs a residual connection structure that can fuse the feature maps of different scales before and after each convolution as the input of the next convolution to reduce feature loss). Each feature map of different scales will perform feature fusion operation with the upper-layer feature map, reducing feature loss and improving the feature extraction ability of the neural network.
[0025] The second optimization: non-maximum suppression algorithm optimization. After completing the candidate box regression on different feature maps, the original non-maximum suppression (NMS) only relies on the direct intersection over union (IoU) of the candidate boxes to determine the list of candidate boxes to be suppressed. After determining the list of candidate boxes to be suppressed, the candidate boxes are retained and suppressed based on whether their classification confidence scores are the highest. This has a huge error for different target candidate boxes with high overlap, and the classification confidence of the target is only the result of the classification network, and has no direct positive correlation with the location information of the candidate boxes. Therefore, the present invention uses Soft-NMS to perform non-maximum suppression of regression candidate frames, sets the intersection-over-union ratio thresholds of each tower crane basic component regression candidate frame and the corresponding tower crane basic component target real frame (annotated frame), and determines the regression candidate frame A with the highest classification confidence (classification confidence refers to the possibility that the target may be a certain category, which is used to judge the reliability of classification) of each tower crane basic component in all regression candidate frames exceeding the intersection-over-union ratio threshold (the IoU threshold is set to 0.5) (exceeding the intersection-over-union ratio threshold indicates that the detection effect has a high accuracy), and then calculates the intersection-over-union ratios of the remaining regression candidate frames of each tower crane basic component and the regression candidate frame A (the intersection-over-union ratio is calculated for screening Candidate frame, can avoid directly removing all non-maximum frames during NMS, thereby improving the recall rate), according to the intersection and union ratio of the remaining regression candidate frames of each tower crane basic component and the regression candidate frame A, use Gaussian function to update the confidence score of the remaining regression candidate frames of each tower crane basic component (use Gaussian function to calculate the weight factor, the larger the IoU value, the smaller the output of Gaussian function, thereby reducing the confidence of the frame), traverse all the regression candidate frames of each tower crane basic component until the confidence scores of all the regression candidate frames of each tower crane basic component are updated, according to the set confidence score threshold, filter out the regression candidate frames greater than the confidence score threshold as the final detection frames corresponding to each tower crane basic component. The final detection frame corresponding to each tower crane basic component is generally one, and the confidence scores of other frames greater than the set confidence threshold are reduced by updating the Gaussian function, unless the overlapping occlusion is particularly serious, the tower crane basic component of the present invention will not have a particularly serious overlapping occlusion.
[0026] In the above technical scheme, the preprocessed tower crane installation and disassembly real-time video stream data frames are randomly divided into training frames and test frames in a ratio of 7:3. The training frames are used to train the target detection model, and the test frames are used to test the training effect of the target detection model. The network weight parameters of the target detection model are iterated according to the effect.
[0027] In the above technical solution, each basic component of the tower crane is judged and marked as a target of a category, and other features of the tower crane (all features except the basic components) are judged as background. The regression candidate boxes with feature similarity greater than the threshold are merged together for Soft-NMS, while the overlapping and occluded regression candidate boxes are subjected to Soft-NMS separately.
[0028] In the above technical scheme, for example, there is a great feature similarity between the base section and the standard section, and there is a great overlap and occlusion between the attachment rod and the attachment frame and the tower crane as a whole. Therefore, the present invention modifies the Soft-NMS that only performs non-maximum suppression processing on a single category to centralized processing of local categories, that is, the classification confidence scores of the base section and the standard section, which are extremely similar in features, are very high in multiple categories (such as the base section and the standard section have very similar features, so in the algorithm recognition result, the confidence scores of a certain base section determined as a base section and a standard section are very high, that is, the confidence scores of a certain target being considered as the basic components of several categories are very high) and the regression candidate boxes are merged together for Soft-NMS, and the regression candidate boxes with high confidence scores in the categories with severe overlap and occlusion are subjected to Soft-NMS separately. The final improved non-maximum suppression method not only greatly reduces missed detections, but also greatly reduces false detections, thereby improving the precision and recall of the algorithm.
[0029] The basic components of the tower crane include a tower crane hook, a foundation section, a standard section, a lifting frame, a slewing assembly, a tower head, a balance arm, a lifting arm, a counterweight, a lifting mechanism, an attachment rod, and an attachment frame. The basic components are manually labeled to be different types of targets, and each basic component of the tower crane is a target of one category.
[0030] The present invention adopts the tower crane installation and disassembly component tracking based on Deep Sort. Deep Sort is a target tracking algorithm. It takes the result of target detection as input, marks the target and tracks the change of the position information of the target in the process of time series iteration. Specifically, in the present invention, the initial frame (the first frame of the video stream) is first used as the starting point of the task input, and the target obtained by the improved SSD network detection is marked (its category, position and ID are manually marked). Compared with the result of target detection, the mark not only saves the category and position information of the target, but also records its task number. Then, the Kalman filter is used to predict the possible position of the tower crane basic component in the current frame on the next frame image, and the improved SSD network detects the next frame image. Then, the Hungarian matching algorithm is applied to calculate the correlation between the prediction information of the tower crane basic component and the target detection information. If the match is successful, the matching targets of the previous and next frames share the same number. In this way, each frame of the video stream is cycled to complete the dynamic tracking of the tower crane basic component. If the match is unsuccessful, the same number ID is not shared.
[0031] In the above technical solution, the specific method for the tower crane installation and disassembly judgment module to judge the installation and disassembly process status of each tower crane basic component is:
[0032] Use the following method to determine the installation process status of each tower crane basic component:
[0033] Based on the identified basic components of the tower crane, a mapping relationship between the basic components of the tower crane and the tower crane installation process is established. According to the tower crane installation process rules, the candidate process sequence interval for the current frame process judgment is narrowed down (for example, when a rotary assembly is detected, it may only appear in the installation of the rotary assembly or subsequent installation processes, and the previous process interval may not be directly analyzed and judged. Similar principles can be used to narrow the candidate judgment process interval. Narrowing the process interval can reduce the amount of calculation and improve the efficiency of the algorithm). Secondly, the target position change measurement function is used to determine whether the currently tracked basic components of the tower crane are in motion according to the displacement amplitude of the detection frame. state, and calculate the total number of frames of stillness and movement of the basic components of the tower crane during the dynamic tracking process (from the first frame of the start of movement to the first frame of the stop of movement, the number of frames in the middle is the total number of frames of movement, and the calculation principle of the total number of still frames is the same, and the total number of frames is used to calculate the time of movement and stillness), and convert it into time units. Then, the installation process state of each basic component of the tower crane is determined by using the position relationship calculation function of different numbered targets. The position relationship calculation function of different numbered targets is based on the intersection-and-combination ratio calculation formula, which is used to judge the position relationship between targets with different numbers. The intersection-and-combination ratio = the ratio of the intersection area of two targets to the combined area of two targets;
[0034] The following method is used to determine the dismantling process status of each tower crane basic component:
[0035] Based on the identified basic components of tower cranes, a mapping relationship between the basic components of tower cranes and the dismantling processes of tower cranes is established. According to the tower crane dismantling process rules, the candidate process sequence interval for the current frame process determination is narrowed. Secondly, the target position change measurement function is used to determine whether the currently tracked basic components of tower cranes are in motion according to the displacement amplitude of the detection frame, and the total number of static and moving frames of the basic components of tower cranes during dynamic tracking is calculated and converted into time units. Then, the dismantling process status of each basic component of tower crane is determined using the target position relationship calculation function with different numbers.
[0036] In the above technical solution, the installation process status of each tower crane basic component is determined by using the different numbered target position relationship calculation function;
[0037] If, according to the numbering of the installation and disassembly sequence, the positional relationship between the basic component of the moving tower crane of the current number and the basic component of the moving tower crane of the previous number does not have an intersection-and-combination ratio, it is indicated that the basic component of the moving tower crane of the current number is waiting for installation; if there is a certain intersection-and-combination ratio but the motion state of the basic component of the moving tower crane of the current number is still in motion, the basic component of the moving tower crane of the current number is also indicated as waiting for installation; if the positional relationship between the basic component of the moving tower crane of the current number and the basic component of the moving tower crane of the previous number is that there is a certain intersection-and-combination ratio (set to 0.1 in this embodiment), and the basic components of the moving tower cranes of these two numbers have changed from a moving state to a static state, and the static time reaches a set time, it is determined that the state of the basic component of the moving tower crane of the current number is being installed, until a new numbered tower crane basic component begins to move, then it is determined that the installation of the basic component of the moving tower crane of the current number is completed;
[0038] The dismantling process status of each tower crane basic component is determined by using the calculation function of the position relationship of different numbered targets;
[0039] If the current tracking tower crane basic component is in motion, then the status of the current tracking tower crane basic component is determined to be being dismantled; if, according to the number of the installation and dismantling sequence, the previous numbered tracking tower crane basic component has changed from motion to static state, and the static time reaches the set time, then the status of the current tracking tower crane basic component is determined to be dismantling completed; until the next numbered tracking tower crane basic component begins to change from static to motion, then the status of the next numbered tracking tower crane basic component is determined to be waiting for dismantling.
[0040] The above technical solution strictly follows the construction process characteristics of actual tower crane installation and disassembly to design the process determination algorithm. Through the target collision matching mechanism, the position information of the tracking moving target and the static and moving time information are calculated to complete the target matching.
[0041] Example 2
[0042] A tower crane installation and disassembly process intelligent identification method comprises the following steps:
[0043] Step 1: Install a high-definition camera at the tower crane installation and dismantling construction site to collect real-time video stream data of the tower crane installation and dismantling, and pre-process the real-time video stream data of the tower crane installation and dismantling;
[0044] Step 2: The backbone neural network is used to extract the feature maps of each tower crane basic component of different scales from the preprocessed video stream data frame. Then, a regression operation is performed on the feature maps of different scales, and non-maximum suppression is performed on the regression candidate frames of each tower crane basic component obtained by the regression operation, so as to obtain the final detection frame corresponding to each tower crane basic component, and then determine the category and position of each tower crane basic component;
[0045] Step 3: Taking the initial frame of the pre-processed video stream data frame as the starting point of target tracking, the tower crane basic component detection module is used to detect the initial frame, determine the target category and position of each tower crane basic component in the initial frame, mark the tower crane basic component target in the initial frame, predict the possible position of the tower crane basic component in the initial frame on the next frame through the Kalman filter, use the tower crane basic component detection module to detect the next frame, obtain the corresponding tower crane basic component target and mark it, and use the Hungarian matching algorithm to calculate the correlation between the predicted position information of each basic component of the tower crane in the same frame and the actual target position information detected by the corresponding tower crane basic component detection module. If the correlation between the predicted position information of a basic component and the actual target position information detected by the corresponding tower crane basic component detection module exceeds the corresponding threshold, the tower crane basic component matching targets of the previous and next frames share the same number, and the numbering order of different tower crane basic components is determined according to the installation order. Each frame of the pre-processed video stream data frame is processed in this way in a loop to complete the tracking and detection of each tower crane basic component;
[0046] Step 4: Use the target position change measurement function to determine the movement status of each tower crane basic component tracked in the entire video stream data frame, thereby determining the installation and disassembly process status of each tower crane basic component.
[0047] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.
Claims
1. An intelligent identification system for tower crane installation and disassembly procedures, It is characterized in that include: The data acquisition and preprocessing module is used to acquire the real-time video stream data of tower crane installation and disassembly, and preprocess the real-time video stream data of tower crane installation and disassembly; The tower crane basic component detection module uses a backbone neural network to extract feature maps of each tower crane basic component of different scales from the preprocessed video stream data frame, performs regression operations on the feature maps of different scales, and performs non-maximum suppression on the regression candidate frames of each tower crane basic component obtained by the regression operation, thereby obtaining the final detection frame corresponding to each tower crane basic component, and then determining the category and position of each tower crane basic component; The tower crane basic component tracking module is used to take the initial frame of the pre-processed video stream data frame as the starting point of target tracking, use the tower crane basic component detection module to detect the initial frame, determine the target category and position of each tower crane basic component in the initial frame, mark the tower crane basic component target in the initial frame, predict the possible position of the tower crane basic component in the initial frame on the next frame, use the tower crane basic component detection module to detect the next frame, obtain the corresponding tower crane basic component target and mark it, calculate the correlation between the predicted position information of each basic component of the tower crane in the same frame and the actual target position information detected by the corresponding tower crane basic component detection module, if the correlation between the predicted position information of a basic component and the actual target position information detected by the corresponding tower crane basic component detection module exceeds the corresponding threshold, then the tower crane basic component matching targets of the previous and next frames share the same number, and the numbering order of different tower crane basic components is determined according to the installation order, so that each frame of the pre-processed video stream data frame is processed cyclically to complete the tracking and detection of each tower crane basic component; The tower crane installation and disassembly judgment module uses the target position change measurement function to judge the movement status of each tower crane basic component tracked in the entire video stream data frame, thereby judging the installation and disassembly process status of each tower crane basic component.
2. The tower crane installation and disassembly process intelligent identification system according to claim 1, It is characterized in that The specific method for preprocessing the real-time video stream data of tower crane installation and disassembly is as follows: the real-time video stream data of tower crane installation and disassembly is firstly grayed out using the weighted average method, and then the noise in the video stream frame is removed based on the Gaussian filter, and then the video stream frame is feature enhanced using spatial transformation and channel transformation, wherein the specific operations of spatial transformation include random scaling, cropping and splicing, rotation and translation, and the channel transformation includes grayscale histogram equalization and channel grayscale level reduction.
3. According to the tower crane installation and disassembly process intelligent identification system of claim 1, It is characterized in that The backbone neural network is a ResNet50 neural network, which fuses the feature maps of each tower crane basic component of different scales before and after each convolution as the input of the next convolution, so that the feature maps of each tower crane basic component of each different scale will perform feature fusion operation with the upper feature map.
4. According to claim 1, the tower crane installation and disassembly process intelligent identification system, It is characterized in that The specific method for performing non-maximum suppression on all regression candidate frames of each tower crane basic component is as follows: Soft-NMS is used to perform non-maximum suppression of regression candidate frames, and the intersection-over-union ratio thresholds of the regression candidate frames of each tower crane basic component and the corresponding tower crane basic component target true frame are set. Among all the regression candidate frames of each tower crane basic component that exceed the intersection-over-union ratio threshold, the regression candidate frame A with the highest classification confidence score of each tower crane basic component is determined, and then the intersection-over-union ratios of the remaining regression candidate frames of each tower crane basic component and the regression candidate frame A are calculated. According to the intersection-over-union ratios of the remaining regression candidate frames of each tower crane basic component and the regression candidate frame A, the confidence scores of the remaining regression candidate frames of each tower crane basic component are updated using the Gaussian function, and all the regression candidate frames of each tower crane basic component are traversed until the confidence scores of all the regression candidate frames of each tower crane basic component are updated. According to the set confidence score threshold, the regression candidate frames greater than the confidence score threshold are filtered out as the final detection frames corresponding to each tower crane basic component.
5. According to the tower crane installation and disassembly process intelligent identification system of claim 1, It is characterized in that The preprocessed tower crane installation and disassembly real-time video stream data frames are randomly divided into training frames and test frames in a ratio of 7:
3. The training frames are used to train the network weight parameters of the target detection model to determine the target detection model. The test frames are used to test the training effect of the target detection model, and the network weight parameters of the target detection model are iterated according to the effect.
6. According to claim 4, the tower crane installation and disassembly process intelligent identification system, It is characterized in that Each basic component of the tower crane is determined and labeled as a target of a category, and other features of the tower crane are determined as background. The regression candidate boxes with feature similarity greater than the threshold are merged together for Soft-NMS, while the overlapping and occluded regression candidate boxes are subjected to Soft-NMS separately.
7. According to claim 1, the tower crane installation and disassembly process intelligent identification system, It is characterized in that The tower crane basic component tracking module is used to predict the possible position of the tower crane basic component in the initial frame in the next frame through a Kalman filter; The tower crane basic component tracking module is used to apply the Hungarian matching algorithm to calculate the correlation between the predicted position information of each basic component of the tower crane in the same frame and the actual target position information detected by the corresponding tower crane basic component detection module.
8. According to claim 1, the tower crane installation and disassembly process intelligent identification system, It is characterized in that The specific method for the tower crane installation and disassembly judgment module to judge the installation and disassembly process status of each tower crane basic component is: Use the following method to determine the installation process status of each tower crane basic component: According to the identified basic components of tower crane, the mapping relationship between the basic components of tower crane and the installation process of tower crane is established, and the candidate process sequence interval of the current frame process determination is narrowed according to the tower crane installation process rules. Secondly, the target position change measurement function is used to determine whether the currently tracked basic components of tower crane are in motion, and the total number of static and moving frames of the basic components of tower crane in the dynamic tracking process is calculated and converted into time units. Then, the installation process status of each basic component of tower crane is determined by using the target position relationship calculation function with different numbers; The following method is used to determine the dismantling process status of each tower crane basic component: Based on the identified basic components of tower cranes, a mapping relationship between the basic components of tower cranes and the dismantling processes of tower cranes is established. According to the tower crane dismantling process rules, the candidate process sequence interval for the current frame process determination is narrowed. Secondly, the target position change measurement function is used to determine whether the currently tracked basic components of tower cranes are in motion, and the total number of static and moving frames of the basic components of tower cranes during dynamic tracking is calculated and converted into time units. Then, the dismantling process status of each basic component of tower cranes is determined using the target position relationship calculation function with different numbers.
9. According to the tower crane installation and disassembly process intelligent identification system of claim 8, It is characterized in that The installation process status of each tower crane basic component is determined by using the target position relationship calculation function with different numbers; If, according to the numbering of the installation and disassembly sequence, the positional relationship between the basic component of a moving tower crane with the current number and the basic component of a moving tower crane with the previous number does not have an intersection-and-combination ratio, it indicates that the basic component of a moving tower crane with the current number is waiting for installation; if there is a certain intersection-and-combination ratio but the motion state of the basic component of a moving tower crane with the current number is still in motion, the basic component of a moving tower crane with the current number is also indicated as waiting for installation; if the positional relationship between the basic component of a moving tower crane with the current number and the basic component of a moving tower crane with the previous number has a certain intersection-and-combination ratio, and both basic components of moving tower cranes with these two numbers have changed from a moving state to a static state, and the static time reaches a set time, it is determined that the state of the basic component of a moving tower crane with the current number is being installed, until a basic component of a tower crane with a new number starts to move, then it is determined that the installation of the basic component of a moving tower crane with the current number is completed; The dismantling process status of each tower crane basic component is determined by using the calculation function of the position relationship of different numbered targets; If the current tracking tower crane basic component is in motion, then the status of the current tracking tower crane basic component is determined to be being dismantled; if, according to the number of the installation and dismantling sequence, the previous numbered tracking tower crane basic component has changed from motion to static state, and the static time reaches the set time, then the status of the current tracking tower crane basic component is determined to be dismantling completed; until the next numbered tracking tower crane basic component begins to change from static to motion, then the status of the next numbered tracking tower crane basic component is determined to be waiting for dismantling.
10. A method for intelligent identification of tower crane installation and disassembly procedures, It is characterized in that It includes the following steps: Step 1: Collect real-time video stream data of tower crane installation and disassembly, and pre-process the real-time video stream data of tower crane installation and disassembly; Step 2: The backbone neural network is used to extract the feature maps of each tower crane basic component of different scales from the preprocessed video stream data frame. Then, a regression operation is performed on the feature maps of different scales, and non-maximum suppression is performed on the regression candidate frames of each tower crane basic component obtained by the regression operation, so as to obtain the final detection frame corresponding to each tower crane basic component, and then determine the category and position of each tower crane basic component; Step 3: Taking the initial frame of the pre-processed video stream data frame as the starting point of target tracking, the tower crane basic component detection module is used to detect the initial frame, determine the target category and position of each tower crane basic component in the initial frame, mark the tower crane basic component target in the initial frame, predict the possible position of the tower crane basic component in the initial frame on the next frame through the Kalman filter, use the tower crane basic component detection module to detect the next frame, obtain the corresponding tower crane basic component target and mark it, and use the Hungarian matching algorithm to calculate the correlation between the predicted position information of each basic component of the tower crane in the same frame and the actual target position information detected by the corresponding tower crane basic component detection module. If the correlation between the predicted position information of a basic component and the actual target position information detected by the corresponding tower crane basic component detection module exceeds the corresponding threshold, the tower crane basic component matching targets of the previous and next frames share the same number, and the numbering order of different tower crane basic components is determined according to the installation order. Each frame of the pre-processed video stream data frame is processed in this way in a loop to complete the tracking and detection of each tower crane basic component; Step 4: Use the target position change measurement function to determine the movement status of each tower crane basic component tracked in the entire video stream data frame, thereby determining the installation and disassembly process status of each tower crane basic component.
