Detection method for large construction machinery based on key points
By combining object detection and key point detection technology, using the Yolov8-Pose model and Soft-NMS algorithm, the identification and distinction problems, dynamic characteristics and occlusion problems in large-scale construction machinery detection are solved, and the accurate identification of the fuselage and robotic arms is achieved. and tracking improves the accuracy and stability of detection and supports intelligent management.
Patent Information
- Application Number
- CN202411051294.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-19
AI Technical Summary
The prior art has difficulties in identifying and distinguishing, challenges in dynamic characteristics and varied postures, and target occlusion problems in large-scale construction machinery inspections, resulting in the impact of detection accuracy and stability.
The combination of object detection and key point detection is adopted to identify and track large-scale construction machinery and robotic arms, and the Yolov8-Pose model and Soft-NMS algorithm are used to realize the precise positioning and attitude capture of the fuselage and robotic arms. Reduce confusion and occlusion interference.
It improves the accuracy and stability of large-scale construction machinery inspection, reduces the risk of misidentification, enhances the safety and efficiency of the construction site, and provides technical support for intelligent management.
Smart Images

Figure CN118982707A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of large-scale construction machinery detection, and specifically relates to a key point-based large-scale construction machinery detection method, a computer-readable storage medium, a computer program product and a computer device. Background Art
[0002] At large construction sites, accurate distance and length measurement of large construction machinery and its mechanical arms is the key to construction planning, safety monitoring and operational efficiency. In recent years, with the rapid development of computer vision technology and machine learning technology, the identification of large construction machinery has ushered in new development opportunities.
[0003] Although some related technologies have been disclosed, such as the power transmission channel hidden danger detection method based on improved YOLOv7 disclosed in CN115797357A and the power transmission line construction machinery hidden danger detection method based on deep learning shown in CN110705414A, they have promoted the technological progress in this field to a certain extent. However, the existing technologies still face multiple challenges in practical applications and are in urgent need of further breakthroughs.
[0004] Specifically, these challenges are mainly reflected in the following aspects: First, the fuselage (referring to the main part of the construction machinery, such as the body of the crane and excavator) and the mechanical arm (the movable part of the construction machinery, such as the lifting arm of the crane and the bucket arm of the excavator) of large-scale construction machinery have high morphological similarity. In addition, the interference of the complex background of the construction site (including other mechanical equipment, building materials and personnel activities) makes it extremely difficult to directly identify and distinguish the two at the same time. The existing technology often uses a single target detection method to deal with the fuselage and the mechanical arm, and fails to fully consider the differences in the morphology and dynamic characteristics of the two. This not only increases the risk of misidentification, but also reduces the accuracy of detection. Secondly, the mechanical arm will continue to move and change its posture during the construction process, which makes it extremely challenging to accurately capture and measure its position. The existing technology is difficult to adapt to this dynamic environment, resulting in the accuracy and stability of the detection results being affected. Thirdly, target occlusion is a common problem at the construction site. When dealing with the situation where the fuselage is occluded, the existing technology often cannot effectively reduce the interference of the overlapping area, resulting in a decrease in the accuracy of the fuselage detection.
[0005] In summary, existing technologies have many technical problems in large-scale construction machinery detection, including identification and differentiation difficulties, challenges of dynamic characteristics and changing postures, and target occlusion. These problems need to be solved urgently to improve the accuracy and efficiency of construction planning, safety monitoring, and operational efficiency. Summary of the invention
[0006] The purpose of the present invention is to provide a key point-based detection method for large construction machinery, aiming to use two strategies, target detection and key point detection, to achieve accurate recognition and tracking of large construction machinery and its mechanical arm. The method quickly and accurately locates the position of the construction machinery and captures the movement and posture changes of the mechanical arm in real time to ensure the stability and accuracy of the detection results. In addition, it pays special attention to reducing the confusion between the fuselage and the mechanical arm during detection, improving the recognition accuracy, and adopts the Soft-NMS algorithm to solve the target occlusion problem, further optimizing the accuracy of the fuselage detection.
[0007] The embodiment of the present invention provides a key point-based detection method for large-scale construction machinery, comprising the following steps:
[0008] S1. Acquire historical images of large-scale construction machinery, where the historical images include the body and mechanical arm of the large-scale construction machinery;
[0009] S2. Annotate the historical images, including the target bounding box and the key point positions of the robotic arm, wherein the target bounding box covers the entire range of the fuselage and the robotic arm, and establish a training data set;
[0010] S3, using the Yolov8-Pose model as the basic model, using the training data set to train the Yolov8-Pose model to obtain a large-scale construction machinery detection model;
[0011] S4, using the trained large-scale construction machinery detection model to perform target detection on the large-scale construction machinery image to be detected, and obtain the target bounding box and the key point position of the robotic arm;
[0012] S5, post-process the target bounding box detected in S4 using the Soft-NMS algorithm to obtain a target detection box;
[0013] S6. Calculate the convex hull of the robot key points detected in S4 to obtain a robot detection frame.
[0014] By means of the above technical solution, the present invention has at least the following advantages:
[0015] (1) The present invention adopts different recognition strategies for the entire large-scale construction machinery and the mechanical arm. For the entire construction machinery, recognition is performed by target detection, which helps to quickly and accurately locate the position of the construction machinery, thereby quickly obtaining the overall position information of the construction machinery; for the mechanical arm, recognition is performed by key point detection. This method can focus on the key parts of the mechanical arm, such as the two ends and joints of the mechanical arm, so as to effectively cope with the movement and posture changes of the mechanical arm, accurately capture the position of the mechanical arm in real time, and maintain the stability and accuracy of the detection results in a dynamic environment.
[0016] (2) By using two different identification methods for the entire large construction machinery and the mechanical arm, the present invention significantly reduces the confusion between the machine body and the mechanical arm when detecting, so that the mechanical arm can be accurately identified and located. This is particularly effective at construction sites with similar shapes and complex backgrounds, greatly reducing the risk of misidentification and providing a strong technical guarantee for the safety and efficiency of the construction site.
[0017] (3) The present invention achieves synchronous tracking of large-scale construction machinery and its robotic arm by combining target detection and key point detection technology, that is, real-time tracking and understanding of the precise position and posture of the construction machinery and the robotic arm. This innovation is of great significance for accurate distance measurement and length measurement at the construction site. By synchronously tracking the construction machinery and the robotic arm, the tracking data can be used for high-precision measurements, such as determining the extended length of the robotic arm, the distance between the construction machinery and the target object, etc. These data provide strong support for construction planning, quality control, safety monitoring and efficiency improvement.
[0018] (4) To address the target occlusion problem, the present invention uses the Soft-NMS algorithm to post-process the target bounding box, thereby alleviating the interference caused by target occlusion by reducing the confidence of the overlapping area, thereby further improving the accuracy of target detection.
[0019] (5) By calculating the convex hull of the key points of the robot arm to obtain the robot arm detection frame, the present invention can not only more accurately define the actual range of the robot arm and flexibly adapt to its posture changes, but also effectively reduce background interference and optimize the use of computing resources. At the same time, in complex construction environments, it can also enhance the stability of robot arm recognition, and maintain relatively accurate detection even in the case of partial occlusion or light changes, thereby improving the performance and efficiency of the recognition system as a whole.
[0020] (6) The present invention utilizes computer vision and machine learning technologies to achieve automated detection of large-scale construction machinery, providing the possibility for intelligent management of construction sites, reducing manpower dependence, and improving management efficiency.
[0021] Preferably, an embodiment of the present invention also discloses that in S2, the key points of the robotic arm include a head end point, a middle point and an end point, the head end point is a point at the head end position of the robotic arm, the end point is a point at the end position of the robotic arm, and the middle point is located between the head end point and the end point.
[0022] Preferably, the embodiment of the present invention further discloses that the intermediate point is a joint point of the robot arm. Using the joint point of the robot arm as a key point for detection can improve the accuracy and efficiency of detection, enhance the generalization ability of the model, and support the execution of more complex tasks.
[0023] Preferably, the embodiment of the present invention further discloses that the number of the key points of the mechanical arm is three. By preferably annotating the mechanical arm with three key points, the model complexity can be reduced, the detection efficiency can be improved, and the risk of overfitting can be reduced while maintaining the detection accuracy, thereby making it easier to implement and deploy.
[0024] Preferably, the embodiment of the present invention further discloses that the specific steps of S5 include:
[0025] S51, input the target bounding box list boxes and the corresponding confidence list confidence, as well as the number of bounding boxes N detected by the large construction machinery detection model in S4;
[0026] S52, initialize the filtered bounding box list filtered_boxes to be empty;
[0027] S53. For each bounding box i from 1 to N, perform the following steps:
[0028] a. Find the target bounding box max_box with the highest confidence and its corresponding index max_index in the current remaining bounding boxes;
[0029] b. Add max_box to the filtered_boxes list as the currently retained bounding box;
[0030] c. Calculate the intersection-over-union ratio of max_box and all other target bounding boxes to get the intersection-over-union ratio list IoUs;
[0031] d. For each target bounding box j from 1 to N, if the intersection and union ratio IoUs between the jth target bounding box and max_box is j If it is greater than the set threshold, the confidence of the jth target bounding box is updated j :
[0032]
[0033] Where σ represents the variance in the Gaussian function;
[0034] S54, if the confidence confidence[max_index] of max_box is greater than the set confidence threshold, continue to process the remaining target bounding boxes from step S53;
[0035] S55. Return the filtered_boxes list as the filtered bounding box result, that is, the target detection box.
[0036] Preferably, the embodiment of the present invention further discloses that the specific steps of S6 include:
[0037] S61, calculating the minimum convex polygon formed by the key points of the robot arm;
[0038] S62. Based on the minimum convex polygon, determine a robot arm detection frame, where the robot arm detection frame is a minimum area rectangle whose length or width direction is parallel or perpendicular to the boundary of the large construction machinery image to be detected and can completely contain the convex polygon.
[0039] Based on the same inventive concept, an embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned key point-based large construction machinery detection method is implemented.
[0040] Based on the same inventive concept, an embodiment of the present invention further discloses a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the above-mentioned key point-based large construction machinery detection method.
[0041] Based on the same inventive concept, an embodiment of the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned key point-based large construction machinery detection method. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a flow chart of the key point-based detection method for large construction machinery in Example 1.
[0043] Figure 2 This is a schematic diagram of annotation of the first scene of the historical image in Example 1.
[0044] Figure 3 This is a schematic diagram of annotation of the second scene of the historical image in Example 1.
[0045] Figure 4 This is a diagram of the recognition results of the first scenario of the key point-based large construction machinery detection method in Example 1.
[0046] Figure 5 This is a diagram of the recognition results in the second scenario of the key point-based large construction machinery detection method in Example 1.
[0047] Figure 6 This is a comparison chart of the recognition results in the third scenario in Example 1, where: Figure 6 a is the recognition result diagram of the prior art. Figure 6 b is the recognition result diagram of the large construction machinery detection method based on key points.
[0048] Among them: 1-target detection box, 2-robotic arm detection box, 3-first endpoint, 4-middle point, 5-end point, 6-target bounding box. DETAILED DESCRIPTION
[0049] The embodiment of the present invention provides a key point-based large-scale construction machinery detection method, which combines target detection and key point detection technology to accurately identify and track large-scale construction machinery and its robotic arms. By adopting the strategies of target detection for the whole body and key point detection for the robotic arm, efficient distinction between the fuselage and the robotic arm is achieved, which significantly reduces confusion during detection and improves recognition accuracy. In addition, the Soft-NMS algorithm is introduced to solve the target occlusion problem, further optimizing the accuracy of target detection. Convex hull calculation of the key points of the robotic arm can more accurately define the range of the robotic arm and reduce background interference. The embodiment of the present invention not only improves the safety and efficiency of the construction site, but also provides a solid technical foundation for intelligent management, reduces manpower dependence, and improves overall management efficiency.
[0050] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0051] like Figure 1 As shown, in Example 1, a detection method for large construction machinery based on key points includes the following steps:
[0052] S1. Acquire historical images of large-scale construction machinery, where the historical images include a body and a mechanical arm of the large-scale construction machinery.
[0053] Large-scale construction machinery includes but is not limited to cranes, excavators, etc. The fuselage refers to the main part of the construction machinery, such as the body of a crane or excavator, etc. The mechanical arm refers to the movable part of the construction machinery, such as the lifting arm of a crane and the bucket arm of an excavator.
[0054] S2. Annotate the historical images, including the annotation of the target bounding box 6 and the key point positions of the robotic arm. The target bounding box 6 covers the entire range of the fuselage and the robotic arm, and establish a training data set.
[0055] like Figure 2 , Figure 3 As shown, the key points of the robot arm include a head end point 3, a middle point 4 and an end point 5. The head end point 3 is a point at the head end position of the robot arm, the end point 5 is a point at the end position of the robot arm, and the middle point 4 is located between the head end point 3 and the end point 5.
[0056] Specifically, the first end point 3 usually represents the connection position between the robot arm and the fuselage, such as Figure 2 A point at the lowest position of the crane, or Figure 3The end point 5 usually represents the position of the mechanical arm away from the body, such as Figure 2 A point at the highest position of the crane, or Figure 3 A point at the center bucket position.
[0057] The intermediate point 4 is the joint point of the robot arm. Figure 2 A point at the connection position of two adjacent telescopic arms of the crane. Using the joint points of the robot arm as key points for detection can improve the accuracy and efficiency of detection, enhance the generalization ability of the model, and support more complex task execution, as follows:
[0058] (1) Improve detection accuracy: The joint point is the most critical part of the robot arm movement and the area with the greatest morphological changes. Detecting the joint point as a key point can more accurately capture the dynamic changes of the robot arm, thereby improving the accuracy of robot arm detection.
[0059] (2) Simplify labeling and data processing: Joint points are usually one of the most prominent and easily identifiable features on a robotic arm. Using joint points as key points can simplify the data labeling process and reduce the complexity of data processing and model training.
[0060] (3) Enhance the generalization ability of the model: Since the joints of the robot are the core of its movement, detecting these points can help the model better understand the overall structure and movement pattern of the robot. This helps to improve the generalization ability of the model, enabling it to cope with robots of different shapes and postures.
[0061] (4) Support for more complex tasks: By accurately detecting joint points, more complex tasks can be further supported, such as posture estimation of the robotic arm, motion trajectory prediction, etc. This is crucial for achieving advanced automation and intelligence of construction machinery.
[0062] (5) Reduce false positive rates: Due to the importance of joints in the motion of the robot, accurately detecting these points helps reduce false positive rates. When the model can accurately identify joints, it is more likely to correctly identify the entire robot arm, thereby reducing the number of instances where other objects are mistakenly identified as the robot arm.
[0063] Preferably, the number of the key points of the robot arm in this embodiment is three. By selecting three key points to annotate the robot arm, the model complexity can be reduced, the detection efficiency can be improved, and the risk of overfitting can be reduced while maintaining the detection accuracy, so that it is easier to implement and deploy, as follows:
[0064] (1) Simplify model complexity: Selecting three key points can reduce the amount of data that the model needs to process, thereby simplifying the complexity of the model. This helps reduce the demand for computing resources, improves detection speed, and makes the model more lightweight and efficient.
[0065] (2) Improve detection efficiency: Fewer key points means that the model needs to search and match fewer feature points when performing key point detection, thereby speeding up the detection process. This is especially important for application scenarios with high real-time requirements.
[0066] (3) Reduce the risk of overfitting: When training a model, a smaller number of key points may reduce the risk of overfitting because the model will not be overly dependent on a large number of specific feature points. This helps improve the generalization ability of the model and makes it more robust when dealing with robotic arms of different shapes and postures.
[0067] (4) Easy to implement and deploy: Using fewer key points simplifies the algorithm implementation process, making the model easier to integrate into existing systems. At the same time, it also reduces the resources required to store and transmit key point data, making it easier to deploy in resource-constrained environments.
[0068] S3. Take the Yolov8-Pose model as the basic model and use the training data set to train the Yolov8-Pose model to learn the target detection of the fuselage and the key point positioning of the robotic arm, and obtain a large-scale construction machinery detection model.
[0069] S4. Use the trained large-scale construction machinery detection model to perform target detection on the large-scale construction machinery image to be detected, and obtain the target bounding box and the key point position of the robotic arm.
[0070] S5: For target occlusion problem, the Soft-NMS algorithm is used to post-process the target bounding box detected in S4 to obtain target detection box 1. The specific steps include:
[0071] S51, input the target bounding box list boxes and the corresponding confidence list confidence, as well as the number of bounding boxes N detected by the large construction machinery detection model in S4;
[0072] S52, initialize the filtered bounding box list filtered_boxes to be empty;
[0073] S53. For each bounding box i from 1 to N, perform the following steps:
[0074] a. Find the target bounding box max_box with the highest confidence and its corresponding index max_index in the current remaining bounding boxes;
[0075] b. Add max_box to the filtered_boxes list as the currently retained bounding box;
[0076] c. Calculate the intersection-over-union ratio of max_box and all other target bounding boxes to get the intersection-over-union ratio list IoUs;
[0077] d. For each target bounding box j from 1 to N, if the intersection and union ratio IoUs between the jth target bounding box and max_box is j If it is greater than the set threshold, the confidence of the jth target bounding box is updated j :
[0078]
[0079] Wherein, σ represents the variance in the Gaussian function, and in this embodiment, it is taken as 0.5;
[0080] S54. If the confidence confidence[max_index] of max_box is greater than the set confidence threshold, continue to process the remaining target bounding boxes starting from step S53.
[0081] S55. Return the filtered_boxes list as the filtered bounding box result, that is, the target detection box 1.
[0082] S6, performing convex hull calculation on the key points of the robot arm detected in S4 to obtain a robot arm detection frame 2. The specific steps include:
[0083] S61, calculating the minimum convex polygon formed by the key points of the robot arm;
[0084] S62. Based on the minimum convex polygon, determine a robot arm detection frame 2, where the robot arm detection frame 2 is a minimum area rectangle whose length or width direction is parallel or perpendicular to the boundary of the large construction machinery image to be detected, and can completely contain the convex polygon.
[0085] It should be noted that in practice, there is an extremely small probability that all the robot arm key points will be collinear. In this case, the robot arm key points are connected into a line segment, and based on the line segment, the robot arm detection frame is determined. The robot arm detection frame is the minimum area rectangle whose length or width direction is parallel or perpendicular to the boundary of the large construction machinery image to be detected, and can completely contain the line segment.
[0086] Figure 4 , Figure 5 The recognition result diagram of the key point-based large construction machinery detection method in this embodiment is shown. The target detection frame 1 and the mechanical arm detection frame 2 can be used for subsequent distance measurement and length measurement.
[0087] Figure 6 FIG. 4 shows a comparison chart of the recognition results in the third scenario of this embodiment. Figure 6In the scene shown, there are three large construction machines, and there is severe occlusion between them. There are interferences such as bridges and trees in the background. The similarity between the robot arm and the fuselage is high, which makes it difficult to identify. Figure 6 As shown in FIG. 1 , the conventional technology misses the detection of the two obscured mechanical arms of the construction machinery; however, the detection method of the large construction machinery based on key points in this embodiment is adopted, as shown in FIG. Figure 6 As shown in b, the robotic arms of the two obscured construction machines are accurately identified.
[0088] By means of the above technical solution, the embodiment of the present invention has at least the following advantages:
[0089] (1) The embodiments of the present invention adopt different recognition strategies for the entire large-scale construction machinery and the mechanical arm. For the entire construction machinery, recognition is performed by target detection, which helps to quickly and accurately locate the position of the construction machinery, thereby quickly obtaining the overall position information of the construction machinery; for the mechanical arm, recognition is performed by key point detection. This method can focus on the key parts of the mechanical arm, such as the two ends and joints of the mechanical arm, so as to effectively cope with the movement and posture changes of the mechanical arm, accurately capture the position of the mechanical arm in real time, and maintain the stability and accuracy of the detection results in a dynamic environment.
[0090] (2) By using two different identification methods for the entire large construction machine and the mechanical arm, the embodiment of the present invention significantly reduces the confusion when detecting the machine body and the mechanical arm, so that the mechanical arm can be accurately identified and located. This is particularly effective at construction sites with similar shapes and complex backgrounds, greatly reducing the risk of misidentification and providing a strong technical guarantee for the safety and efficiency of the construction site.
[0091] (3) The embodiments of the present invention achieve synchronous tracking of large-scale construction machinery and its mechanical arm by combining target detection and key point detection technology, that is, real-time tracking and understanding of the precise position and posture of the construction machinery and the mechanical arm. This innovation is of great significance for accurate distance measurement and length measurement at the construction site. By synchronously tracking the machine body and the mechanical arm, the tracking data can be used for high-precision measurement, such as determining the extended length of the mechanical arm, the distance between the machine body and the target object, etc. These data provide strong support for construction planning, quality control, safety monitoring and efficiency improvement.
[0092] (4) To address the target occlusion problem, the embodiment of the present invention uses the Soft-NMS algorithm to post-process the target bounding box, and reduces the interference caused by the target occlusion by reducing the confidence of the overlapping area, thereby further improving the accuracy of the fuselage detection.
[0093] (5) By calculating the convex hull of the key points of the robot arm to obtain the robot arm detection frame, the embodiment of the present invention can not only more accurately define the actual range of the robot arm and flexibly adapt to its posture changes, but also effectively reduce background interference and optimize the use of computing resources. At the same time, in a complex construction environment, it can also enhance the stability of robot arm recognition, and maintain relatively accurate detection even in the case of partial occlusion or light changes, thereby improving the performance and efficiency of the recognition system as a whole.
[0094] (6) The embodiments of the present invention utilize computer vision and machine learning technologies to achieve automated fuselage and robotic arm detection, thereby providing the possibility for intelligent management of construction sites, reducing reliance on manpower, and improving management efficiency.
[0095] Based on the same inventive concept, an embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned key point-based large construction machinery detection method is implemented.
[0096] Based on the same inventive concept, an embodiment of the present invention further discloses a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the above-mentioned key point-based large construction machinery detection method.
[0097] Based on the same inventive concept, an embodiment of the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned key point-based large construction machinery detection method.
[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A detection method for large construction machinery based on key points, characterized in that: The following steps are involved: S1. Acquire historical images of large-scale construction machinery, where the historical images include the body and mechanical arm of the large-scale construction machinery; S2. Annotate the historical images, including the target bounding box and the key point positions of the robotic arm, wherein the target bounding box covers the entire range of the fuselage and the robotic arm, and establish a training data set; S3, using the Yolov8-Pose model as the basic model, using the training data set to train the Yolov8-Pose model to obtain a large-scale construction machinery detection model; S4, using the trained large-scale construction machinery detection model to perform target detection on the large-scale construction machinery image to be detected, and obtain the target bounding box and the key point position of the robotic arm; S5, post-process the target bounding box detected in S4 using the Soft-NMS algorithm to obtain a target detection box; S6. Calculate the convex hull of the robot key points detected in S4 to obtain a robot detection frame.
2. The key point based large construction machinery detection method according to claim 1, characterized in that: In S2, the key points of the robot arm include a head end point, a middle point and an end point, the head end point is a point at the head end position of the robot arm, the end point is a point at the end position of the robot arm, and the middle point is located between the head end point and the end point.
3. The key point-based large construction machinery detection method according to claim 2, characterized in that: The middle point is the joint point of the robot arm.
4. The key point-based large construction machinery detection method according to claim 2, characterized in that: The number of key points of the robotic arm is three.
5. The key point based large construction machinery detection method according to claim 1, characterized in that: The specific steps of S5 include: S51, input the target bounding box list boxes and the corresponding confidence list confidence, as well as the number of bounding boxes N detected by the large construction machinery detection model in S4; S52, initialize the filtered bounding box list filtered_boxes to be empty; S53. For each bounding box i from 1 to N, perform the following steps: a. Find the target bounding box max_box with the highest confidence and its corresponding index max_index in the current remaining bounding boxes; b. Add max_box to the filtered_boxes list as the currently retained bounding box; c. Calculate the intersection-over-union ratio of max_box and all other target bounding boxes to get the intersection-over-union ratio list IoUs; d. For each target bounding box j from 1 to N, if the intersection and union ratio IoUs between the jth target bounding box and max_box is j If it is greater than the set threshold, the confidence of the jth target bounding box is updated j : Where σ represents the variance in the Gaussian function; S54, if the confidence confidence[max_index] of max_box is greater than the set confidence threshold, continue to process the remaining target bounding boxes from step S53; S55. Return the filtered_boxes list as the filtered bounding box result, that is, the target detection box.
6. The key point based large construction machinery detection method according to claim 1, characterized in that: The specific steps of S6 include: S61, calculating the minimum convex polygon formed by the key points of the robot arm; S62. Based on the minimum convex polygon, determine a robot arm detection frame, where the robot arm detection frame is a minimum area rectangle whose length or width direction is parallel or perpendicular to the boundary of the large construction machinery image to be detected and can completely contain the convex polygon.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the key point-based large construction machinery detection method according to any one of claims 1 to 5 is implemented.
8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the key point-based large construction machinery detection method according to any one of claims 1 to 5 is implemented.
9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the key point-based large construction machinery detection method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Power transmission line construction machinery hidden danger detection method based on deep learning
CN110705414A
Power transmission channel hidden danger detection method based on improved YOLOv7
CN115797357A
Mechanical monomer construction operation efficiency detection method and system based on posture recognition
CN114418310A
Mechanical arm target detecting, sensing and positioning method and device based on deep learning, storage medium and product
CN116543039A
Excavator posture detection method based on deep learning
CN117058619A
