Method and device for generating attitude of engineering mechanical equipment, terminal equipment and storage medium
By using a node prediction model to label key points of engineering machinery and calculating angle and length ratios, the accuracy of equipment posture recognition in complex environments is solved, enabling timely early warning and improved safety.
Patent Information
- Application Number
- CN202410944648.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
Existing technologies, relying on overall contour recognition methods for engineering machinery and equipment in complex terrain and environmental conditions, cannot accurately determine the actual posture of the equipment, resulting in the inability to provide timely warnings of potential dangers.
By acquiring images of engineering machinery and equipment to be identified, a node prediction model is used to label target nodes such as the center point of the vehicle body, the connection point of the robotic arm, the hinge point, and the joint point. The target angle and length ratio are calculated and compared with a preset threshold to determine the static state, working state, or abnormal posture of the equipment.
Accurately identifying equipment posture in complex environments and providing timely warnings of potential hazards improves the safety and efficiency of construction sites.
Smart Images

Figure CN118918365A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of posture generation of mechanical equipment, and in particular to a posture generation method, device, terminal equipment and storage medium for engineering mechanical equipment. Background Art
[0002] At construction sites, the movement and operation of construction machinery and equipment (such as cranes and excavators) may create a risk of collision with surrounding power transmission lines, buildings or other equipment. By identifying the posture of these equipment (such as stationary, working or abnormal) in real time, potential dangers can be warned in advance to prevent accidents. In addition, posture recognition of construction machinery and equipment such as cranes or excavators can also help delineate safe operating areas to ensure that the equipment operates within the permitted range and avoids exceeding safety limits.
[0003] When the prior art recognizes the posture of engineering machinery and equipment, it often intercepts the overall posture image of the engineering machinery and equipment in the real-time monitoring video frame, and then recognizes the overall contour of the engineering machinery and equipment through image processing technologies such as edge detection and contour extraction, and then judges the posture of the engineering machinery and equipment at the construction site based on the overall contour, such as static state, working state (such as crane lifting, excavator digging) and abnormal posture (such as excessive tilt, imbalance, etc.). However, when working under complex terrain and environmental conditions, such as muddy and rugged ground, even if the engineering machinery and equipment is in a static state, its corresponding overall contour will not be the same as the overall contour in conventional terrain. In muddy, rugged ground or other non-standard working environments, the overall contour of the engineering machinery and equipment will change significantly due to the ground conditions, and the posture of the engineering machinery and equipment and the relative position between the components will change in correlation, and these changes may far exceed the expectations in conventional terrain. In this case, the recognition method that only relies on the overall contour will have obvious limitations, resulting in the inability to accurately determine the actual posture of the engineering machinery and equipment, and thus fail to timely warn of potential dangers. Summary of the invention
[0004] The embodiments of the present invention provide a posture generation method, device, terminal device and storage medium for engineering machinery equipment, which can make the posture determination related to the relative angle and relative length ratio of the current components of the engineering machinery equipment, and can accurately determine the actual posture of the engineering machinery equipment. It can effectively solve the problem that the recognition method in the prior art that only relies on the overall contour in a non-standard working environment has obvious limitations, resulting in the inability to accurately determine the actual posture of the engineering machinery equipment.
[0005] An embodiment of the present invention provides a method for generating a posture of an engineering machinery device, comprising:
[0006] Acquire an image to be identified that includes a plurality of engineering machinery equipment;
[0007] The image to be identified is input into a preset node prediction model, so that the node prediction model marks a corresponding calibration frame for each engineering machinery equipment in the image to be identified, and after marking the target node in each calibration frame, an identified image is generated; the target node includes: the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the external tool;
[0008] For each engineering machinery equipment in the identified image, a target angle and a target length ratio between the vehicle body and the mechanical arm are generated according to the coordinate information of each corresponding target node in the calibration frame; then the target angle and the target length ratio are compared with the preset angle threshold and the length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison results; wherein the posture includes: static state, working state or abnormal posture;
[0009] The training process of the preset node prediction model includes:
[0010] The node prediction model is iteratively trained with image sample data containing several engineering machinery equipment as the input, and the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as the output of the node prediction model, until the node prediction model converges; wherein the actual annotation point is the body center annotation point of the engineering machinery equipment in the image sample data, the connection annotation point between the body and the front end of the mechanical arm, the hinge annotation point between the end of the mechanical arm and the external tool, or the joint annotation point on the mechanical arm for connecting the connecting rod.
[0011] Preferably, the engineering mechanical equipment includes: an excavator; the hinge point between the end of the mechanical arm and the external tool is: the hinge point between the end of the mechanical arm and the bucket; the bottom point of the external tool is: the bottom point of the bottom of the bucket;
[0012] The method of generating a target angle and a target length ratio between the vehicle body and the robotic arm according to the coordinate information of each corresponding target node in the calibration frame includes:
[0013] Obtain the coordinate information of the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the bucket, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the bottom of the bucket in the calibration frame;
[0014] Generate a target angle between the vehicle body and the mechanical arm according to the coordinate information of the connection point between the vehicle body and the head end of the mechanical arm, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the bottom point of the bottom of the bucket;
[0015] Generate a first distance between the center point of the vehicle body and the joint point and a second distance between the center point of the vehicle body and the hinge point according to the coordinate information of the center point of the vehicle body of the engineering machinery equipment, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the hinge point between the end of the mechanical arm and the bucket;
[0016] According to the ratio between the first distance and the second distance, a target length ratio between the vehicle body and the robotic arm is determined.
[0017] Preferably, comparing the target angle and the target length ratio with a preset angle threshold and a length ratio threshold respectively, and determining the posture of the engineering machinery equipment according to the comparison results, comprises:
[0018] Generate a third distance between the end of the robotic arm and the center point of the vehicle body according to the coordinate information of the bottom point of the bucket bottom and the coordinate information of the center point of the vehicle body;
[0019] The target angle is compared with a preset angle threshold, and when it is determined that the target angle is less than the preset angle threshold, the third distance is compared with the preset distance threshold, and when it is determined that the third distance is less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a stationary state and the mechanical arm is aligned with the vehicle body; when it is determined that the third distance is not less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a working state;
[0020] When it is determined that the target angle is not less than the preset angle threshold, the target length ratio is compared with the preset length ratio threshold. When it is determined that the target length ratio is less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be an abnormal state; when it is determined that the target length ratio is not less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be a working state.
[0021] Preferably, the engineering mechanical equipment comprises: a crane; the hinge point between the end of the mechanical arm and the external tool is: the hinge point between the end of the mechanical arm and the hook, and the bottom point of the external tool is: the bottom point of the bottom of the hook;
[0022] The method of generating a target angle and a target length ratio between the vehicle body and the robotic arm according to the coordinate information of each corresponding target node in the calibration frame includes:
[0023] Obtain the coordinate information of the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the hook, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the bottom of the hook in the calibration frame;
[0024] Generate a target angle between the vehicle body and the robotic arm based on the coordinate information of the center point of the vehicle body, the coordinate information of the connection point between the vehicle body and the head end of the robotic arm, and the coordinate information of the bottom point of the bottom of the hook;
[0025] Generate a fourth distance between the center point of the vehicle body and the joint point and a fifth distance between the center point of the vehicle body and the hinge point according to the coordinate information of the center point of the vehicle body of the engineering machinery equipment, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the hinge point between the end of the mechanical arm and the hook;
[0026] According to the ratio between the fourth distance and the fifth distance, a target length ratio between the vehicle body and the robotic arm is determined.
[0027] Preferably, comparing the target angle and the target length ratio with a preset angle threshold and a length ratio threshold respectively, and determining the posture of the engineering machinery equipment according to the comparison results, comprises:
[0028] Generate a sixth distance between the end of the robotic arm and the center point of the vehicle body according to the coordinate information of the bottom point of the hook bottom and the coordinate information of the center point of the vehicle body;
[0029] The target angle is compared with a preset angle threshold, and when it is determined that the target angle is less than the preset angle threshold, the sixth distance is compared with the preset distance threshold, and when it is determined that the sixth distance is less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a stationary state and the mechanical arm is aligned with the vehicle body; when it is determined that the sixth distance is not less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a working state;
[0030] When it is determined that the target angle is not less than the preset angle threshold, the target length ratio is compared with the preset length ratio threshold. When it is determined that the target length ratio is less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be an abnormal state; when it is determined that the target length ratio is not less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be a working state.
[0031] Preferably, the node prediction model comprises a deformable convolution layer and an output layer;
[0032] The step of marking the target node in each calibration frame includes:
[0033] For each calibration frame, the deformable convolution layer in the node prediction model predicts the offset of each sampling point on the engineering machinery equipment in the calibration frame through a lightweight convolution layer;
[0034] The deformable convolution layer performs non-uniform sampling on the points on the engineering machinery equipment in the calibration frame based on the offset of each sampling point, and generates feature information of points at different parts of the engineering machinery equipment;
[0035] The output layer marks the target nodes on the engineering machinery equipment according to the feature information of different parts.
[0036] Preferably, each time the node prediction model is trained, image sample data containing a plurality of engineering machinery equipment is input into the node prediction model, so that the node prediction model outputs a prediction result of a calibration frame corresponding to the image sample data and a prediction labeling result of each node in the calibration frame;
[0037] The prediction results of the calibration box corresponding to the image sample data and the predicted annotation results of each node in the calibration box are compared with the actual calibration box corresponding to the image sample data and each actual annotation point, and the network parameters of the node prediction model are adjusted according to the comparison results.
[0038] Based on the above method embodiments, the present invention provides corresponding device embodiments.
[0039] An embodiment of the present invention provides a posture generation device for engineering machinery equipment, comprising: an image acquisition module, a node recognition module and a posture determination module;
[0040] The image acquisition module is used to acquire an image to be identified that contains a number of engineering machinery equipment;
[0041] The node recognition module is used to input the image to be recognized into a preset node prediction model, so that the node prediction model marks a corresponding calibration frame for each engineering machinery equipment in the image to be recognized, and after marking the target node in each calibration frame, a recognized image is generated; wherein the target node includes: the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the external tool;
[0042] The posture determination module is used to generate a target angle and a target length ratio between the vehicle body and the mechanical arm for each engineering machinery equipment in the identified image according to the coordinate information of each corresponding target node in the calibration frame; then the target angle and the target length ratio are compared with the preset angle threshold and the length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison result;
[0043] The training process of the preset node prediction model includes:
[0044] The node prediction model is iteratively trained with image sample data containing several engineering machinery equipment as the input, and the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as the output of the node prediction model, until the node prediction model converges; wherein the actual annotation point is the body center annotation point of the engineering machinery equipment in the image sample data, the connection annotation point between the body and the front end of the mechanical arm, the hinge annotation point between the end of the mechanical arm and the external tool, or the joint annotation point on the mechanical arm for connecting the connecting rod.
[0045] Based on the above method embodiments, the present invention provides corresponding terminal device embodiments.
[0046] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the posture generation method of an engineering machinery equipment described in the above-mentioned embodiment of the invention.
[0047] Based on the above method embodiments, the present invention provides corresponding storage medium item embodiments.
[0048] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the posture generation method of the engineering machinery equipment described in the above-mentioned embodiment of the invention.
[0049] The following beneficial effects are achieved by implementing the present invention:
[0050] The embodiment of the present invention provides a posture generation method, device, terminal device and storage medium for engineering machinery equipment. The present invention inputs an image to be identified containing several engineering machinery equipment into a preset node prediction model, so that the node prediction model can mark the engineering machinery equipment and mark the target nodes on the engineering machinery equipment. Each target node can be the center point of the body of the engineering machinery equipment, the connection point between the body and the front end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod and the bottom point of the external tool, thereby realizing the marking of the key points of each component on the engineering machinery equipment. Finally, the angle and length ratio between the body and the mechanical arm are calculated based on the position coordinates of these points, and then compared with the preset threshold value, so as to more accurately identify the actual posture of the engineering machinery equipment. Compared with the prior art, the present invention accurately marks the key points of each component of the engineering machinery and equipment, and judges the posture based on the position information of each node, so that the posture determination is related to the relative angles and relative length ratios of the current components of the engineering machinery and equipment, and when the engineering machinery and equipment are in different postures, the relative angles and relative length ratios of the various components inside it will not be different due to changes in the terrain. Therefore, even if the engineering machinery and equipment is currently in a muddy, rugged ground or other non-standard working environment, based on the component point marking and angle, and the comparison of the angle and length ratio of the present invention, the actual posture of the engineering machinery and equipment can be accurately determined. The present invention can cope with various complex and changeable working environments, and based on the accurately identified posture information, it can promptly detect possible abnormal states of the engineering machinery and equipment (such as excessive tilt, imbalance, etc.), thereby issuing early warnings in time to avoid accidents and improve the safety and efficiency of the construction site. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 It is a flow chart of a method for generating a posture of an engineering machinery device provided in one embodiment of the present invention.
[0052] Figure 2 It is a schematic diagram of the annotation of various nodes in the engineering machinery equipment provided by an embodiment of the present invention.
[0053] Figure 3 It is a structural schematic diagram of a posture generating device for engineering machinery equipment provided by one embodiment of the present invention. DETAILED DESCRIPTION
[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] like Figure 1 FIG. 1 is a flow chart of a method for generating a posture of an engineering machinery device provided by an embodiment of the present invention. The method for generating a posture of an engineering machinery device comprises:
[0056] Step S1: obtaining an image to be identified containing a plurality of engineering machinery equipment;
[0057] Step S2: inputting the image to be identified into a preset node prediction model, so that the node prediction model marks a corresponding calibration frame for each engineering machinery equipment in the image to be identified, and after marking the target node in each calibration frame, generating an identified image; wherein the target node includes: the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the external tool;
[0058] The training process of the preset node prediction model includes:
[0059] The node prediction model is iteratively trained by using image sample data containing several engineering machinery equipment as input, and the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as output of the node prediction model, until the node prediction model converges; wherein the actual annotation point is the body center annotation point of the engineering machinery equipment in the image sample data, the connection annotation point between the body and the head end of the mechanical arm, the hinge annotation point between the end of the mechanical arm and the external tool, or the joint annotation point on the mechanical arm for connecting the connecting rod;
[0060] Step S3: for each engineering machinery equipment in the identified image, a target angle and a target length ratio between the vehicle body and the mechanical arm are generated according to the coordinate information of each corresponding target node in the calibration frame; then the target angle and the target length ratio are compared with the preset angle threshold and the length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison results;
[0061] The posture includes: static state, working state or abnormal posture.
[0062] For step S1, in a preferred embodiment, the images acquired by the present invention include images of several engineering machinery equipment to be identified, which may come from various scenes, such as construction sites, mines, ports, etc., in which the engineering machinery equipment is performing various operations or is in different states. Specifically, the images to be identified may include various types of engineering machinery equipment, such as excavators, loaders, cranes, road rollers, and cranes.
[0063] Schematically, the construction machinery equipment in the image may be in different postures, including a stationary state (such as standby or parked), a working state (such as digging, loading or lifting), and possible abnormal postures (such as a twisted or broken mechanical arm or equipment tipping over, etc.).
[0064] Illustratively, the image background may include other objects, people, buildings, etc., forming a complex environment. In a preferred embodiment, the node prediction model of the present invention can accurately distinguish between engineering machinery and the background to avoid misidentification.
[0065] For step S2, in a preferred embodiment, by collecting images or video frame data of excavators and cranes, ensuring that various postures and angles are included, and formulating key point annotation standards to annotate the collected images, such as the joints of the robotic arm, the center of the vehicle body, etc., the Yolov8-Pose model can be used as the basic model of the preset node prediction model for training, thereby realizing the training of key point annotated images of excavators and cranes.
[0066] Therefore, when the trained Yolov8-Pose model is applied to the image to be detected, the trained Yolov8-Pose model can be used as a preset node prediction model. The key points and coordinates of the key points of various types of engineering machinery and equipment in the image to be identified can be detected through the preset node prediction model, so that the position, angle, direction and other posture information between the components of the engineering machinery and equipment can be calculated through geometric transformation according to the coordinates of the key points, thereby determining the current actual posture of each engineering machinery and equipment in the image to be identified.
[0067] Specifically, in the data preparation stage, we first collect a large amount of excavator and crane image or video data, covering various working scenes and postures. Next, we annotate each mechanical equipment image with key points, manually or automatically annotate multiple key points on each image, such as annotating the joints of the excavator and crane arm and the center of the vehicle body. Figure 2As shown in the figure, the center point of the body of the engineering machinery equipment, the connection point between the body and the head of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, and the joint point on the mechanical arm for connecting the connecting rod can be annotated, and these nodes can fully describe the key parts and posture information of the equipment. Through the above data preparation, a data set containing multiple engineering machinery equipment images and their key point annotations can be obtained, providing necessary input for subsequent model training and posture estimation.
[0068] In a preferred embodiment, the present invention can use image sample data containing a number of engineering machinery equipment as the input of the node prediction model, use the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as the output of the node prediction model, and iteratively train the node prediction model until the node prediction model converges;
[0069] Each time the node prediction model is trained, image sample data containing a number of engineering machinery equipment is input into the node prediction model, so that the node prediction model outputs the prediction results of the calibration frame corresponding to the image sample data and the prediction and annotation results of each node in the calibration frame;
[0070] The prediction results of the calibration box corresponding to the image sample data and the predicted annotation results of each node in the calibration box are compared with the actual calibration box corresponding to the image sample data and each actual annotation point, and the network parameters of the node prediction model are adjusted according to the comparison results.
[0071] In a preferred embodiment, the node prediction model of the present invention is a Yolov8-Pose model, and DCNv3 (Deformable Convolutional Networks version 3) is introduced into the YOLOv8-Pose model to optimize DyHead (Dynamic Head, the part used for multi-scale prediction and feature fusion in YOLOv8), that is, DyHead is a dynamic head structure used for multi-scale feature fusion and prediction in YOLOv8.
[0072] Schematically, DCNv3 is an improved convolution method that enhances the modeling ability of the model for geometric transformation by increasing the offset of each sampling point, and can make outstanding contributions to tasks that require precise spatial positioning, such as posture estimation. That is, the DCNv3 of the present invention adjusts the sampling position of the convolution kernel by learning the offset, so as to better capture the deformation and displacement of the object. Compared with ordinary convolution, DCNv3 can better adapt to irregularly shaped objects and local deformations. When using DCNv3 to optimize the YOLOv8-Pose model after DyHead, the detection and positioning capabilities of deformed objects and key points will be improved compared to the original model, especially in the subsequent processing of posture recognition tasks of mechanical equipment such as excavators and cranes, better performance can be obtained.
[0073] In schematic form, by marking the calibration frame, the excavator can be accurately separated from the complex background, reducing the possibility of misidentification or missed identification, and the coordinate information of key points can be obtained more accurately.
[0074] In a preferred embodiment, the node prediction model includes a deformable convolutional layer and an output layer;
[0075] The node prediction model specifically includes:
[0076] For each calibration frame, the deformable convolution layer in the node prediction model predicts the offset of each sampling point on the engineering machinery equipment in the calibration frame through a lightweight convolution layer;
[0077] The deformable convolution layer performs non-uniform sampling on the points on the engineering machinery equipment in the calibration frame based on the offset of each sampling point, and generates feature information of points at different parts of the engineering machinery equipment;
[0078] The output layer marks the target nodes on the engineering machinery equipment according to the feature information of different parts.
[0079] The deformable convolution layer of the present invention predicts the offset of each sampling point, allowing the network to perform non-uniform sampling during the sampling process, thereby being able to more flexibly capture the geometric shape and deformation of the target (i.e., engineering machinery and equipment). When sampling and identifying engineering machinery and equipment in complex scenes, since they may change significantly due to differences in viewing angle, posture or ground conditions, the feature information generated by the deformable convolution layer of the present invention can better reflect the structure and details of the engineering machinery and equipment in complex scenes, so that the output layer can more accurately locate the various target nodes on the engineering machinery and equipment, such as the center point of the vehicle body, connection points, hinge points, etc., thereby improving the accuracy of subsequent posture discrimination.
[0080] Therefore, when a node prediction model including a deformable convolutional layer and an output layer is used to process images of construction machinery and equipment, target nodes at different positions can be accurately marked in each calibration frame to generate an identified image. The node prediction model of the present invention not only improves the accuracy of target node positioning, but also has good generalization ability and computational efficiency.
[0081] For step S3, in a preferred embodiment, the engineering machinery equipment may be an excavator or a crane. Regardless of whether it is an excavator or a crane, the embodiment of the present invention may calculate the target angle between the vehicle body and the mechanical arm and the target length ratio between the vehicle body and the mechanical arm based on the marked positions of each target node of the engineering machinery equipment, thereby judging the real-time posture of the engineering machinery equipment, i.e., the excavator or the crane.
[0082] When the engineering machinery equipment is an excavator, the hinge point between the end of the mechanical arm and the external tool is: the hinge point between the end of the mechanical arm and the bucket; the bottom point of the external tool is: the bottom point of the bottom of the bucket; that is, the present invention can mark the bottom point of the bucket and obtain its coordinate information.
[0083] Then, when the target angle and target length ratio between the vehicle body and the robotic arm are generated according to the coordinate information of each corresponding target node in the calibration frame, it is specifically as follows:
[0084] Obtain the coordinate information of the center point of the vehicle body of the engineering machinery equipment (i.e., excavator), the connection point between the vehicle body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the bucket, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the bottom of the bucket in the calibration frame;
[0085] Generate a target angle between the vehicle body and the mechanical arm according to the coordinate information of the connection point between the vehicle body and the head end of the mechanical arm, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the bottom point of the bottom of the bucket;
[0086] Generate a first distance between the center point of the vehicle body and the joint point and a second distance between the center point of the vehicle body and the hinge point according to the coordinate information of the center point of the vehicle body of the engineering machinery equipment, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the hinge point between the end of the mechanical arm and the bucket;
[0087] According to the ratio between the first distance and the second distance, a target length ratio between the vehicle body and the robotic arm is determined.
[0088] Furthermore, when the engineering machinery equipment is an excavator, the target angle and the target length ratio are compared with a preset angle threshold and a length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison results, specifically including:
[0089] Generate a third distance between the end of the robotic arm and the center point of the vehicle body according to the coordinate information of the bottom point of the bucket bottom and the coordinate information of the center point of the vehicle body;
[0090] The target angle is compared with a preset angle threshold, and when it is determined that the target angle is less than the preset angle threshold, the third distance is compared with the preset distance threshold, and when it is determined that the third distance is less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment (i.e., the excavator) is in a stationary state and the mechanical arm is aligned with the vehicle body; when it is determined that the third distance is not less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a working state;
[0091] When it is determined that the target angle is not less than the preset angle threshold, the target length ratio is compared with the preset length ratio threshold. When it is determined that the target length ratio is less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be an abnormal state; when it is determined that the target length ratio is not less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be a working state.
[0092] In order to explain the above posture determination process more clearly, the coordinate information of the center point (C) of the vehicle body is: (x_c, y_c), the coordinate information of the connection point (A) between the vehicle body and the head end of the robotic arm is: (x_a, y_a), the coordinate information of the hinge point (B) between the end of the robotic arm and the bucket is: (x_b, y_b), the coordinate information of the joint point (J) on the robotic arm for connecting the connecting rod is: (x_j, y_j), and the coordinate information of the bottom point (D) of the bottom of the bucket is: (x_d, y_d);
[0093] When generating the target angle between the vehicle body and the robotic arm, the target angle between the vehicle body and the robotic arm can be generated according to the coordinate information of the connection point, the joint point and the bottom point, that is, the angle formed by point A, point J and point D can be used as an approximate value of the target angle.
[0094] The embodiment of the present invention can use vectors to calculate the angle and distance, and can accurately obtain the angle and relative position relationship between the robot arm and the vehicle body. Compared with the traditional method based on sensors or empirical values, the present invention can be more accurate and reliable, and then:
[0095] Vector AJ is: (x_j-x_a, y_j-y_a);
[0096] Vector AD is: (x_d-x_a, y_d-y_a);
[0097] Then the angle θ = arccos((AJ·AD) / (|AJ|*|AD|)), where · represents the vector dot product and || represents the vector modulus.
[0098] Further, the first distance d1 between the center point C of the vehicle body and the joint point J can be calculated based on the coordinate information of the center point C of the vehicle body and the coordinate information of the joint point J on the mechanical arm for connecting the connecting rod; the second distance d2 between the center point C of the vehicle body and the hinge point B can be calculated based on the coordinate information of the center point of the vehicle body and the coordinate information of the hinge point between the end of the mechanical arm and the bucket. Based on the coordinate information of the bottom point D of the bottom of the bucket and the center point C of the vehicle body, the third distance d3 between the end of the mechanical arm and the center point of the vehicle body is generated;
[0099] Indicative, first distance
[0100] Second distance
[0101] The third distance
[0102] Then the target length ratio between the vehicle body and the robotic arm is ratio = d2 / d1;
[0103] Assume that the preset angle threshold is θ thresh , the length ratio threshold is ratio thresh , the distance threshold is dist thresh , then the target angle θ is compared with the preset angle threshold θ thresh Compare, if θ<θ thresh (This means that the current angle between the vehicle body and the robotic arm is very small and close to horizontal), and d3 <dist thresh (It means that the bottom point D of the bucket is very close to the center point C of the vehicle body, so it can be inferred that the construction machinery equipment is in a stationary state, and the bucket part is in a very close alignment with the vehicle body). Then it can be determined that the posture of the construction machinery equipment is in a stationary state and the mechanical arm is aligned with the vehicle body.
[0104] If d3≥dist thresh (The description shows that there is a certain distance between the bottom point D of the bucket and the center point C of the vehicle body, which means that the mechanical arm is performing work, causing the bucket to be away from the center point of the vehicle body), then the posture of the engineering machinery equipment is determined to be in a working state.
[0105] Furthermore, if θ ≥ θ thresh (indicating that the current angle between the vehicle body and the robotic arm is large), the target length ratio is compared with the preset length ratio threshold. <ratio thresh (Indicating that the extension length of the robot arm is relatively small), because the target angle θ is not less than the preset angle threshold θ threshWhen the target length ratio is less than the preset length ratio threshold ratio, the angle between the robot arm and the vehicle body is relatively large, indicating that the robot arm should be performing some non-horizontal or non-static operation. thresh , indicating that although the angle between the robot arm and the vehicle body is large, the extended length of the robot arm is relatively small. This situation does not conform to the normal operating logic or the design expectations of the robot arm, because at a large angle, the robot arm is usually expected to be able to extend accordingly to complete a specific task. When the target length ratio is less than the preset length ratio threshold, it can be determined that the posture of the construction machinery equipment is in an abnormal state.
[0106] When θ ≥ θ thresh , and ratio>ratio thresh (Indicating that the extension length of the robot arm is relatively large), and θ ≥ θ thresh It indicates that the angle between the robot arm and the vehicle body is relatively large, and it is not in a near-horizontal or static state, which usually means that the robot arm is performing some dynamic operation, such as digging, loading, moving, etc. thresh This indicates that the relative ratio between the extended length of the robotic arm and the vehicle body exceeds the preset threshold, which further illustrates that not only is the angle between the robotic arm and the vehicle body large, but its extended length is also significantly increased to adapt to the current work task, so that it can be determined that the posture of the construction machinery equipment is in a working state.
[0107] The embodiment of the present invention comprehensively considers the angle (θ) between the mechanical arm and the vehicle body, the extension length of the mechanical arm (reflected by d1, d2 and ratio), and the distance between the bottom of the bucket and the center point of the vehicle body (d3), and the above parameters together provide a full description of the posture of the mechanical arm, so that the final posture determination result is more accurate and comprehensive. Since all parameters can be calculated through the coordinate information of the real-time annotation points, the present invention can also reflect the posture changes of the engineering machinery equipment in real time.
[0108] When the engineering machinery equipment is a crane, the hinge point between the end of the mechanical arm and the external tool is: the hinge point between the end of the mechanical arm and the hook, and the bottom point of the external tool is: the bottom point of the bottom of the hook, that is, the bottom point of the bottom of the hook can be marked and its coordinate information can be obtained.
[0109] Then, when the target angle and target length ratio between the vehicle body and the robotic arm are generated according to the coordinate information of each corresponding target node in the calibration frame, it is specifically as follows:
[0110] Obtain the coordinate information of the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the hook, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the bottom of the hook in the calibration frame;
[0111] Generate the target angle between the vehicle body and the robotic arm according to the coordinate information of the center point of the vehicle body, the coordinate information of the connection point between the vehicle body and the head end of the robotic arm, and the coordinate information of the bottom point between the end of the robotic arm and the bottom of the hook;
[0112] Generate a fourth distance between the center point of the vehicle body and the joint point and a fifth distance between the center point of the vehicle body and the hinge point according to the coordinate information of the center point of the vehicle body of the engineering machinery equipment, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the hinge point between the end of the mechanical arm and the hook;
[0113] According to the ratio between the fourth distance and the fifth distance, a target length ratio between the vehicle body and the robotic arm is determined.
[0114] Furthermore, comparing the target angle and the target length ratio with a preset angle threshold and a length ratio threshold respectively, and determining the posture of the engineering machinery equipment according to the comparison results, includes:
[0115] Generate a sixth distance between the end of the robotic arm and the center point of the vehicle body according to the coordinate information of the bottom point of the hook bottom and the coordinate information of the center point of the vehicle body;
[0116] The target angle is compared with a preset angle threshold, and when it is determined that the target angle is less than the preset angle threshold, the sixth distance is compared with the preset distance threshold, and when it is determined that the sixth distance is less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a stationary state and the mechanical arm is aligned with the vehicle body; when it is determined that the sixth distance is not less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a working state;
[0117] When it is determined that the target angle is not less than the preset angle threshold, the target length ratio is compared with the preset length ratio threshold. When it is determined that the target length ratio is less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be an abnormal state; when it is determined that the target length ratio is not less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be a working state.
[0118] The crane posture determination process of the embodiment of the present invention is similar to the calculation process and comparison principle of the excavator posture determination process, which will not be described in detail here. Schematically, the embodiment of the present invention can accurately determine the current state of the crane by accurately calculating the key point coordinates, distances and angles of each component, thereby reducing the risk of erroneous operation or accidents caused by misjudgment.
[0119] like Figure 3 As shown, based on the above-mentioned embodiments of the posture generation method of various engineering machinery equipment, the present invention provides a corresponding device embodiment;
[0120] An embodiment of the present invention provides a posture generation device for engineering machinery equipment, comprising: an image acquisition module, a node recognition module and a posture determination module;
[0121] The image acquisition module is used to acquire an image to be identified that contains a number of engineering machinery equipment;
[0122] The node recognition module is used to input the image to be recognized into a preset node prediction model, so that the node prediction model marks a corresponding calibration frame for each engineering machinery equipment in the image to be recognized, and after marking the target node in each calibration frame, a recognized image is generated; wherein the target node includes: the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the external tool;
[0123] The posture determination module is used to generate a target angle and a target length ratio between the vehicle body and the mechanical arm for each engineering machinery equipment in the identified image according to the coordinate information of each corresponding target node in the calibration frame; then the target angle and the target length ratio are compared with the preset angle threshold and the length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison result;
[0124] The training process of the preset node prediction model includes:
[0125] The node prediction model is iteratively trained with image sample data containing several engineering machinery equipment as the input, and the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as the output of the node prediction model, until the node prediction model converges; wherein the actual annotation point is the body center annotation point of the engineering machinery equipment in the image sample data, the connection annotation point between the body and the front end of the mechanical arm, the hinge annotation point between the end of the mechanical arm and the external tool, or the joint annotation point on the mechanical arm for connecting the connecting rod.
[0126] It should be noted that the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without paying creative labor.
[0127] Those skilled in the art can clearly understand that, for the sake of convenience and simplicity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0128] Based on the above-mentioned embodiments of the posture generation method of various engineering machinery equipment, the present invention provides corresponding embodiments of terminal equipment items.
[0129] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a posture generation method for engineering machinery equipment described in any method embodiment of the present invention.
[0130] The terminal device may be a computing terminal device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.
[0131] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.
[0132] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.
[0133] Based on the above-mentioned embodiments of the posture generation method of various engineering machinery equipment, the present invention provides corresponding storage medium item embodiments.
[0134] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a posture generation method for engineering machinery equipment described in any method embodiment of the present invention.
[0135] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment described above can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0136] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for generating a posture of an engineering machinery device, characterized in that: include: Acquire an image to be identified that includes a plurality of engineering machinery equipment; The image to be identified is input into a preset node prediction model, so that the node prediction model marks a corresponding calibration frame for each engineering machinery equipment in the image to be identified, and after marking the target node in each calibration frame, an identified image is generated; wherein the target node includes: the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the external tool; For each engineering machinery equipment in the identified image, a target angle and a target length ratio between the vehicle body and the mechanical arm are generated according to the coordinate information of each corresponding target node in the calibration frame; then the target angle and the target length ratio are compared with the preset angle threshold and the length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison results; wherein the posture includes: static state, working state or abnormal posture; The training process of the preset node prediction model includes: The node prediction model is iteratively trained with image sample data containing several engineering machinery equipment as the input, and the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as the output of the node prediction model, until the node prediction model converges; wherein the actual annotation point is the body center annotation point of the engineering machinery equipment in the image sample data, the connection annotation point between the body and the front end of the mechanical arm, the hinge annotation point between the end of the mechanical arm and the external tool, or the joint annotation point on the mechanical arm for connecting the connecting rod.
2. A method for generating a posture of an engineering machinery device as claimed in claim 1, characterized in that: The engineering mechanical equipment includes: an excavator; the hinge point between the end of the mechanical arm and the external tool is: the hinge point between the end of the mechanical arm and the bucket; the bottom point of the external tool is: the bottom point of the bottom of the bucket; The method of generating a target angle and a target length ratio between the vehicle body and the robotic arm according to the coordinate information of each corresponding target node in the calibration frame includes: Obtain the coordinate information of the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the bucket, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the bottom of the bucket in the calibration frame; Generate a target angle between the vehicle body and the mechanical arm according to the coordinate information of the connection point between the vehicle body and the head end of the mechanical arm, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the bottom point of the bottom of the bucket; Generate a first distance between the center point of the vehicle body and the joint point and a second distance between the center point of the vehicle body and the hinge point according to the coordinate information of the center point of the vehicle body of the engineering machinery equipment, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the hinge point between the end of the mechanical arm and the bucket; According to the ratio between the first distance and the second distance, a target length ratio between the vehicle body and the robotic arm is determined.
3. A method for generating a posture of an engineering machinery device as claimed in claim 2, characterized in that: The target angle and the target length ratio are compared with a preset angle threshold and a length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison results, including: Generate a third distance between the end of the robotic arm and the center point of the vehicle body according to the coordinate information of the bottom point of the bucket bottom and the coordinate information of the center point of the vehicle body; The target angle is compared with a preset angle threshold, and when it is determined that the target angle is less than the preset angle threshold, the third distance is compared with the preset distance threshold, and when it is determined that the third distance is less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a stationary state and the mechanical arm is aligned with the vehicle body; when it is determined that the third distance is not less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a working state; When it is determined that the target angle is not less than the preset angle threshold, the target length ratio is compared with the preset length ratio threshold. When it is determined that the target length ratio is less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be an abnormal state; when it is determined that the target length ratio is not less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be a working state.
4. The method for generating a posture of an engineering machinery device according to claim 1, characterized in that: The engineering mechanical equipment comprises: a crane; the hinge point between the end of the mechanical arm and the external tool is: the hinge point between the end of the mechanical arm and the hook, and the bottom point of the external tool is: the bottom point of the bottom of the hook; The method of generating a target angle and a target length ratio between the vehicle body and the robotic arm according to the coordinate information of each corresponding target node in the calibration frame includes: Obtain the coordinate information of the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the hook, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the bottom of the hook in the calibration frame; Generate the target angle between the vehicle body and the robotic arm according to the coordinate information of the center point of the vehicle body, the coordinate information of the connection point between the vehicle body and the head end of the robotic arm, and the coordinate information of the bottom point between the end of the robotic arm and the bottom of the hook; Generate a fourth distance between the center point of the vehicle body and the joint point and a fifth distance between the center point of the vehicle body and the hinge point according to the coordinate information of the center point of the vehicle body of the engineering machinery equipment, the coordinate information of the joint point on the mechanical arm for connecting the connecting rod, and the coordinate information of the hinge point between the end of the mechanical arm and the hook; According to the ratio between the fourth distance and the fifth distance, a target length ratio between the vehicle body and the robotic arm is determined.
5. A method for generating a posture of an engineering machinery device as claimed in claim 4, characterized in that: The target angle and the target length ratio are compared with a preset angle threshold and a length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison results, including: Generate a sixth distance between the end of the robotic arm and the center point of the vehicle body according to the coordinate information of the bottom point of the hook bottom and the coordinate information of the center point of the vehicle body; The target angle is compared with a preset angle threshold, and when it is determined that the target angle is less than the preset angle threshold, the sixth distance is compared with the preset distance threshold, and when it is determined that the sixth distance is less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a stationary state and the mechanical arm is aligned with the vehicle body; when it is determined that the sixth distance is not less than the preset distance threshold, it is determined that the posture of the engineering machinery equipment is in a working state; When it is determined that the target angle is not less than the preset angle threshold, the target length ratio is compared with the preset length ratio threshold. When it is determined that the target length ratio is less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be an abnormal state; when it is determined that the target length ratio is not less than the preset length ratio threshold, the posture of the engineering machinery equipment is determined to be a working state.
6. The method for generating a posture of an engineering machinery device according to claim 1, characterized in that: The node prediction model includes a deformable convolution layer and an output layer; The step of marking the target node in each calibration frame includes: For each calibration frame, the deformable convolution layer in the node prediction model predicts the offset of each sampling point on the engineering machinery equipment in the calibration frame through a lightweight convolution layer; The deformable convolution layer performs non-uniform sampling on the points on the engineering machinery equipment in the calibration frame based on the offset of each sampling point, and generates feature information of points at different parts of the engineering machinery equipment; The output layer marks the target nodes on the engineering machinery equipment according to the feature information of different parts.
7. The method for generating a posture of an engineering machinery device according to claim 1, characterized in that: Each time the node prediction model is trained, image sample data containing a number of engineering machinery equipment is input into the node prediction model, so that the node prediction model outputs the prediction result of the calibration frame corresponding to the image sample data and the prediction and annotation result of each node in the calibration frame; The prediction results of the calibration box corresponding to the image sample data and the predicted annotation results of each node in the calibration box are compared with the actual calibration box corresponding to the image sample data and each actual annotation point, and the network parameters of the node prediction model are adjusted according to the comparison results.
8. A posture generating device for engineering machinery equipment, characterized in that: include: Image acquisition module, node recognition module and posture determination module; The image acquisition module is used to acquire an image to be identified that contains a number of engineering machinery equipment; The node recognition module is used to input the image to be recognized into a preset node prediction model, so that the node prediction model marks a corresponding calibration frame for each engineering machinery equipment in the image to be recognized, and after marking the target node in each calibration frame, a recognized image is generated; wherein the target node includes: the center point of the body of the engineering machinery equipment, the connection point between the body and the head end of the mechanical arm, the hinge point between the end of the mechanical arm and the external tool, the joint point on the mechanical arm for connecting the connecting rod, and the bottom point of the external tool; The posture determination module is used to generate a target angle and a target length ratio between the vehicle body and the mechanical arm for each engineering machinery equipment in the identified image according to the coordinate information of each corresponding target node in the calibration frame; then the target angle and the target length ratio are compared with the preset angle threshold and the length ratio threshold respectively, and the posture of the engineering machinery equipment is determined according to the comparison result; The training process of the preset node prediction model includes: The node prediction model is iteratively trained with image sample data containing several engineering machinery equipment as the input, and the actual calibration frame of each engineering machinery equipment in the image sample data and each actual annotation point in each engineering machinery equipment as the output of the node prediction model, until the node prediction model converges; wherein the actual annotation point is the body center annotation point of the engineering machinery equipment in the image sample data, the connection annotation point between the body and the front end of the mechanical arm, the hinge annotation point between the end of the mechanical arm and the external tool, or the joint annotation point on the mechanical arm for connecting the connecting rod.
9. A terminal device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a posture generation method for engineering machinery equipment as described in any one of claims 1 to 7 is implemented.
10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the posture generation method of engineering machinery equipment as described in any one of claims 1 to 7.
Citation Information
Cited By
External damage object posture recognition method and device based on key point model, computer equipment and readable storage medium
CN118887201A
Railway wagon coupler operation safety monitoring method and system based on HRNet
CN120932213A
No-prior hinge type object state generation and joint estimation method
CN122176689A
