External damage object posture recognition method and device based on key point model, computer equipment and readable storage medium
Through the method based on the key point model, the pre-trained key point detection model is used to extract key position points in the power monitoring image, and identify the posture state of the external broken object, which solves the problem of low accuracy in the existing technology of foreign broken object recognition. Accurate identification and security in complex environments.
Patent Information
- Application Number
- CN202411256919.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art Chinese and foreign stance recognition has low accuracy in complex backgrounds, and it is difficult to accurately identify the posture of construction machinery in a distributed transmission line environment in the field, resulting in difficulty in discovering safety hazards.
Using a key point model-based method, the power monitoring image is extracted through the pre-trained key point detection model, the coordinates of the key position point of the external broken object are obtained, and the relative position and size proportional relationship between key components is identified based on these coordinates. , and then identify the posture state of the external objects.
It improves the accuracy of external object-breaking posture recognition and the recognition performance in complex environments, enhances the accurate recognition ability of external object-breaking postures, and improves the reliability of identification technology.
Smart Images

Figure CN118887201A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for identifying postures of external objects based on a key point model. Background Art
[0002] In recent years, with the continuous expansion of my country's power grid, the coverage of transmission lines has gradually become wider and wider; on the other hand, many regions have vigorously developed construction, but the use of construction machinery in transmission channels will greatly threaten the safety of transmission lines.
[0003] Since most power transmission lines are located outdoors and are extremely scattered, when there are external objects such as excavators, cranes, tower cranes, pile drivers, and bulldozers on the transmission lines, in order to ensure timely, effective, and accurate detection of potential safety hazards, it is necessary to perform posture recognition on the external objects. However, in the related art, the posture recognition method for external objects in complex backgrounds is easily affected by environmental interference, resulting in reduced recognition accuracy.
[0004] Therefore, there is a problem in the related art that the accuracy of posture recognition of external broken objects is low. Summary of the invention
[0005] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for external object posture recognition based on a key point model, which can improve the accuracy of external object posture recognition.
[0006] In a first aspect, the present application provides a method for identifying postures of external objects based on a key point model, comprising:
[0007] Acquire a power monitoring image; the power monitoring image is an image containing external broken objects in the power operation site; the external broken objects include construction machinery and equipment;
[0008] Input the power monitoring image into a pre-trained key point detection model, and output the coordinates of at least two key position points of the external broken object in the power monitoring image; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on the key components of the external broken object;
[0009] According to the coordinates of each of the key position points, identifying the relative position and / or size ratio relationship between each of the key components;
[0010] According to the relative position and / or the size ratio, the posture state of the external breaking object is identified.
[0011] In one embodiment, the identifying the posture state of the external breaking object according to the relative position and / or the size ratio relationship includes:
[0012] Determining the layout ratio between the key components according to the relative positions between the key components;
[0013] The posture state recognition result of the external breaking object is obtained according to whether the layout ratio between the key components meets the preset layout ratio.
[0014] In one embodiment, the identifying the posture state of the external breaking object according to the relative position and / or the size ratio relationship includes:
[0015] According to the size ratio relationship, evaluating the shape of the external broken object;
[0016] According to whether the shape of the external breaking object conforms to a preset shape, a posture state recognition result of the external breaking object is obtained.
[0017] In one embodiment, the identifying the relative position and / or size ratio relationship between the key components according to the coordinates of the key position points includes:
[0018] According to the key components to which the key position points belong, based on the corresponding coordinates, determine the size information of each key component;
[0019] The size ratio relationship is determined according to the size information of each of the key components.
[0020] In one embodiment, the size information includes length information and width information; and determining the size ratio relationship according to the size information of each of the key components includes:
[0021] For the same key component, determining the length-to-width ratio information of the same key component according to the length information and width information of the same key component;
[0022] The size ratio relationship of the same key component is determined according to the length-to-width ratio information of the same key component.
[0023] In one embodiment, the key component includes a first key component and a second key component; the size information includes length information; and determining the size ratio relationship according to the size information of each of the key components includes:
[0024] Obtaining component length ratio information according to ratio information between the length information of the first key component and the length information of the second key component;
[0025] The size ratio relationship between the first key component and the second key component is determined according to the component length ratio information.
[0026] In a second aspect, the present application also provides a device for identifying postures of external objects based on a key point model, comprising:
[0027] An acquisition module is used to acquire a power monitoring image; the power monitoring image is an image containing external broken objects in the power operation site; the external broken objects include construction machinery and equipment;
[0028] A detection module, used for inputting the power monitoring image into a pre-trained key point detection model, and outputting the coordinates of at least two key position points of the external broken object in the power monitoring image; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on key components of the external broken object;
[0029] A component information identification module, used to identify the relative position and / or size ratio relationship between the key components according to the coordinates of the key position points;
[0030] The posture recognition module is used to recognize the posture state of the external breaking object according to the relative position and / or the size ratio relationship.
[0031] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above method are implemented.
[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.
[0033] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0034] The above-mentioned external breaking object posture recognition method, device, computer equipment, computer readable storage medium and computer program product based on the key point model obtains a power monitoring image; the power monitoring image is an image containing external breaking objects in the power operation site; the external breaking objects include construction machinery and equipment; the power monitoring image is input into a pre-trained key point detection model, and the coordinates of at least two key position points of the external breaking object in the power monitoring image are output; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on the key components of the external breaking object; according to the coordinates of each key position point, the relative position and / or size ratio relationship between the key components is identified; according to the relative position and / or size ratio relationship, the posture state of the external breaking object is identified.
[0035] In this way, the key position points of the power monitoring image containing the external breaking objects in the power operation site are detected through the pre-trained key point detection model. The pre-trained key point detection model can use deformable convolution to perform feature extraction processing on the power monitoring image. The deformable convolution can better capture the deformation and displacement of the object, so that the key position points can be detected more accurately in the power monitoring image, so that the coordinates of the output key position points are more accurate, and then the relative position and / or size ratio relationship between the key components of the external breaking object can be more accurately identified according to the coordinates of each key position point, so as to track the posture changes of the external breaking object during the movement more comprehensively according to the relative position and / or size ratio relationship, solve the problems of low posture recognition accuracy and insufficient recognition ability of objects with complex postures in related technologies, and realize accurate recognition of the posture of external breaking objects by introducing the key point detection model and combining it with the posture estimation technology, thereby improving the recognition performance and reliability of the external breaking object posture recognition technology in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 It is a flowchart of a method for identifying posture of an external object based on a key point model in one embodiment;
[0038] Figure 2 A schematic diagram of a power monitoring annotated image with key location points annotated in one embodiment;
[0039] Figure 3A schematic diagram of key position points detected in a power monitoring image by a pre-trained key point detection model in one embodiment;
[0040] Figure 4 is a flow chart of a method for identifying postures of external objects based on a key point model in another embodiment;
[0041] Figure 5 is a structural block diagram of a device for identifying postures of external objects based on a key point model in one embodiment;
[0042] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0044] In one embodiment, Figure 1 As shown, a method for predicting the trajectory of an external object based on key points is provided. This embodiment uses the method applied to a computer device as an example. It can be understood that the computer device can be a server, a terminal, or a system including a terminal and a server. In this embodiment, the method includes the following steps:
[0045] Step S110, acquiring a power monitoring image.
[0046] The power monitoring image is an image containing external broken objects in the power operation site.
[0047] Among them, the power operation site includes but is not limited to the transmission channel scene.
[0048] Among them, external broken objects include construction machinery and equipment.
[0049] Among them, construction machinery and equipment include but are not limited to excavators, cranes, tower cranes and other mechanical equipment that require human-machine cooperation / collaboration.
[0050] The power monitoring image may be a frame image in a video stream data containing external damage of construction machinery and equipment at a power operation site.
[0051] Step S120, inputting the power monitoring image into a pre-trained key point detection model, and outputting the coordinates of at least two key position points of the external object in the power monitoring image.
[0052] Among them, the coordinates of the key position points (key points) are obtained by using the pre-trained key point detection model to perform feature extraction processing on the power monitoring image using deformable convolution.
[0053] Among them, the key position points are located on the key parts of the external object. For example, the key parts can be the boom and body of a crane, the bucket and digging arm of an excavator, etc.; the key position points can be the joints of the mechanical arm, the center of the body, etc.
[0054] In a specific implementation, a computer device can input a power monitoring image into a pre-trained key point detection model. The pre-trained key point detection model can use deformable convolution to perform feature extraction processing on the power monitoring image, thereby locating the position of the external object in the power monitoring image and determining the positions of at least two key position points of the external object, and then outputting the coordinates of the key position points.
[0055] In practical applications, the pre-trained key point detection model can be the yolov8-pose key point model that uses DCNV3 (Deformable Convolutional Networks version 3) to optimize the DyHead.
[0056] DCNV3 is an improved deformable convolution operation that adjusts the sampling position of the convolution kernel by learning the offset to better capture the deformation and displacement of the object. Compared with ordinary convolution, DCNV3 can better adapt to irregularly shaped objects and local deformations.
[0057] When using the yolov8-pose model after DCNV3 to optimize DyHead, the detection and positioning capabilities of deformed objects and key points will be improved compared to the original model, especially in the subsequent processing of posture recognition tasks of construction machinery and equipment such as excavators and cranes, which can more accurately detect key position points for more accurate posture recognition.
[0058] Step S130, identifying the relative positions and / or size ratios between the key components according to the coordinates of the key position points.
[0059] The size ratio relationship may include the size ratio relationship of the same key component and the size ratio relationship between different key components.
[0060] In a specific implementation, the computer device may identify the relative positions and / or size ratios between the key components based on the coordinates of the key position points and the key components to which the key position points belong.
[0061] Furthermore, the computer device can determine the relative positions between the key position points based on the coordinates of the key position points, and then identify the relative positions between the key components based on the relative positions between the key position points and the key components to which the key position points belong.
[0062] The computer equipment can determine the distance ratio relationship between each key position point based on the coordinates of each key position point, and then identify the size ratio relationship between each key component based on the distance ratio relationship between each key position point and the key components to which each key position point belongs.
[0063] The distance ratio relationship between the key position points can be determined according to the ratio of the distances between the key position points. According to the distances between the key position points and the key components to which the key position points belong, the size information of the key components can be determined, so that the size ratio relationship between the key components can be obtained according to the size information of the key components. The size ratio relationship between the key components can be determined according to the ratio information between the size information of the key components.
[0064] Step S140, identifying the posture state of the external object according to the relative position and / or size ratio.
[0065] In a specific implementation, the computer device can identify the posture state of the external breaking object based on the relative position and / or size ratio, for example, whether the layout of the key components of the external breaking object conforms to the preset posture, whether the shape of the external breaking object meets the preset shape, whether the external breaking object is stationary, moving or operating, etc.
[0066] In some embodiments, the computer device can calculate the position, direction and other posture information of the construction machinery and equipment through geometric transformation based on the coordinates of the key position points. Algorithms such as ICP (Iterative Closest Point) and RANSAC (Random Sample Consensus) are used to optimize the alignment of key position points and improve the accuracy of posture estimation.
[0067] In the above-mentioned method for identifying the posture of external breaking objects based on the key point model, a power monitoring image is obtained; the power monitoring image is an image containing external breaking objects in the power operation site; the external breaking objects include construction machinery and equipment; the power monitoring image is input into a pre-trained key point detection model, and the coordinates of at least two key position points of the external breaking object in the power monitoring image are output; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on the key components of the external breaking object; according to the coordinates of each key position point, the relative position and / or size ratio relationship between the key components is identified; according to the relative position and / or size ratio relationship, the posture state of the external breaking object is identified.
[0068] In this way, the key position points of the power monitoring image containing the external breaking objects in the power operation site are detected through the pre-trained key point detection model. The pre-trained key point detection model can use deformable convolution to perform feature extraction processing on the power monitoring image. The deformable convolution can better capture the deformation and displacement of the object, so that the key position points can be detected more accurately in the power monitoring image, so that the coordinates of the output key position points are more accurate, and then the relative position and / or size ratio relationship between the key components of the external breaking object can be more accurately identified according to the coordinates of each key position point, so as to track the posture changes of the external breaking object during the movement more comprehensively according to the relative position and / or size ratio relationship, solve the problems of low posture recognition accuracy and insufficient recognition ability of objects with complex postures in related technologies, realize accurate recognition of the posture of external breaking objects, thereby improving the recognition performance and reliability of external breaking object posture recognition technology in complex environments.
[0069] In one embodiment, the posture state of the external breaking object is identified based on the relative position and / or size ratio, including: determining the layout ratio between the key components based on the relative positions between the key components; and obtaining the posture state recognition result of the external breaking object based on whether the layout ratio between the key components meets the preset layout ratio.
[0070] In a specific implementation, when the computer device identifies the posture state of the external breaking object according to the relative position and / or size ratio, the computer device can determine the layout ratio between the key components according to the relative positions between the key components. According to whether the layout ratio between the key components meets the preset layout ratio, the posture state recognition result of the external breaking object is obtained.
[0071] Among them, when the layout ratio does not meet the preset layout ratio, the posture state recognition result of the external broken object is determined to be that the layout between the key components does not conform to the normal posture; when the layout ratio meets the preset layout ratio, the posture state recognition result of the external broken object is determined to be that the layout between the key components conforms to the normal posture.
[0072] For example, the layout ratio between the crane boom position and the vehicle body position can be used to determine whether the crane boom is aligned with the vehicle body, and the layout ratio between the excavator bucket position and the excavator arm position can be used to determine whether the excavator bucket maintains a suitable relative position with the excavator arm.
[0073] The technical solution of this embodiment determines the layout ratio between the key components according to the relative positions between the key components; and obtains the posture state recognition result of the external object according to whether the layout ratio between the key components meets the preset layout ratio. In this way, the layout ratio between the key components can be determined more accurately according to the relative positions between the key components, so that the posture state of the external object can be more comprehensively recognized according to whether the layout ratio between the key components meets the preset layout ratio.
[0074] In one embodiment, the posture state of the external breaking object is identified based on the relative position and / or size ratio, including: evaluating the shape of the external breaking object based on the size ratio; and obtaining the posture state identification result of the external breaking object based on whether the shape of the external breaking object conforms to a preset shape.
[0075] In a specific implementation, when a computer device identifies the posture state of an external breaking object based on relative position and / or size ratio, the computer device can evaluate the shape of the external breaking object based on the size ratio between key components, and obtain a posture state recognition result of the external breaking object based on whether the shape of the external breaking object conforms to a preset shape.
[0076] Among them, when the shape of the external broken object does not conform to the preset shape, the posture state recognition result of the external broken object is determined to be abnormal in shape; when the shape of the external broken object conforms to the preset shape, the posture state recognition result of the external broken object is determined to be normal in shape.
[0077] In some embodiments, in the process of identifying the relative position and / or size ratio relationship between key components according to the coordinates of each key position point, the computer device can determine the size information of each key component according to the key component to which each key position point belongs, based on the corresponding coordinates. Specifically, the computer device can determine the distance between each key position point according to the coordinates of each key position point, and determine the size information of each key component according to the distance between each key position point and the key component to which it belongs. In this way, the computer device can determine the size ratio relationship between key components according to the size information of each key component.
[0078] In some embodiments, the size information includes length information and width information; when the computer device determines the size ratio relationship between key components based on the size information of each key component, the computer device can determine the length-to-width ratio information of the same key component based on the length-to-width ratio information of the same key component. In this way, the computer device can determine the size ratio relationship of the same key component based on the length-to-width ratio information of the same key component, and thus, the computer device can evaluate the shape of the external breaking object based on the size ratio relationship of the same key component, thereby identifying the posture state of the external breaking object.
[0079] For example, the key component may be a bucket of an excavator. By obtaining the ratio of the length to the width of the bucket of the excavator, it can be determined whether the shape of the excavator conforms to a preset shape.
[0080] In some embodiments, the key components include a first key component and a second key component; the size information includes length information; in the process of the computer device determining the size ratio relationship based on the size information of each key component, the computer device can obtain the component length ratio information based on the ratio information between the length information of the first key component and the length information of the second key component, and determine the size ratio relationship between the first key component and the second key component based on the component length ratio information, so that the shape of the external breaking object can be evaluated based on the size ratio relationship between the first key component and the second key component, thereby identifying the posture state of the external breaking object.
[0081] For example, the first key component and the second key component may be a crane boom and a crane body, and the computer device may obtain the ratio of the length of the crane boom to the length of the crane body to determine whether the shape of the crane conforms to a preset shape.
[0082] In other embodiments, the computer device can integrate the relative positions and size ratios between key components to identify the posture state of the external object, such as whether the external object is stationary, moving, or operating. In this way, the external object posture assessment method of the present application not only considers angles and distances, but also includes a comprehensive assessment of relative positions and ratios, which can more accurately and comprehensively identify the posture of the external object.
[0083] In some embodiments, the method also includes: obtaining a power monitoring sample image set and a power monitoring annotated image set; the power monitoring sample images in the power monitoring sample image set are sample images containing external objects in the power operation site; the power monitoring annotated image set includes power monitoring annotated images corresponding to each power monitoring sample image; the power monitoring annotated images are obtained by annotating the actual positions of the key position points of the external objects in the power monitoring sample images; each power monitoring sample image is input into a key point detection model to be trained, and a key point prediction image corresponding to each power monitoring sample image is output; according to the difference between the power monitoring annotated image corresponding to each power monitoring sample image and the key point prediction image corresponding to each power monitoring sample image, the key point detection model to be trained is trained to obtain a pre-trained key point detection model.
[0084] The computer device may obtain external object video stream data for training the key point detection model to be trained, and the external object video stream data may contain construction machinery and equipment at the power operation site, such as excavators and cranes, and other construction machinery and equipment in various working scenes and postures. Then, the computer device extracts multiple frames of images from the external object video stream data at a preset frequency, and uses the multiple frames of images as a sample image set for power monitoring.
[0085] Among them, it is necessary to mark key positions of each power monitoring sample image, and manually or automatically mark 6 key positions on each power monitoring sample image. For example, for excavators and cranes, the joints of the mechanical arm, the center of the vehicle body and other key positions are marked to obtain the corresponding power monitoring annotated image. In order to facilitate the understanding of those skilled in the art, Figure 2 A schematic diagram of an annotated power monitoring image with key location points is provided, such as Figure 2 As shown, the marked key position points may include key point 21, key point 22, key point 23, key point 24, key point 25 and key point 26. These key position points can fully describe the key components and posture information of the construction machinery and equipment.
[0086] In this way, the computer device can input each power monitoring sample image into the key point detection model to be trained, and output the key point prediction image corresponding to each power monitoring sample image. Among them, the key point prediction image can be an image containing the predicted position of the key position point. The computer device can train the key point detection model to be trained according to the difference between the power monitoring annotated image corresponding to each power monitoring sample image and the key point prediction image corresponding to each power monitoring sample image, and obtain a pre-trained key point detection model.
[0087] Specifically, the computer device can update the model parameters of the key point detection model to be trained according to the difference between the power monitoring annotated image corresponding to each power monitoring sample image and the key point prediction image corresponding to each power monitoring sample image, and retrain the key point detection model after the model parameters are updated until the trained key point detection model meets the preset training end condition, thereby obtaining a pre-trained key point detection model. In some embodiments, during the training process, transfer learning and data enhancement techniques can be used to improve model performance. To facilitate understanding by those skilled in the art, Figure 3 A schematic diagram of key position points detected by a pre-trained key point detection model in a power monitoring image is provided, and the detected key position points include key point 31, key point 32, key point 33, key point 34, key point 35 and key point 36.
[0088] In this way, by customizing the key position points of external broken objects and marking the key position points of external broken objects in the power monitoring sample images, data preparation before training is achieved, and then the yolov8-pose improvement of DyHead is optimized by DCNV3, and the pre-trained key point detection model is obtained to better identify the key position points of external broken objects. Finally, the relative position and size ratio relationship is combined to solve the problem of false detection of external broken objects in related technologies, meet the practical requirements of intelligent recognition of drone patrol images, and ensure the safe and stable operation of transmission lines and power grids.
[0089] In another embodiment, Figure 4 As shown, a method for identifying postures of external objects based on a key point model is provided, and the method is applied to a computer device as an example for explanation, and includes the following steps:
[0090] Step S402, obtaining a power monitoring image, inputting the power monitoring image into a pre-trained key point detection model, and outputting the coordinates of at least two key position points of the external object in the power monitoring image.
[0091] Step S404, according to the key components to which the key position points belong, based on the corresponding coordinates, determine the size information of each key component.
[0092] Step S406, determining the size ratio relationship between the key components according to the size information of the key components.
[0093] Step S408, identifying the relative positions of the key components according to the coordinates of the key position points.
[0094] Step S410, identifying the posture state of the external object according to the relative position and / or size ratio.
[0095] It should be noted that the specific definition of the above steps can refer to the specific definition of a method for identifying the posture of an external object based on a key point model mentioned above.
[0096] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0097] Based on the same inventive concept, the embodiment of the present application also provides a device for identifying postures of external objects based on a key point model for implementing the above-mentioned method for identifying postures of external objects based on a key point model. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the device for identifying postures of external objects based on a key point model provided below can refer to the limitations of the method for identifying postures of external objects based on a key point model above, and will not be repeated here.
[0098] In an exemplary embodiment, Figure 5 As shown, a device for identifying postures of external objects based on a key point model is provided, comprising: an acquisition module 510, a detection module 520, a component information identification module 530 and a posture recognition module 540, wherein:
[0099] The acquisition module 510 is used to acquire a power monitoring image; the power monitoring image is an image containing external broken objects in the power operation site; the external broken objects include construction machinery and equipment.
[0100] The detection module 520 is used to input the power monitoring image into a pre-trained key point detection model, and output the coordinates of at least two key position points of the external broken object in the power monitoring image; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on the key components of the external broken object.
[0101] The component information identification module 530 is used to identify the relative position and / or size ratio relationship between the key components according to the coordinates of the key position points.
[0102] The posture recognition module 540 is used to recognize the posture state of the external object according to the relative position and / or the size ratio.
[0103] In one embodiment, the posture recognition module 540 is specifically used to determine the layout ratio between the key components according to the relative positions between the key components; and obtain the posture state recognition result of the external object according to whether the layout ratio between the key components meets the preset layout ratio.
[0104] In one embodiment, the posture recognition module 540 is specifically used to evaluate the shape of the external breaking object according to the size ratio relationship; and obtain the posture state recognition result of the external breaking object according to whether the shape of the external breaking object meets the preset shape.
[0105] In one embodiment, the component information identification module 530 is specifically used to determine the size information of each key component according to the key component to which each key position point belongs based on the corresponding coordinates; and determine the size ratio relationship according to the size information of each key component.
[0106] In one embodiment, the size information includes length information and width information; the component information identification module 530 is specifically used to determine the length-to-width ratio information of the same key component according to the length information and width information of the same key component; and determine the size ratio relationship of the same key component according to the length-to-width ratio information of the same key component.
[0107] In one embodiment, the key component includes a first key component and a second key component; the size information includes length information; the component information identification module 530 is specifically used to obtain component length ratio information based on the ratio information between the length information of the first key component and the length information of the second key component; and determine the size ratio relationship between the first key component and the second key component based on the component length ratio information.
[0108] Each module in the above-mentioned device for identifying postures of foreign objects based on key point models can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.
[0109] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store model parameter data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying the posture of an external object based on a key point model is implemented.
[0110] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0111] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0112] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0113] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0115] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0117] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for identifying posture of external objects based on key point model, characterized in that: The method comprises: Acquire a power monitoring image; the power monitoring image is an image containing external broken objects in the power operation site; the external broken objects include construction machinery and equipment; Input the power monitoring image into a pre-trained key point detection model, and output the coordinates of at least two key position points of the external broken object in the power monitoring image; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on the key components of the external broken object; According to the coordinates of each of the key position points, identifying the relative position and / or size ratio relationship between each of the key components; According to the relative position and / or the size ratio, the posture state of the external breaking object is identified.
2. The method according to claim 1, characterized in that The step of identifying the posture state of the external breaking object according to the relative position and / or the size ratio includes: Determining the layout ratio between the key components according to the relative positions between the key components; The posture state recognition result of the external breaking object is obtained according to whether the layout ratio between the key components meets the preset layout ratio.
3. The method according to claim 1, characterized in that The step of identifying the posture state of the external breaking object according to the relative position and / or the size ratio includes: According to the size ratio relationship, evaluating the shape of the external broken object; According to whether the shape of the external breaking object conforms to a preset shape, a posture state recognition result of the external breaking object is obtained.
4. The method according to claim 1, characterized in that The step of identifying the relative positions and / or size ratios between the key components according to the coordinates of the key position points includes: According to the key components to which the key position points belong, based on the corresponding coordinates, determine the size information of each key component; The size ratio relationship is determined according to the size information of each of the key components.
5. The method according to claim 4, characterized in that The size information includes length information and width information; and determining the size ratio relationship according to the size information of each of the key components includes: For the same key component, determining the length-to-width ratio information of the same key component according to the length information and width information of the same key component; The size ratio relationship of the same key component is determined according to the length-to-width ratio information of the same key component.
6. The method according to claim 4, characterized in that The key components include a first key component and a second key component; the size information includes length information; and determining the size ratio relationship according to the size information of each of the key components includes: Obtaining component length ratio information according to ratio information between the length information of the first key component and the length information of the second key component; The size ratio relationship between the first key component and the second key component is determined according to the component length ratio information.
7. A device for identifying posture of external objects based on key point model, characterized in that: The device comprises: An acquisition module is used to acquire a power monitoring image; the power monitoring image is an image containing external broken objects in the power operation site; the external broken objects include construction machinery and equipment; A detection module, used for inputting the power monitoring image into a pre-trained key point detection model, and outputting the coordinates of at least two key position points of the external broken object in the power monitoring image; the coordinates of the key position points are obtained by the pre-trained key point detection model using deformable convolution to perform feature extraction processing on the power monitoring image; the key position points are located on key components of the external broken object; A component information identification module, used to identify the relative position and / or size ratio relationship between the key components according to the coordinates of the key position points; The posture recognition module is used to recognize the posture state of the external breaking object according to the relative position and / or the size ratio relationship.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Excavator working state identification method based on hybrid LBF shape regression model
CN106650622A
Conflict behavior detection method and device, computer device and storage medium
CN111563480A
Image feature point extraction method and system
CN114140623A
Mechanical monomer construction operation efficiency detection method and system based on posture recognition
CN114418310A
Underground pipe network risk early warning method, device and equipment and computer storage medium
CN117218605A
