Target object direction detection method, device, electronic device and storage medium

Abstract

The target object direction detection method, device, electronic device and storage medium provided by the embodiments of the present invention include determining the target area where the target object is located in a first image, intercepting a second image of the target area, determining the human body key points of the target object based on the second image, and when there are specific key point pairs in the human body key points, calculating the human body angle score corresponding to the target object based on the specific key point pairs, and determining the direction of the target object in the first image according to the human body angle score. Based on the 2D human body key point detection algorithm, the present invention can quantitatively calculate the human body angle score, and accurately determine the human body direction based on the human body angle score. It not only avoids the problem of unclear label boundaries in the deep learning classification algorithm, but also compared with the data collection and annotation of the 3D model algorithm, the data acquisition cost of the 2D key point detection algorithm of the present application is much smaller, solving the technical problems of low accuracy and high cost of human body direction detection.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of target detection technology, and in particular to a method, device, electronic device, and storage medium for detecting the direction of a target object. Background Art

[0002] With the development of computer technology and the widespread application of computer vision principles, research on real-time target tracking using computer image processing techniques has become increasingly popular. Dynamic, real-time target tracking and positioning has broad applications in intelligent transportation systems, intelligent surveillance systems, military target detection, and surgical instrument positioning during medical navigation surgery. Commonly used by ordinary users, for example, this method can be used to analyze body orientation information (such as facing the camera, sideways, or back) from exercise and fitness videos, allowing for physical training analysis. Another example is the use of this method in film and television dramas, where character orientation information (such as facing the camera, sideways, or back) can be used to analyze body posture and facilitate keyframe selection.

[0003] Currently, there are two common methods for detecting the direction of a person in a video: one is to classify the person's posture using a deep learning algorithm, and the other is to make judgments using the person's 3D information. However, both existing methods fail to meet user needs due to their own inherent problems and cannot be widely promoted. For example, the method of using a deep learning algorithm to classify the person's posture suffers from low classification accuracy and blurred boundaries between labels. For example, when the person's angle of facing the camera is between 30 and 60 degrees, it is difficult to determine whether it should be classified as facing the camera head-on or sideways. In the method of using the person's 3D information for judgment, the 3D model suffers from high computational complexity and difficulty in collecting data. 3D data requires specialized equipment for data collection and labeling, and the cost of acquiring data is enormous.

[0004] Currently, no effective solution has been proposed to address the problems of low accuracy and high cost in human direction detection in related technologies. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, device, electronic device, and storage medium for detecting the direction of a target object, which can effectively improve the accuracy of human body direction detection and reduce detection costs.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting the direction of a target object, wherein the method includes:

[0007] Determining a target area where a target object is located in a first image; wherein the first image is a single-frame image in a video sequence;

[0008] intercepting a second image of the target area, and determining key points of a human body of the target object based on the second image;

[0009] When a specific key point pair exists among the key points of the human body, a human body angle score corresponding to the target object is calculated based on the specific key point pair; wherein the specific key point pair includes a first specific key point and a second specific key point, and a first human body part corresponding to the first specific key point and a second human body part corresponding to the second specific key point are symmetrical human body parts in the human body structure;

[0010] The orientation of the target object in the first image is determined according to the human body angle score.

[0011] In one possible implementation, determining a target area where the target object is located in the first image includes:

[0012] Inputting the first image into the first neural network model to extract position features of the first image using the first neural network model to obtain a first feature map of the first image;

[0013] Generate multiple anchor boxes for each point on the first feature map, where each anchor box has a different size;

[0014] For each anchor box, calculate the confidence that the target object exists in the anchor box;

[0015] The first target anchor box whose confidence is greater than or equal to the first threshold is retained, and the area surrounded by the first target anchor box is determined as the target area where the target object is located.

[0016] In one possible implementation, determining key points of a target object based on the second image includes:

[0017] Inputting the second image into the second neural network model to extract human features from the second image using the second neural network model to obtain a second feature map of the second image;

[0018] Converting the second feature map into multiple heat maps, wherein one heat map corresponds to one preset human body key point;

[0019] Determine the heat value of each point on the heat map in each heat map, and sort each point on the heat map according to the heat value;

[0020] Performing weighted averaging on the coordinate positions of the points preceding the target sort position to obtain coordinate information of the predicted human body key points corresponding to each preset human body key point, wherein the human body key point includes the coordinate information of the predicted human body key point;

[0021] Draw the predicted human body key points in the second image according to the coordinate information.

[0022] In one possible implementation, calculating the human body angle score corresponding to the target object based on a specific key point pair includes:

[0023] In the second image, obtaining first horizontal coordinate information of the first specific key point and second horizontal coordinate information of the second specific key point;

[0024] A human body angle score corresponding to the target object is calculated based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image.

[0025] In one possible implementation, calculating the human body angle score corresponding to the target object based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image includes:

[0026] Determining a normalized angle score of the key point based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image;

[0027] determining a front-side-backside judgment score based on the first horizontal coordinate information and the second horizontal coordinate information;

[0028] The human body angle score corresponding to the target object is calculated based on the normalized score of the key point angle and the front and back judgment score.

[0029] In one possible implementation, determining the front-side / back-side judgment score based on the first horizontal coordinate information and the second horizontal coordinate information includes:

[0030] When the first horizontal coordinate information is greater than the second horizontal coordinate information, the first preset value is determined as the front-side and back-side judgment score; wherein the first preset value is a positive value; or

[0031] When the first horizontal coordinate information is not greater than the second horizontal coordinate information, the second preset value is determined as the front-side and back-side judgment score; wherein the second preset value is a negative value.

[0032] In one possible implementation, determining the orientation of the target object in the first image according to the human body angle score includes:

[0033] Searching a direction query table for a target score range corresponding to a human body angle score; wherein the direction query table stores a plurality of different directions and a score range corresponding to each direction;

[0034] The target direction corresponding to the target score range is determined as the direction of the target object in the first image.

[0035] In one possible implementation, the method further includes:

[0036] When there is no specific key point pair for the human body key points, an incomplete human body prompt information is generated.

[0037] In a second aspect, an embodiment of the present invention provides a device for detecting the direction of a target object, wherein the device includes:

[0038] A first determining module is configured to determine a target area where a target object is located in a first image; wherein the first image is a single-frame image in a video sequence;

[0039] A second determining module is used to intercept a second image of the target area and determine the human body key points of the target object based on the second image;

[0040] a calculation module, configured to calculate a human body angle score corresponding to the target object based on a specific key point pair when a specific key point pair exists among the human body key points; wherein the specific key point pair includes a first specific key point and a second specific key point, and a first human body part corresponding to the first specific key point and a second human body part corresponding to the second specific key point are symmetrical human body parts in the human body structure;

[0041] The third determination module is configured to determine a direction of the target object in the first image according to the human body angle score.

[0042] In a third aspect, an embodiment of the present invention provides an electronic device, which includes: a processor and a memory, wherein the processor is used to execute a target object direction detection program stored in the memory to implement the above-mentioned target object direction detection method.

[0043] In a fourth aspect, an embodiment of the present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned target object direction detection method.

[0044] The target object direction detection method, device, electronic device and storage medium provided by the embodiments of the present invention include determining the target area where the target object is located in a first image, intercepting a second image of the target area, determining the human body key points of the target object based on the second image, and when there are specific key point pairs in the human body key points, calculating the human body angle score corresponding to the target object based on the specific key point pairs, and determining the direction of the target object in the first image according to the human body angle score. Based on the 2D human body key point detection algorithm, the present invention can quantitatively calculate the human body angle score, and accurately determine the human body direction based on the human body angle score, which not only avoids the problem of unclear label boundaries in the deep learning classification algorithm, but also improves the detection accuracy. Moreover, compared with the data collection and labeling of the 3D model algorithm, the data acquisition cost of the 2D key point detection algorithm of the present application is much smaller, which solves the technical problems of low accuracy and high cost of human body direction detection in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the hardware environment of a method for detecting the direction of a target object provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a flow chart of a method for detecting the direction of a target object provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the distribution of preset key points of the human body provided in an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of the direction of a target object provided by an embodiment of the present invention;

[0049] Figure 5 A block diagram of an embodiment of a target object direction detection device provided by an embodiment of the present invention;

[0050] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0052] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below with reference to the accompanying drawings. The embodiments do not limit the embodiments of the present invention.

[0053] In this embodiment, the above-mentioned target object direction detection method can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, the server 103 is connected to the terminal 101 via a network, and can be used to provide services (such as direction detection services) for the terminal or a client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network, and the terminal 101 includes but is not limited to a PC, a mobile phone, a tablet computer, etc.

[0054] A method for detecting the direction of a target object in this embodiment can be executed by the server 103, or can be executed jointly by the server 103 and the terminal 101, such as Figure 2 As shown, the method may include the following steps:

[0055] Step 201, determining a target area where a target object is located in a first image;

[0056] Among them, the first image is a single-frame image in a video sequence; specifically, the first image is a single-frame image in a video sequence, and the video sequence includes but is not limited to videos in various existing business fields, such as surveillance videos in the security field, sports and fitness recording videos, videos of cultural film and television works, etc. The single-frame image is an image including the target object obtained by extracting frames from the video sequence.

[0057] Step 202: capturing a second image of the target area, and determining key points of the target object based on the second image;

[0058] Step 203: If a specific key point pair exists among the key points of the human body, a human body angle score corresponding to the target object is calculated based on the specific key point pair;

[0059] The specific key point pair includes a first specific key point and a second specific key point, and the first human body part corresponding to the first specific key point and the second human body part corresponding to the second specific key point are symmetrical human body parts in the human body structure.

[0060] The specific key point pair is pre-set, and usually two symmetrical human key points on the upper body are set as the specific key point pair. In actual application, the human key points at the left and right shoulders can be set as the specific key point pair, or the human key points at the left and right chests can be set as the specific key point pair. There is no limitation here.

[0061] When a specific key point pair exists among the key points of the human body, it indicates that the human body of the target object is relatively complete. The angle score of the human body of the target object facing the camera, ie, the human body angle score, can be calculated based on the specific key point pair.

[0062] If the specific key point pair does not exist in the key points of the human body, it indicates that the human body of the target object is incomplete. Therefore, in this embodiment, if the specific key point pair does not exist in the key points of the human body, a prompt message indicating that the human body is incomplete is generated. The generated prompt message can be sent to the display terminal of the execution subject of the direction detection method for display.

[0063] Step 204: Determine the direction of the target object in the first image according to the human body angle score.

[0064] The method for detecting the direction of a target object provided by an embodiment of the present invention includes determining a target area where the target object is located in a first image, intercepting a second image of the target area, determining the human body key points of the target object based on the second image, and when there are specific key point pairs in the human body key points, calculating the human body angle score corresponding to the target object based on the specific key point pairs, and determining the direction of the target object in the first image according to the human body angle score. Based on the 2D human body key point detection algorithm, the present invention can quantitatively calculate the human body angle score, and accurately determine the human body direction based on the human body angle score, which not only avoids the problem of unclear label boundaries in the deep learning classification algorithm, but also improves the detection accuracy. Moreover, compared with the data collection and labeling of the 3D model algorithm, the data acquisition cost of the 2D key point detection algorithm of the present application is much smaller, which solves the technical problems of low accuracy and high cost of human body direction detection in related technologies.

[0065] In some embodiments, the above step 201 can be implemented by the following steps:

[0066] Step A1: inputting a first image into a first neural network model to extract position features of the first image using the first neural network model to obtain a first feature map of the first image;

[0067] In order to meet the input requirements of the first neural network model, the first image can be scaled to a fixed size and then input into the first neural network model. The first neural network model can be trained based on deep learning networks such as convolutional neural networks and recurrent neural networks. The first neural network model is used to extract the position features of the human body area in the first image to obtain a first feature map.

[0068] Step A2: generating multiple anchor boxes for each point on the first feature map, where each anchor box has a different size;

[0069] Each generated anchor frame has a different size and can be used to match human targets of different sizes. The number of anchor frames can be set according to actual needs and is not limited here.

[0070] In step A3, for each anchor frame, the confidence that the target object exists in the anchor frame is calculated; in step A4, the first target anchor frame whose confidence is greater than or equal to the first threshold is retained, and the area surrounded by the first target anchor frame is determined as the target area where the target object is located.

[0071] Specifically, the score (confidence) of the presence of the target object in each anchor frame in the first feature map can be calculated, and thus the presence of a human body in each anchor frame can be determined based on the score. The anchor frame containing the human body is the first target anchor frame whose confidence is greater than or equal to a first threshold. The area surrounded by the first target anchor frame is the target area where the target object is located in the first image. The first threshold can be set according to actual needs and is not limited here.

[0072] In order to avoid repeated detection when multiple anchor frames hit the same object, the non-maximum suppression algorithm can be used to select the best one from multiple anchor frames. Specifically, when there are multiple first target anchor frames, the intersection-and-union ratio of the multiple first target anchor frames is determined; when the intersection-and-union ratio is greater than or equal to the second threshold, the second target anchor frame with the highest confidence is retained, and the area surrounded by the second target anchor frame is determined as the target area where the target object is located.

[0073] First, the intersection-and-union ratios of multiple first target anchor frames are calculated, and then the intersection-and-union ratios are used to identify which first target anchor frames hit the same human body. Among them, the first target anchor frames whose intersection-and-union ratios are greater than or equal to the second threshold are the anchor frames that hit the same human body. The second threshold can be set according to actual needs; then, from the multiple first target anchor frames that hit the same human body, the second target anchor frame with the highest confidence is selected as the anchor frame of the final human area detection result, and the area surrounded by the second target anchor frame is determined as the target area where the target object is located.

[0074] In this embodiment, in addition to using the anchor frame-based method in step 201 to determine the target area of ​​the target object, other target detection methods may also be used, which are not limited here.

[0075] In some embodiments, the above step 202 can be implemented by the following steps:

[0076] Step B1: inputting the second image into a second neural network model to extract human features from the second image using the second neural network model to obtain a second feature map of the second image;

[0077] To meet the input requirements of the second neural network model, the second image can be scaled to a fixed size before being input into the second neural network model. The second neural network model can be trained based on a deep learning network such as a convolutional neural network or a recurrent neural network. The second neural network model is used to extract the human features of the target object in the second image and obtain a second feature map.

[0078] Step B2: converting the second feature map into multiple heat maps, wherein one heat map corresponds to one preset human body key point;

[0079] Step B3, determining the thermal value of each point on the thermal map in each thermal map, and sorting each point on the thermal map according to the thermal value;

[0080] like Figure 3 As shown, the actual positions of multiple preset human body key points can be predicted, and the above-mentioned preset human body key points include key points of the top of the head, left ear, right ear, left eye, right eye, nose, left shoulder, right shoulder, left elbow, left wrist, right elbow, right wrist, left hip, right hip, left knee, left ankle, right knee, right ankle, left chest, right chest, navel and other human body parts.

[0081] The number of the above-mentioned heat maps is the same as the number of preset human body key points and corresponds one to one. The heat value of each point on each heat map is the probability value of the point belonging to the corresponding preset human body key point, and the probability value is calculated by the second neural network model.

[0082] Step B4, performing weighted averaging on the coordinate positions of the points preceding the target sort position to obtain coordinate information of the predicted human body key points corresponding to each preset human body key point, wherein the human body key points include the coordinate information of the predicted human body key points;

[0083] The target ranking position can be the top 2, top 3, etc., and can be set according to actual needs. The coordinate positions of the points before the target ranking position are weighted averaged to obtain the actual position of the predicted human body key point corresponding to the preset human body key point.

[0084] Step B5: Draw the predicted human body key points in the second image according to the coordinate information.

[0085] The predicted human key points are the human key points of the target object determined based on the second image. The determined human key points include one or more of the following: top of the head, left ear, right ear, left eye, right eye, nose, left shoulder, right shoulder, left elbow, left wrist, right elbow, right wrist, left hip, right hip, left knee, left ankle, right knee, right ankle, left chest, right chest, and navel.

[0086] In this embodiment, in addition to the human body key point detection method based on the heat map in the above step 202, other human body key point detection methods can also be used, which are not limited here.

[0087] In some embodiments, the above step 203 can be implemented by the following steps:

[0088] Step C1: acquiring first horizontal coordinate information of a first specific key point and second horizontal coordinate information of a second specific key point in the second image;

[0089] Typically, the lower left corner of the image is used as the image coordinate origin of the second image, the first horizontal coordinate information is the horizontal distance of the first specific key point relative to the image coordinate origin, and the second horizontal coordinate information is the horizontal distance of the second specific key point relative to the image coordinate origin.

[0090] Step C2: Calculate the human body angle score corresponding to the target object based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image.

[0091] The horizontal width information is the actual horizontal width of the second image, which can be obtained by calculating the point coordinates of the lower right corner of the second image and the image coordinate origin. For example, if the point coordinates of the lower right corner of the second image are (4, 0), then the horizontal width information is 4.

[0092] The human body angle score can be calculated from steps D1 to D3:

[0093] Step D1, determining a normalized angle score of a key point based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image;

[0094] The specific process of calculating the normalized score of the key point angle is to first calculate the horizontal axis distance between the first specific key point and the second specific key point based on the first horizontal coordinate information and the second horizontal coordinate information, and then divide the calculated horizontal axis distance by the horizontal width information to obtain the normalized score of the key point angle.

[0095] In this embodiment, the left shoulder is taken as the first specific key point and the right shoulder is taken as the second specific key point as an example, where the first horizontal coordinate information of the left shoulder is x1, the second horizontal coordinate information of the right shoulder is x2, and the horizontal width information of the second image is w. The normalized score of the key point angle can be calculated by the following formula: |x1-x2| / w.

[0096] Step D2, determining a front-side and back-side judgment score based on the first horizontal coordinate information and the second horizontal coordinate information;

[0097] In this embodiment, the front-back judgment score is used to indicate the direction of the target object facing the camera. If the front-back judgment score is positive, it indicates that the target object is a human body facing the camera from the front or side. If the front-back judgment score is negative, it indicates that the target object is a human body facing the camera from the back.

[0098] The specific process of determining the front-side and back-side judgment score is: when the first horizontal coordinate information is greater than the second horizontal coordinate information, the first preset value is determined as the front-side and back-side judgment score; wherein the first preset value is a positive value; or, when the first horizontal coordinate information is not greater than the second horizontal coordinate information, the second preset value is determined as the front-side and back-side judgment score; wherein the second preset value is a negative value.

[0099] The first preset value and the second preset value can be set according to actual needs and are not limited here.

[0100] Step D3: Calculate the human body angle score corresponding to the target object based on the normalized key point angle score and the front and back judgment score.

[0101] The human body angle score is obtained by multiplying the normalized key point angle score by the front and back judgment score. Step 203 above can quantitatively calculate the human body angle score of the target object facing the camera. Furthermore, based on the human body angle score, the direction of the target object facing the camera in the first image can be accurately detected, effectively improving the accuracy of human body direction detection.

[0102] In some embodiments, the above step 204 can be implemented by the following steps:

[0103] Step E1, searching the direction lookup table for the target score range corresponding to the human body angle score;

[0104] The direction query table stores multiple different directions and the score range corresponding to each direction. In this embodiment, the directions are mainly divided into front, side and back. For ease of understanding, Figure 4 As shown, different directions are shown. Based on the above description, as shown in Table 1 below, it is an example of a direction query table:

[0105] direction Score range front Greater than 0.45 side [0.45，-045] back Less than -0.45

[0106] It should be noted that Table 1 only shows an example of the correspondence between directions and score ranges. The specific correspondence between directions and score ranges can be set according to actual needs and is not limited here.

[0107] Step E2: determining the target direction corresponding to the target score range as the direction of the target object in the first image.

[0108] If the human body angle score is 0.3, referring to Table 1, the human body angle score is in the range of [0.45, -0.45], then the direction corresponding to the human body angle score is the side, and it can be determined that the direction of the target object in the first image is the side.

[0109] See also Figure 5 , is a block diagram of an embodiment of a target object direction detection device provided by an embodiment of the present invention; Figure 5 As shown, the device may include:

[0110] A first determining module 501 is configured to determine a target area where a target object is located in a first image; wherein the first image is a single frame image in a video sequence;

[0111] A second determining module 502 is configured to capture a second image of the target area and determine key points of the target object based on the second image;

[0112] A calculation module 503 is configured to calculate a human body angle score corresponding to the target object based on a specific key point pair when a specific key point pair exists among the human body key points; wherein the specific key point pair includes a first specific key point and a second specific key point, and a first human body part corresponding to the first specific key point and a second human body part corresponding to the second specific key point are symmetrical human body parts in the human body structure;

[0113] The third determining module 504 is configured to determine the direction of the target object in the first image according to the human body angle score.

[0114] The first determining module 501 is further configured to:

[0115] Inputting the first image into the first neural network model to extract position features of the first image using the first neural network model to obtain a first feature map of the first image;

[0116] Generate multiple anchor boxes for each point on the first feature map, where each anchor box has a different size;

[0117] For each anchor box, calculate the confidence that the target object exists in the anchor box;

[0118] The first target anchor box whose confidence is greater than or equal to the first threshold is retained, and the area surrounded by the first target anchor box is determined as the target area where the target object is located.

[0119] The second determining module 502 is further configured to:

[0120] Inputting the second image into the second neural network model to extract human features from the second image using the second neural network model to obtain a second feature map of the second image;

[0121] Converting the second feature map into multiple heat maps, wherein one heat map corresponds to one preset human body key point;

[0122] Determine the heat value of each point on the heat map in each heat map, and sort each point on the heat map according to the heat value;

[0123] Performing weighted averaging on the coordinate positions of the points preceding the target sort position to obtain coordinate information of the predicted human body key points corresponding to each preset human body key point, wherein the human body key point includes the coordinate information of the predicted human body key point;

[0124] Draw the predicted human body key points in the second image according to the coordinate information.

[0125] The calculation module 503 is further configured to:

[0126] In the second image, obtaining first horizontal coordinate information of the first specific key point and second horizontal coordinate information of the second specific key point;

[0127] A human body angle score corresponding to the target object is calculated based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image.

[0128] The calculation module 503 is further configured to:

[0129] Determining a normalized angle score of the key point based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image;

[0130] determining a front-side-backside judgment score based on the first horizontal coordinate information and the second horizontal coordinate information;

[0131] The human body angle score corresponding to the target object is calculated based on the normalized score of the key point angle and the front and back judgment score.

[0132] The calculation module 503 is further configured to:

[0133] When the first horizontal coordinate information is greater than the second horizontal coordinate information, the first preset value is determined as the front-side and back-side judgment score; wherein the first preset value is a positive value; or

[0134] When the first horizontal coordinate information is not greater than the second horizontal coordinate information, the second preset value is determined as the front-side and back-side judgment score; wherein the second preset value is a negative value.

[0135] The third determining module 504 is further configured to:

[0136] Searching a direction query table for a target score range corresponding to a human body angle score; wherein the direction query table stores a plurality of different directions and a score range corresponding to each direction;

[0137] The target direction corresponding to the target score range is determined as the direction of the target object in the first image.

[0138] The above device also includes (not shown in the figure):

[0139] When there is no specific key point pair for the human body key points, an incomplete human body prompt information is generated.

[0140] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is provided. Figure 6The electronic device 500 shown includes: at least one processor 501, a memory 502, at least one network interface 504 and another user interface 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 505 is not shown in FIG. Figure 6 Various buses are labeled as bus system 505.

[0141] The user interface 503 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touchpad, or a touch screen).

[0142] It is understood that the memory 502 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0143] In some embodiments, the memory 502 stores the following elements, executable units, or data structures, or a subset thereof, or an extended set thereof: an operating system 5021 and application programs 5022 .

[0144] The operating system 5021 includes various system programs, such as a framework layer, a core library layer, and a driver layer, for implementing various basic services and handling hardware-based tasks. Application programs 5022 include various application programs, such as a media player and a browser, for implementing various application services. Programs implementing the methods of the embodiments of the present invention may be included in application programs 5022.

[0145] In the embodiment of the present invention, the processor 501 is configured to execute the method steps provided in each method embodiment by calling a program or instruction stored in the memory 502 , specifically, a program or instruction stored in the application 5022 .

[0146] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 501 or by software instructions. The above processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software units in the decoding processor. The software units can be located in storage media well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 502 , and the processor 501 reads the information in the memory 502 and completes the steps of the above method in combination with its hardware.

[0147] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof.

[0148] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0149] The electronic device provided in this embodiment may be Figure 6 The electronic device shown in FIG. 1 can perform the following operations: Figure 2 All steps of the target object direction detection method in Figure 2 For details on the technical effects of the target object direction detection method, please refer to Figure 2 For the sake of brevity, the relevant description will not be repeated here.

[0150] An embodiment of the present invention further provides a storage medium (computer-readable storage medium). The storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and the memory may also include a combination of the aforementioned types of memory.

[0151] When one or more programs in the storage medium can be executed by one or more processors, the above-mentioned method for detecting the direction of the target object can be implemented.

[0152] The processor is used to execute a target object direction detection program stored in a memory to implement the following steps of a target object direction detection method.

[0153] Professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0154] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0155] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting the direction of a target object, characterized in that: The method comprises: Determining a target area where a target object is located in a first image; wherein the first image is a single-frame image in a video sequence; intercepting a second image of the target area, and determining key points of a human body of the target object based on the second image; In the case where a specific key point pair exists among the human body key points, a human body angle score corresponding to the target object is calculated based on the specific key point pair; wherein the specific key point pair includes a first specific key point and a second specific key point, and a first human body part corresponding to the first specific key point and a second human body part corresponding to the second specific key point are symmetrical human body parts in the human body structure; The calculating the human body angle score corresponding to the target object based on the specific key point pair includes: In the second image, obtaining first horizontal coordinate information of the first specific key point and second horizontal coordinate information of the second specific key point; Calculating a human body angle score corresponding to the target object based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image, including: determining a key point angle normalization score based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image; determining a front-side and back-side judgment score based on the first horizontal coordinate information and the second horizontal coordinate information; and calculating a human body angle score corresponding to the target object based on the key point angle normalization score and the front-side and back-side judgment score, where the human body angle score is obtained by multiplying the key point angle normalization score by the front-side and back-side judgment score. Determining the direction of the target object in the first image based on the human body angle score includes: searching a target score range corresponding to the human body angle score in a direction lookup table; wherein the direction lookup table stores a plurality of different directions and a score range corresponding to each of the directions; and determining the target direction corresponding to the target score range as the direction of the target object in the first image.

2. The method according to claim 1, characterized in that The determining of a target area where the target object is located in the first image includes: Inputting the first image into a first neural network model to extract position features of the first image using the first neural network model to obtain a first feature map of the first image; generating a plurality of anchor boxes for each point on the first feature map, wherein each anchor box has a different size; For each anchor frame, calculating the confidence that the target object exists in the anchor frame; The first target anchor frame whose confidence is greater than or equal to the first threshold is retained, and the area surrounded by the first target anchor frame is determined as the target area where the target object is located.

3. The method according to claim 1, characterized in that The determining of the human body key points of the target object based on the second image includes: Inputting the second image into a second neural network model to extract human features from the second image using the second neural network model to obtain a second feature map of the second image; Converting the second feature map into multiple heat maps, wherein one heat map corresponds to one preset human body key point; Determining a thermal value of each point on each of the thermal maps, and sorting each point on the thermal map according to the thermal value; Performing weighted averaging on the coordinate positions of the points preceding the target sorting position to obtain coordinate information of the predicted human body key points corresponding to each of the preset human body key points, wherein the human body key points include the coordinate information of the predicted human body key points; The predicted human body key points are drawn in the second image according to the coordinate information.

4. The method according to claim 1, wherein The determining of the front-side and back-side judgment scores based on the first horizontal coordinate information and the second horizontal coordinate information includes: In the case where the first horizontal coordinate information is greater than the second horizontal coordinate information, a first preset value is determined as the front-side and back-side judgment score; wherein the first preset value is a positive value; or When the first horizontal coordinate information is not greater than the second horizontal coordinate information, a second preset value is determined as the front-side and back-side judgment score; wherein the second preset value is a negative value.

5. The method according to claim 1, wherein The method further comprises: When no specific key point pair exists among the key points of the human body, prompt information indicating that the human body is incomplete is generated.

6. A device for detecting the direction of a target object, characterized in that: The device comprises: A first determining module is configured to determine a target area where a target object is located in a first image; wherein the first image is a single-frame image in a video sequence; A second determining module is used to intercept a second image of the target area and determine the human body key points of the target object based on the second image; a calculation module, configured to calculate, when a specific key point pair exists among the human body key points, a human body angle score corresponding to the target object based on the specific key point pair; wherein the specific key point pair includes a first specific key point and a second specific key point, and a first human body part corresponding to the first specific key point and a second human body part corresponding to the second specific key point are symmetrical human body parts in the human body structure; The calculating the human body angle score corresponding to the target object based on the specific key point pair includes: In the second image, obtaining first horizontal coordinate information of the first specific key point and second horizontal coordinate information of the second specific key point; Calculating a human body angle score corresponding to the target object based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image, including: determining a key point angle normalization score based on the first horizontal coordinate information, the second horizontal coordinate information, and the horizontal width information of the second image; determining a front-side and back-side judgment score based on the first horizontal coordinate information and the second horizontal coordinate information; and calculating a human body angle score corresponding to the target object based on the key point angle normalization score and the front-side and back-side judgment score, where the human body angle score is obtained by multiplying the key point angle normalization score by the front-side and back-side judgment score. A third determination module is used to determine the direction of the target object in the first image based on the human body angle score, including: searching a target score range corresponding to the human body angle score in a direction query table; wherein the direction query table stores multiple different directions and a score range corresponding to each of the directions; and determining the target direction corresponding to the target score range as the direction of the target object in the first image.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is configured to execute a target object direction detection program stored in the memory to implement the target object direction detection method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the target object direction detection method according to any one of claims 1 to 5.

Images