Human head external frame area prediction method and device, electronic equipment, medium and vehicle

Abstract

The invention provides a head external frame area prediction method and device, electronic equipment, a storage medium and a vehicle. The method comprises the following steps: determining a key point linear distance between a chin point and a head top point of a person object to be predicted, and taking the key point linear distance as a target head circumscribed frame length; determining an intersection point straight line intersected with the initial human head circumscribed frame, and determining a tangent value of a head rotation angle of the intersection point straight line; determining an intersection point linear distance in the initial head circumscribed frame by using the tangent value, the vertex coordinate of the initial head circumscribed frame and the chin point coordinate of the chin point, and taking the intersection point linear distance as a target head circumscribed frame width; and multiplying the width of the target head circumscribed frame by the length of the target head circumscribed frame to obtain the area of the target head circumscribed frame, and obtaining the area of the target head circumscribed frame of the to-be-predicted figure object. The difference between the predicted human head external frame area and the actual human head area is small, so that the accuracy and the reliability of tasks such as follow-up anti-collision alarm are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, electronic device, storage medium, and vehicle for predicting the area of ​​a human head circumference frame. Background Art

[0002] Inside the cockpit, tasks like head collision avoidance are extremely sensitive to parameters related to head area, which is the primary information used for depth prediction. Therefore, accurately predicting the area of ​​the bounding box surrounding the head becomes a key issue.

[0003] Currently, most related technologies directly use the head bounding box area obtained from the human head detection and classification network as the final result for tasks such as head collision avoidance alarms. However, in related technologies, if the same person's head is at the same actual depth, the head bounding box area obtained from the human head detection and classification network can differ significantly from the actual head area due to different head rotation angles (such as roll angle). This difference can affect the accuracy of depth prediction, thereby affecting the accuracy and reliability of tasks such as head collision avoidance alarms. Summary of the Invention

[0004] The present disclosure provides a method, device, electronic device, storage medium, and vehicle for predicting the area of ​​a human head circumference frame.

[0005] According to a first aspect of the present disclosure, a method for predicting the area of ​​a head circumscribed frame is provided, the method comprising: determining a key point straight line distance of a key point straight line between a chin point and a head vertex of a person object to be predicted, and using the key point straight line distance as a target head circumscribed frame length of the person object to be predicted; determining an intersection straight line that intersects with an initial head circumscribed frame of the person object to be predicted, and determining a tangent value of a head rotation angle corresponding to the intersection straight line, the intersection straight line passing through the chin point, and the intersection straight line being perpendicular to the key point straight line; determining an intersection straight line distance of the intersection straight line within the initial head circumscribed frame using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and using the intersection straight line distance as a target head circumscribed frame width of the person object to be predicted; and multiplying the target head circumscribed frame width by the target head circumscribed frame length to obtain the target head circumscribed frame area of ​​the person object to be predicted.

[0006] In some embodiments of the present disclosure, determining the straight-line distance between the key points of the chin point and the head vertex of the person object to be predicted includes: obtaining the vertex coordinates of the initial human body circumference frame and the vertex coordinates of the initial head circumference frame of the person object to be predicted in the input image; using the vertex coordinates of the initial human body circumference frame to crop the input image according to the initial human body circumference frame to obtain the human body image of the person object to be predicted in the input image; performing key point detection on the human body image of the person object to be predicted to obtain the human body key points of the person object to be predicted; determining the chin point and the head vertex among the human body key points of the person object to be predicted, and determining the straight-line distance between the key points using the chin point coordinates of the chin point and the head vertex coordinates of the head vertex.

[0007] In some embodiments of the present disclosure, determining the key point straight-line distance between the chin point and the head vertex of the person object to be predicted, and using the key point straight-line distance as the target head circumscribed frame length of the person object to be predicted includes: using the coordinate distance between the chin point coordinates and the head vertex coordinates of the person object to be predicted, determining the key point straight-line distance between the chin point and the head vertex, and using the key point straight-line distance as the target head circumscribed frame length of the person object to be predicted.

[0008] In some embodiments of the present disclosure, determining an intersection straight line that intersects with an initial head circumscribed frame of a person object to be predicted, and determining a tangent value of a head rotation angle corresponding to the intersection straight line includes: drawing a straight line at a position passing through a chin point and perpendicular to a key point straight line to obtain an intersection straight line that intersects with the initial head circumscribed frame; taking the angle between the intersection straight line and a boundary line of the initial head circumscribed frame of the person object to be predicted as the head rotation angle of the person object to be predicted, and determining a tangent value of the head rotation angle.

[0009] In some embodiments of the present disclosure, the angle between the intersection straight line and the boundary line of the initial head circumscribed frame of the person object to be predicted is used as the head rotation angle of the person object to be predicted, and determining the tangent value of the head rotation angle includes: determining the horizontal coordinate difference and the vertical coordinate difference between the chin point coordinates and the head vertex coordinates, and determining the ratio of the horizontal coordinate difference to the vertical coordinate difference, and using the ratio as the tangent value of the head rotation angle.

[0010] In some embodiments of the present disclosure, the intersection straight line distance of the intersection straight lines is determined using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and the intersection straight line distance is used as the target head circumscribed frame width of the person object to be predicted, including: using the tangent value as the slope of the linear function, and inputting the chin point coordinates into the linear formula to determine the linear function corresponding to the intersection straight line; inputting the vertex coordinates of the initial head circumscribed frame into the linear function to determine the first intersection coordinates corresponding to the first intersection point of the intersection straight line in the initial head circumscribed frame and the second intersection coordinates corresponding to the second intersection point of the intersection straight line in the initial head circumscribed frame; using the coordinate distance between the first intersection coordinate and the second intersection coordinate to determine the intersection straight line distance between the first intersection point and the second intersection point, and using the intersection straight line distance as the target head circumscribed frame width of the person object to be predicted.

[0011] According to a second aspect of the present disclosure, a device for predicting the area of ​​a human head circumscribed frame is provided, comprising:

[0012] a first distance determination unit, configured to determine a key point straight line distance between a chin point and a head vertex of a person object to be predicted, and use the key point straight line distance as a length of a target head circumscribed frame of the person object to be predicted;

[0013] An angle determination unit, configured to determine an intersection line that intersects with the initial head circumscribed frame of the person to be predicted, and to determine a tangent value of the head rotation angle corresponding to the intersection line, wherein the intersection line passes through the chin point and is perpendicular to the key point line;

[0014] a second distance determining unit, configured to determine a straight line distance of intersection points within the initial head circumscribed frame using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and use the straight line distance of the intersection points as a width of the target head circumscribed frame of the person object to be predicted;

[0015] The area determination unit is used to multiply the width of the target head circumference frame and the length of the target head circumference frame to obtain the area of ​​the target head circumference frame of the person object to be predicted.

[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0021] According to a fifth aspect of the present disclosure, a vehicle is provided, comprising the device for predicting the area of ​​a human head circumference frame as described in the second aspect or the electronic device as described in the third solution.

[0022] The present disclosure provides a method, device, electronic device, storage medium and vehicle for predicting the area of ​​a head external frame. The method determines the key point straight line distance between the chin point and the head vertex of the person to be predicted, and uses the key point straight line distance as the target head external frame length of the person to be predicted; determines an intersection straight line that intersects with the initial head external frame of the person to be predicted, and determines the tangent value of the head rotation angle corresponding to the intersection straight line, the intersection straight line passes through the chin point, and the intersection straight line is perpendicular to the key point straight line; uses the tangent value, the vertex coordinates of the initial head external frame and the chin point coordinates of the chin point to determine the intersection straight line distance within the initial head external frame, and uses the intersection straight line distance as the target head external frame length of the person to be predicted; determines the intersection straight line that intersects with the initial head external frame of the person to be predicted, and determines the tangent value of the head rotation angle corresponding to the intersection straight line, the intersection straight line passes through the chin point, and the intersection straight line is perpendicular to the key point straight line; uses the tangent value, the vertex coordinates of the initial head external frame and the chin point coordinates of the chin point to determine the intersection straight line distance within the initial head external frame, and uses the intersection straight line distance as the target head external frame length. As the target head external frame width of the person object to be predicted; multiply the target head external frame width and the target head external frame length to obtain the target head external frame area of ​​the person object to be predicted, so that when the head of the person object to be predicted is at the same actual depth, the tangent value of the head rotation angle of the current person object to be predicted is determined, and the head external frame area of ​​the person object to be predicted is predicted based on the tangent value of the head rotation angle, avoiding the problem of inaccurate head external frame area prediction due to different head rotation angles when predicting the head external frame area, making the difference between the predicted head external frame area and the actual head area smaller, thereby improving the accuracy and reliability of subsequent tasks such as anti-collision head alarm.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0025] Figure 1 A schematic flow chart of a method for predicting the area of ​​a human head circumference frame provided in an embodiment of the present disclosure;

[0026] Figure 2 A schematic flow chart of another method for predicting the area of ​​a human head circumference frame provided by an embodiment of the present disclosure;

[0027] Figure 3A schematic diagram of obtaining the vertex coordinates of an initial human body circumference frame and an initial head circumference frame provided by an embodiment of the present disclosure;

[0028] Figure 4 A schematic diagram of obtaining coordinates of key points of a human body provided in an embodiment of the present disclosure;

[0029] Figure 5 A schematic diagram of the distribution of key points of a human body provided in an embodiment of the present disclosure;

[0030] Figure 6 A schematic diagram of an initial human head circumference frame intersecting with an intersection line provided by an embodiment of the present disclosure;

[0031] Figure 7 A schematic diagram of a head rotation angle provided in an embodiment of the present disclosure;

[0032] Figure 8 A schematic diagram of the structure of a device for predicting the area of ​​a human head circumference frame provided by an embodiment of the present disclosure;

[0033] Figure 9 A schematic block diagram of an exemplary electronic device 900 provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0035] In order to solve the problems in the related art, the present invention determines the tangent value of the head rotation angle of the current person to be predicted when the head of the person to be predicted is at the same actual depth, and predicts the head external frame area of ​​the person to be predicted based on the tangent value of the head rotation angle. The purpose is to regress the head external frame areas of all people with different head rotation angles to the head external frame area when the head rotation angle is 0, thereby avoiding the problem of inaccurate prediction of the head external frame area due to different head rotation angles when predicting the head external frame area, so that the difference between the predicted head external frame area and the actual head area is smaller, thereby improving the accuracy and reliability of subsequent tasks such as anti-collision head alarm.

[0036] The following describes a method, device, electronic device, storage medium, and vehicle for predicting the area of ​​a human head circumference frame according to embodiments of the present disclosure with reference to the accompanying drawings.

[0037] Figure 1 This is a flow chart of a method for predicting the area of ​​a human head's external frame provided in an embodiment of the present disclosure. The method can be executed by an electronic device, specifically, by an electronic device such as a PC or a server. The method can be applied to scenarios where a human head is recognized and located in an image or video, specifically, in a smart cockpit scenario or a security monitoring scenario of a car, and is not limited in the present embodiment. Figure 1 As shown, the method includes:

[0038] Step 101: Determine the key point straight line distance between the chin point and the head vertex of the person object to be predicted, and use the key point straight line distance as the target head circumscribed frame length of the person object to be predicted.

[0039] In some embodiments, the person object to be predicted refers to any individual or person that needs to be detected in the input image collected by the acquisition device. In the present disclosure, the person object to be predicted can be a person who can be identified in front of a surveillance camera, or a user who is using a device that requires head tracking or anti-collision functions (such as a virtual reality helmet).

[0040] The key point linear distance refers to the linear distance between two key points, and in this disclosure, refers to the linear distance between the chin point and the head vertex. Specifically, the key point linear distance can be calculated by combining the detected chin point coordinates and the head vertex coordinates with a distance formula.

[0041] A head bounding box is a geometric model used to represent the external contours of a head and to estimate and identify its size and position. The head bounding box can be a simplified head shape composed of a series of line segments, describing the basic shape and size of the head. In this disclosure, the head bounding box takes the form of an outer rectangular box surrounding the head. The target head bounding box is the outer rectangular box that most closely approximates the actual size of a head.

[0042] Step 102: Determine an intersection line that intersects with the initial head circumscribed frame of the person to be predicted, and determine the tangent value of the head rotation angle corresponding to the intersection line. The intersection line passes through the chin point and is perpendicular to the key point line.

[0043] In some embodiments, the initial head bounding box is the outer rectangular box of the head of the person to be predicted, obtained by a human head detection and classification network. The intersection line is a line that intersects the initial head bounding box. In the present disclosure, this line passes through the chin point and is perpendicular to the key point line between the chin point and the head vertex.

[0044] Since the intersection line intersects with the initial head circumscribed frame, an angle is formed between the intersection line and the boundary line of the initial head circumscribed frame. This angle can reflect the rotation angle of the head relative to the vertical direction, which is the head rotation angle in the present disclosure. The head rotation angle in the present disclosure refers to the Roll angle, also called the roll angle, which describes the rotation angle of an object (such as a head) around the z-axis. The tangent value of the head rotation angle is the ratio of the head rotation angle to the horizontal plane. Specifically, if the tangent value is tanθ, then θ is the size of the head rotation angle (ie, the roll angle). The tangent value is used to describe the change in the head rotation angle, which can be used to further estimate the depth information of the head and other related tasks.

[0045] Step 103: Using the tangent value, the vertex coordinates of the initial head bounding box, and the chin point coordinates, determine the intersection straight line distance within the initial head bounding box, and use the intersection straight line distance as the target head bounding box width of the person object to be predicted.

[0046] In some embodiments, the straight line distance between the first intersection point and the second intersection point of the intersection line with the initial head circumscribed frame can be calculated based on the determined tangent value of the head rotation angle, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point using the same distance formula as in step 101, and the determined straight line distance is used as the width of the target head circumscribed frame.

[0047] Step 104: Multiply the width of the target head bounding box by the length of the target head bounding box to obtain the area of ​​the target head bounding box of the person object to be predicted.

[0048] In some embodiments, the product of the width of the target head bounding box and the length of the target head bounding box can be used as the target head bounding box area of ​​the person object to be predicted to avoid the problem of inaccurate head bounding box area due to different head rotation angles.

[0049] In summary, the technical solution provided by the present invention determines the length and width of the target head external frame, and considers the influence of the head rotation angle on the prediction of the head external frame area when determining the width of the target head external frame. It can achieve the goal of determining the tangent value of the head rotation angle of the current person to be predicted when the head of the person to be predicted is at the same actual depth, and predicting the head external frame area of ​​the person to be predicted based on the tangent value of the head rotation angle, thereby avoiding the problem of inaccurate prediction of the head external frame area due to different head rotation angles when predicting the head external frame area, so that the difference between the predicted head external frame area and the actual head area is smaller, thereby improving the accuracy and reliability of subsequent tasks such as anti-collision head alarm.

[0050] Figure 2 This is a flow chart of another method for predicting the area of ​​a human head circumference frame provided in an embodiment of the present disclosure. The method can be executed by an electronic device, specifically, by an electronic device such as a PC or a server. Figure 2 based on Figure 1 In the embodiment shown, step 101, step 102 and step 103 are further defined. Figure 2 In the embodiment shown, step 101 includes step 201, step 202, step 203 and step 204, and step 102 includes step 205 and step 206. Figure 2 As shown, the method may include:

[0051] Step 201: Obtain the vertex coordinates of the initial human body circumference frame and the vertex coordinates of the initial head circumference frame of the person object to be predicted in the input image.

[0052] In some embodiments, the input image can be processed by a human head detection and classification network to obtain the vertex coordinates of the initial human body bounding box and the vertex coordinates of the initial head bounding box of the human object to be predicted in the input image. The input image is obtained by an image acquisition device, such as a camera.

[0053] Among them, the vertex coordinates of the initial human body external bounding box and the vertex coordinates of the initial human body external bounding box can be the vertex coordinates of the upper left corner and the vertex coordinates of the lower right corner of the external bounding box of the human body and head, and a rectangular frame can be formed by the vertex coordinates of the upper left corner and the vertex coordinates of the lower right corner.

[0054] The human head detection and classification network is a deep learning network built for detection and classification scenarios. It is usually composed of a backbone network, a shoulder network (neck), and a head network (head). It is used to detect whether there is a human body or a human head and return the vertex coordinates of the human body's bounding box and the vertex coordinates of the head's bounding box. In the present disclosure, before processing the input image through the human head detection and classification network, it is necessary to train a human head detection and classification network through a large amount of training data. These training data can be various image or video data sets, which contain different human and head postures, different backgrounds and lighting conditions, etc. Figure 3 The diagram shown is a schematic diagram of obtaining the vertex coordinates of the initial human body external bounding box and the vertex coordinates of the initial head external bounding box. Before inputting the input image into the human head detection and classification network, the input image (original image) needs to be preprocessed. The preprocessing may include operations such as scaling, cropping, and normalization to facilitate the human head detection and classification network to better extract features. Then, the human head detection and classification network is used to extract features from the preprocessed input image through the backbone network (backbone), shoulder network (neck), and head network (head) in sequence, and the output of the head network is parsed and processed through post-processing, such as non-maximum suppression (NMS) operations, to eliminate overlapping detection frames and return the vertex coordinates of the initial human body external bounding box (human body frame coordinates) and the vertex coordinates of the initial head external bounding box (head frame coordinates).

[0055] Step 202: Using the vertex coordinates of the initial human body bounding box, the input image is cropped according to the initial human body bounding box to obtain a human body image of the person object to be predicted in the input image.

[0056] In some embodiments, since a rectangular frame can be formed by the vertex coordinates of the initial human body circumference frame, the present disclosure can use a preset cropping function to input the input image to be cropped and the obtained vertex coordinates of the initial human body circumference frame into this cropping function, thereby directly obtaining the cropped human body image.

[0057] Specifically, the present disclosure can determine the image area to be cropped using the vertex coordinates of the initial human body circumscribed frame through a cropping function set in OpenCV, and read the input image to be cropped to perform image cropping.

[0058] It should be noted that the cropping function preset in the present disclosure can be set according to the actual situation of the input image and the actual work experience of the staff, and is not limited in the embodiments of the present disclosure.

[0059] Step 203: Perform key point detection on the human body image to obtain the human body key points of the human object to be predicted.

[0060] In some embodiments, the cropped human body image may be processed by a human body key point detection network to obtain a plurality of human body key points in the human body image and the coordinates of the plurality of human body key points.

[0061] The human key point detection network is a deep learning network built for detection scenarios. It can be composed of a backbone network, a shoulder network (neck), and a head network (head). It is used to detect whether there are human key points and return the coordinates of the human key points. In the present disclosure, before processing human images through the human key point detection network, it is necessary to train a human key point detection network through a large amount of training data. These training data can be various human images, including annotated and unannotated human images of different scales, lighting conditions, and backgrounds. Figure 4 As shown, a schematic diagram of obtaining the coordinates of human key points is shown. Before the human key point detection network detects the human image (original image), the input image needs to be cropped according to the size of the human body external frame obtained above to obtain the human body image, and the human body image is preprocessed. The preprocessing may include operations such as scaling, cropping, and normalization to facilitate the human key point detection network to better extract features. The preprocessed human body image is then sequentially subjected to feature extraction through the backbone network (backbone), shoulder network (neck), and head network (head). The output of the head network is then parsed and processed through post-processing, such as non-maximum suppression (NMS) and other operations to eliminate overlapping key points, thereby determining the human body key points and returning the human body key point coordinates corresponding to the human body key points.

[0062] Step 204: Determine the chin point and the head vertex among the key points of the human body of the person object to be predicted, and use the coordinate distance between the chin point coordinates and the head vertex coordinates to determine the key point straight-line distance between the chin point and the head vertex, and use the key point straight-line distance as the target head external frame length of the person object to be predicted.

[0063] In the present disclosure, determining the coordinate distance between the chin point coordinates and the head vertex coordinates can be achieved using a distance formula, which is used to determine the straight-line distance between the chin point and the head vertex using the chin point coordinates and the head vertex coordinates.

[0064] In some embodiments, as Figure 5 The figure shows a distribution diagram of key points of the human body, wherein point 1 is the head vertex of the key points of the human body in the present disclosure, and point 4 is the chin point of the key points of the human body in the present disclosure.

[0065] The distance formula is used to calculate the distance between two points on a two-dimensional plane. The formula calculates the difference between the horizontal coordinates (i.e., x-coordinates) and the vertical coordinates (i.e., y-coordinates) of the two points. The squares of these two differences are then added together, and the square root of the sum is taken. This result is the straight-line distance between the two key points, which is used as the length h of the bounding box of the target head.

[0066] The specific distance formula is shown in Formula 1:

[0067] \begin{equation}h=\sqrt{{(x1-x2)^2}+{(y1-y2)^2}}\end{equation}Formula 1

[0068] Among them, (x1, y1) is the coordinate of the chin point, and (x2, y2) is the coordinate of the head vertex.

[0069] Step 205: Draw a straight line at a position passing through the chin point and perpendicular to the key point straight line to obtain an intersection straight line that intersects with the initial head circumscribed frame.

[0070] In some embodiments, the present disclosure can draw a straight line at a position passing through the chin point and perpendicular to the key point straight line, obtaining a straight line passing through the chin point and perpendicular to the line connecting the head vertex and the chin point. This straight line intersects with the initial head circumscribed frame, so it is the intersection straight line in the present disclosure. Figure 6 A schematic diagram of an initial human head external frame intersecting with the intersection line is shown, referring to Figure 6 , Figure 6 Taking the left turn of the head as an example, the intersection line intersects the initial head bounding box at points (x4, y4) and (x5, y5). The intersection distance between points (x4, y4) and (x5, y5) is the width of the target head bounding box. The inner rectangular box is the target head bounding box, and the outer rectangular box is the initial head bounding box.

[0071] Step 206: taking the angle between the intersection line and the boundary line of the initial head circumscribed frame of the person object to be predicted as the head rotation angle of the person object to be predicted, and determining the tangent value of the head rotation angle.

[0072] In some embodiments, the horizontal coordinate difference and the vertical coordinate difference between the chin point coordinate and the head vertex coordinate are determined, and the ratio of the horizontal coordinate difference to the vertical coordinate difference is determined, and the ratio is used as the tangent value of the head rotation angle.

[0073] In an optional embodiment of the present disclosure, referring to Figure 6 The angle a between the intersection line and the boundary line of the initial head circumference frame is the head rotation angle in this disclosure. Figure 7A schematic diagram of a head rotation angle is shown, where the pitch angle is the rotation angle about the x-axis, corresponding to Figure 7 Pitch in; yaw angle SHI is the rotation angle about the y-axis, corresponding to Figure 7 Yaw in; the roll angle is the rotation angle about the z axis, corresponding to Figure 7 The standard head posture is when the head is facing the camera, that is, the pitch angle, yaw angle, and roll angle are all 0. In this disclosure, the head rotation angle takes the roll angle as an example.

[0074] The tangent value k of the head rotation angle a is calculated using the chin point coordinates and the head vertex coordinates, as shown in Formula 2:

[0075] k = (x1 - x2) / (y2 - y1) Formula 2

[0076] Among them, (x1, y1) is the coordinate of the chin point, and (x2, y2) is the coordinate of the head vertex.

[0077] Step 207: Determine the intersection straight line distance of the intersection straight lines using the tangent value, the vertex coordinates of the initial head bounding box, and the chin point coordinates, and use the intersection straight line distance as the target head bounding box width of the person object to be predicted.

[0078] In some embodiments, the tangent value is used as the slope of the linear function, and the chin point coordinates are input into the linear formula to determine the linear function corresponding to the intersection line; the vertex coordinates of the initial head circumscribed frame are input into the linear function to determine the first intersection coordinates corresponding to the first intersection point of the intersection line in the initial head circumscribed frame and the second intersection coordinates corresponding to the second intersection point of the intersection line in the initial head circumscribed frame; the coordinate distance between the first intersection coordinate and the second intersection coordinate is used to determine the intersection line distance of the intersection line between the first intersection point and the second intersection point, and the intersection line distance is used as the target head circumscribed frame width of the person object to be predicted.

[0079] In an optional embodiment of the present disclosure, the straight line equation between the points (x4, y4) and (x5, y5), i.e., the linear function, can be obtained using the above-determined tangent value and the chin point coordinates, as shown in Formula 3:

[0080] Y = k * X + y1 - k * x1 Formula 3

[0081] Where k is the tangent value, x1 is the horizontal coordinate of the chin point, and y1 is the vertical coordinate of the chin point.

[0082] Substitute the tangent value, the vertex coordinates of the initial head bounding box, and the chin point coordinates into the straight line equation to calculate the coordinates of the points (x4, y4) and (x5, y5), that is, the first intersection coordinates and the second intersection coordinates, as shown in Formula 4:

[0083] x4=(y4-y1+k*x1) / k

[0084] y4=t3

[0085] x5=x3

[0086] y5 = k * x3 + y1 - k * x1 Formula 4

[0087] Wherein, k is the tangent value, x1 is the horizontal coordinate of the chin point coordinate, y1 is the vertical coordinate of the chin point coordinate, x3 is the horizontal coordinate of the vertex coordinate of the initial head external frame, and y3 is the vertical coordinate of the vertex coordinate of the initial head external frame.

[0088] The coordinate distance between the first intersection coordinate and the second intersection coordinate can be calculated using the same distance formula as in step 204 to calculate the width w of the target head bounding box, as shown in Formula 5:

[0089] \begin{equation}w=\sqrt{{(x4-x5)^2}+{(y4-y5)^2}}\end{equation}Formula 5

[0090] Among them, (x4, y4) is the coordinate of the first intersection point, and (x5, y5) is the coordinate of the second intersection point.

[0091] It should be noted that the first intersection coordinates and the second intersection coordinates in this disclosure are merely examples, and the order of the first intersection coordinates and the second intersection coordinates is not limited in this disclosure. Furthermore, the distance formula in this disclosure is merely an example; any formula that can calculate the distance between two points is applicable. The distance formulas used to calculate the straight-line distance between intersections and key points can be the same or different in this disclosure.

[0092] Step 208: Multiply the width of the target head bounding box by the length of the target head bounding box to obtain the area of ​​the target head bounding box of the person object to be predicted.

[0093] In some embodiments, the target head bounding box length h obtained in step 204 and the target head bounding box width w obtained in step 207 are multiplied to obtain the target head bounding box area S that is closest to the actual head area, as shown in Formula 6:

[0094] S = w * h Formula 6

[0095] In summary, the technical solution provided by the present invention can correct the initial head bounding box inside the cockpit according to the key points of the head, so as to determine the area of ​​the target head bounding box. Specifically, the geometric relationship between the head vertex, chin point, initial head bounding box and predicted head bounding box can be used to cleverly obtain the length and width of the target head bounding box that is close to the actual head area, and successfully regress the areas of the initial head bounding box with different head rotation angles (i.e., Roll angles) to the area of ​​the target head bounding box when the Rroll angle is 0. Under this same scale standard, using the area of ​​the target head bounding box to perform depth estimation is more robust and accurate, which facilitates more accurate depth estimation for subsequent tasks such as head collision avoidance, thereby improving the precision and recall of these tasks.

[0096] Corresponding to the aforementioned head circumference frame area prediction method, the present invention also provides a head circumference frame area prediction device. Since the device embodiment of the present invention corresponds to the aforementioned method embodiment, any details not disclosed in the device embodiment can be referred to the aforementioned method embodiment and will not be further described in this invention.

[0097] Figure 8 This is a schematic diagram of a device for predicting the area of ​​a head circumference frame provided by an embodiment of the present disclosure. Figure 8 As shown, the device includes:

[0098] The first distance determining unit 810 is configured to determine a key point straight line distance between the chin point and the head vertex of the person to be predicted, and use the key point straight line distance as the length of the target head bounding box of the person to be predicted;

[0099] Angle determination unit 820, configured to determine an intersection line that intersects the initial head bounding box of the person to be predicted, and to determine the tangent value of the head rotation angle corresponding to the intersection line, where the intersection line passes through the chin point and is perpendicular to the key point line;

[0100] A second distance determining unit 830 is configured to determine a straight line distance between the intersection points within the initial head circumscribed frame using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and use the straight line distance as the width of the target head circumscribed frame of the person to be predicted;

[0101] The area determination unit 840 is configured to multiply the width of the target head bounding box by the length of the target head bounding box to obtain the area of ​​the target head bounding box of the person object to be predicted.

[0102] In some embodiments of the present disclosure, the first distance determination unit 810 is used to: obtain the vertex coordinates of the initial human body circumference frame and the vertex coordinates of the initial head circumference frame of the person object to be predicted in the input image; use the vertex coordinates of the initial human body circumference frame to crop the input image according to the initial human body circumference frame to obtain the human body image of the person object to be predicted in the input image; perform key point detection on the human body image of the person object to be predicted to obtain the human body key points of the person object to be predicted; determine the chin point and the head vertex among the human body key points of the person object to be predicted, and use the chin point coordinates of the chin point and the head vertex coordinates of the head vertex to determine the straight-line distance of the key points.

[0103] In some embodiments of the present disclosure, the first distance determination unit 810 is used to: use the coordinate distance between the chin point coordinates and the head vertex coordinates of the person object to be predicted to determine the key point straight-line distance between the chin point and the head vertex, and use the key point straight-line distance as the target head circumscribed frame length of the person object to be predicted.

[0104] In some embodiments of the present disclosure, the angle determination unit 820 is used to: draw a straight line at a position passing through the chin point and perpendicular to the key point straight line to obtain an intersection straight line that intersects with the initial head circumscribed frame; use the angle between the intersection straight line and the boundary line of the initial head circumscribed frame of the person object to be predicted as the head rotation angle of the person object to be predicted, and determine the tangent value of the head rotation angle.

[0105] In some embodiments of the present disclosure, the angle determination unit 820 is further configured to determine the horizontal coordinate difference and the vertical coordinate difference between the chin point coordinates and the head vertex coordinates, and determine the ratio of the horizontal coordinate difference to the vertical coordinate difference, and use the ratio as the tangent value of the head rotation angle.

[0106] In some embodiments of the present disclosure, the second distance determination unit 830 is used to: use the tangent value as the slope of the linear function, and input the chin point coordinates into the linear formula to determine the linear function corresponding to the intersection line; input the vertex coordinates of the initial head circumscribed frame into the linear function to determine the first intersection coordinates corresponding to the first intersection point of the intersection line in the initial head circumscribed frame and the second intersection coordinates corresponding to the second intersection point of the intersection line in the initial head circumscribed frame; use the coordinate distance between the first intersection coordinate and the second intersection coordinate to determine the intersection line distance between the first intersection point and the second intersection point, and use the intersection line distance as the target head circumscribed frame width of the person object to be predicted.

[0107] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principles are the same, which is not limited in this embodiment.

[0108] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a vehicle.

[0109] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0110] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 902 or a computer program loaded from a storage unit 908 into a RAM (Random Access Memory) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An I / O (Input / Output) interface 905 is also connected to the bus 904.

[0111] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0112] The computing unit 901 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), various specialized AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the head circumference box area prediction method. For example, in some embodiments, the head circumference box area prediction method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the aforementioned head circumscribed frame area prediction method in any other appropriate manner (for example, by means of firmware).

[0113] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0117] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0118] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0119] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0120] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0121] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for predicting the area of ​​a head circumference frame, characterized in that: The method comprises: Determine a key point straight line distance between the chin point and the head vertex of the person object to be predicted, and use the key point straight line distance as the target head circumscribed frame length of the person object to be predicted; Determine an intersection line that intersects with the initial head circumscribed frame of the person object to be predicted, and determine a tangent value of the head rotation angle corresponding to the intersection line, wherein the intersection line passes through the chin point and is perpendicular to the key point line; Determine a straight-line distance of the intersection points within the initial head circumscribed frame using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and use the straight-line distance as a target head circumscribed frame width for the person to be predicted; The target head circumference frame width and the target head circumference frame length are multiplied to obtain the target head circumference frame area of ​​the person object to be predicted.

2. The method according to claim 1, characterized in that Determining the key point straight-line distance between the chin point and the head vertex of the person object to be predicted includes: Obtain the vertex coordinates of the initial human body bounding box and the vertex coordinates of the initial head bounding box of the person object to be predicted in the input image; Using the vertex coordinates of the initial human body bounding box, the input image is cropped according to the initial human body bounding box to obtain a human body image of the person object to be predicted in the input image; Performing key point detection on the human body image of the human object to be predicted to obtain the human body key points of the human object to be predicted; Determine the chin point and the head vertex among the key points of the human body of the to-be-predicted person object, and determine the straight-line distance between the key points using the chin point coordinates of the chin point and the head vertex coordinates of the head vertex.

3. The method according to claim 2, characterized in that The step of determining the key point straight-line distance between the chin point and the head vertex of the person object to be predicted, and using the key point straight-line distance as the target head circumscribed frame length of the person object to be predicted comprises: The coordinate distance between the chin point coordinates and the head vertex coordinates of the person object to be predicted is used to determine the key point straight-line distance between the chin point and the head vertex, and the key point straight-line distance is used as the target head circumscribed frame length of the person object to be predicted.

4. The method according to claim 2, characterized in that The step of determining an intersection line intersecting with the initial head circumscribed frame of the person object to be predicted, and determining a tangent value of a head rotation angle corresponding to the intersection line includes: Draw a straight line at a position passing through the chin point and perpendicular to the key point straight line to obtain an intersection straight line intersecting with the initial head circumscribed frame; The angle between the intersection line and the boundary line of the initial head circumscribed frame of the person object to be predicted is used as the head rotation angle of the person object to be predicted, and the tangent value of the head rotation angle is determined.

5. The method according to claim 4, characterized in that The step of using the angle between the intersection line and the boundary line of the initial head circumscribed frame of the person object to be predicted as the head rotation angle of the person object to be predicted, and determining the tangent value of the head rotation angle includes: Determine the horizontal coordinate difference and the vertical coordinate difference between the chin point coordinate and the head vertex coordinate, and determine the ratio of the horizontal coordinate difference to the vertical coordinate difference, and use the ratio as the tangent value of the head rotation angle.

6. The method according to claim 2, characterized in that The step of determining the intersection straight line distance of the intersection straight lines by using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and using the intersection straight line distance as the target head circumscribed frame width of the person object to be predicted includes: Using the tangent value as the slope of a linear function, and inputting the chin point coordinates into a linear formula, to determine the linear function corresponding to the intersection line; Inputting the vertex coordinates of the initial head circumscribed frame into the linear function, determining the first intersection coordinate corresponding to the first intersection point of the intersection line within the initial head circumscribed frame and the second intersection coordinate corresponding to the second intersection point of the intersection line within the initial head circumscribed frame; The coordinate distance between the first intersection coordinate and the second intersection coordinate is used to determine the intersection straight line distance between the first intersection and the second intersection, and the intersection straight line distance is used as the target head circumscribed frame width of the person object to be predicted.

7. A device for predicting the area of ​​a head circumference frame, characterized in that: include: a first distance determining unit, configured to determine a key point straight line distance between a chin point and a head vertex of a person object to be predicted, and use the key point straight line distance as a length of a target head circumscribed frame of the person object to be predicted; an angle determination unit, configured to determine an intersection line intersecting with an initial head circumscribed frame of the person to be predicted, and to determine a tangent value of a head rotation angle corresponding to the intersection line, wherein the intersection line passes through the chin point and is perpendicular to the key point line; a second distance determining unit, configured to determine a straight line distance of intersection points of the intersection points within the initial head circumscribed frame using the tangent value, the vertex coordinates of the initial head circumscribed frame, and the chin point coordinates of the chin point, and use the straight line distance of the intersection points as a target head circumscribed frame width of the person object to be predicted; The area determination unit is used to multiply the width of the target head circumference frame by the length of the target head circumference frame to obtain the area of ​​the target head circumference frame of the person object to be predicted.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

10. A vehicle, characterized in that: It includes the device for predicting the area of ​​the external frame of a human head as described in claim 7 or the electronic device as described in claim 8.

Images