A method, device, electronic device and storage medium for calculating a person's height

Abstract

The present invention relates to the technical field of passenger flow cameras, and in particular to a method, device, electronic device and storage medium for calculating the height of a person. The method includes: obtaining a person image collected by an image acquisition device; obtaining the two-dimensional head center point coordinates and two-dimensional foot center point coordinates of the person from the person image; converting the two-dimensional head center point coordinates and two-dimensional foot center point coordinates into three-dimensional head center point coordinates and three-dimensional foot center point coordinates through the conversion from a two-dimensional pixel coordinate system to a three-dimensional world coordinate system; obtaining the auxiliary point coordinates of a reference plane, constructing an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculating the estimated distance from the three-dimensional foot center point coordinates to the auxiliary line; calculating the height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the human body frame coordinates, the head frame coordinates and the two-dimensional foot center point coordinates of the person. The estimation method in this embodiment has a small visual difference and a higher accuracy of the calculated person height.

Description

Technical Field

[0001] The present invention relates to the technical field of passenger flow cameras, and in particular to a method, device, electronic device and storage medium for calculating the height of a person. Background Art

[0002] A passenger flow camera refers to a type of camera that can automatically and real-time analyze the captured video stream to output customer information entering and leaving a specified area, such as analyzing and outputting the number of people entering and leaving a certain store, the height of people, whether they wear masks, whether they wear hats and other attributes. Calculating the height of people in the passenger flow is an important human attribute and index of the passenger flow camera.

[0003] The existing technology generally obtains the distance between the foot point and the head point by finding points in a 2D image, and then adds the error between the top of the head and the head point. However, the 2D image is projected from the 3D world coordinate system through the camera into the 2D coordinate system, resulting in visual errors and low accuracy of the calculation results. It can be seen that there is a problem of low accuracy when the existing passenger flow camera calculates the height of a person. Summary of the Invention

[0004] An embodiment of the present invention provides a method for calculating the height of a person, aiming to solve the problem of low accuracy when the existing passenger flow camera calculates the height of a person.

[0005] In a first aspect, an embodiment of the present invention provides a method for calculating the height of a person, the method comprising the following steps:

[0006] Obtain a person image collected by an image acquisition device;

[0007] Obtain the two-dimensional key point coordinates of the person included in the person image from the person image, the two-dimensional key point coordinates including the two-dimensional center point coordinates of the head and the two-dimensional center point coordinates of the foot;

[0008] Convert the two-dimensional center point coordinates of the head and the two-dimensional center point coordinates of the foot through the conversion from the two-dimensional pixel coordinate system to the three-dimensional world coordinate system to obtain the three-dimensional center point coordinates of the head and the three-dimensional center point coordinates of the foot;

[0009] Obtain the auxiliary point coordinates of a reference plane, form an auxiliary line based on the auxiliary point coordinates and the three-dimensional center point coordinates of the head, and calculate the estimated distance from the three-dimensional center point coordinates of the foot to the auxiliary line;

[0010] Calculate the height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the human body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image, and the two-dimensional center point coordinates of the foot.

[0011] In a second aspect, an embodiment of the present invention provides a device for calculating a person's height, including:

[0012] A first acquisition module, configured to acquire a person image collected by an image acquisition device;

[0013] A second acquisition module, configured to acquire two-dimensional key point coordinates of the person included in the person image, where the two-dimensional key point coordinates include two-dimensional head center point coordinates and two-dimensional foot center point coordinates;

[0014] A conversion module, configured to convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates through a two-dimensional pixel coordinate system to a three-dimensional world coordinate system, so as to obtain three-dimensional head center point coordinates and three-dimensional foot center point coordinates;

[0015] A first calculation module, configured to acquire auxiliary point coordinates of a reference plane, form an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculate a predicted distance from the three-dimensional foot center point coordinates to the auxiliary line;

[0016] A second calculation module, configured to calculate the height of the person included in the person image based on the predicted distance, the basic parameters of the image acquisition device, the human body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image, and the two-dimensional foot center point coordinates.

[0017] In a third aspect, an electronic device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the steps in a method for calculating a person's height provided by an embodiment of the present invention are implemented.

[0018] In a fourth aspect, a computer-readable storage medium stores a computer program, where when the computer program is executed by a processor, the steps in a method for calculating a person's height provided by an embodiment of the present invention are implemented.

[0019] In an embodiment of the present invention, by acquiring a personnel image collected by an image acquisition device; obtaining two-dimensional key point coordinates of the personnel included in the personnel image, the two-dimensional key point coordinates including two-dimensional head center point coordinates and two-dimensional foot center point coordinates; converting the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates through a two-dimensional pixel coordinate system to a three-dimensional world coordinate system to obtain three-dimensional head center point coordinates and three-dimensional foot center point coordinates; obtaining auxiliary point coordinates of a reference plane, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculating an estimated distance from the three-dimensional foot center point coordinates to the auxiliary line; calculating the personnel height of the personnel included in the personnel image based on the estimated distance, the basic parameters of the image acquisition device, the body frame coordinates of the personnel included in the personnel image, the head frame coordinates of the personnel included in the personnel image, and the two-dimensional foot center point coordinates. After acquiring the personnel image collected by the image acquisition device in the present application, the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates of the personnel in the image are obtained, and then the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates are obtained through the conversion of the two-dimensional pixel coordinate system to the three-dimensional world coordinate system, and the estimated distance is calculated by selecting the auxiliary point coordinates and then the personnel height is calculated. Through such an estimation method, the visual difference is small and the accuracy of the calculated personnel height is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 is a flowchart of a method for calculating personnel height provided by an embodiment of the present invention;

[0022] Figure 2 is a flowchart of another method for calculating personnel height provided by an embodiment of the present invention;

[0023] Figure 3 is provided by an embodiment of the present invention Figure 2 flowchart of step S203 in;

[0024] Figure 4 is a schematic structural diagram of a device for calculating personnel height provided by an embodiment of the present invention;

[0025] Figure 5 is a schematic structural diagram of another device for calculating personnel height provided by an embodiment of the present invention;

[0026] Figure 6 It is a schematic structural diagram of another calculation device for the height of a person provided by an embodiment of the present invention;

[0027] Figure 7 It is a schematic structural diagram of another calculation device for the height of a person provided by an embodiment of the present invention;

[0028] Figure 8 It is a schematic structural diagram of another calculation device for the height of a person provided by an embodiment of the present invention;

[0029] Figure 9 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

[0031] The terms "comprising" and "having" and any variations thereof in the specification, claims and drawings of this application are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification, claims or drawings of this application are used to distinguish different objects, rather than to describe a specific order. The mention of "embodiment" in this article means that a specific feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0032] Please refer to Figure 1 , Figure 1 It is a flowchart of a method for calculating the height of a person provided by an embodiment of the present invention. As Figure 1 shown, it includes the following steps:

[0033] S101. Obtain a person image collected by an image acquisition device.

[0034] In an embodiment of the present invention, the scenarios in which the method for calculating the height of a person is implemented include, but are not limited to, various shopping malls, stores, scenic spots, and stations, and are gradually extended to airports, airport transit halls, parking lots, fitness venues, and so on. The electronic device on which the above method for calculating the height of a person runs can obtain person images and the like through a wired connection method or a wireless connection method. Among them, the wireless connection method may include, but is not limited to, 3G / 4G connection, WiFi (Wireless-Fidelity) connection, Bluetooth connection, WiMAX (Worldwide Interoperability for Microwave Access) connection, Zigbee (low-power local area network protocol, also known as the Purple Peak protocol) connection, UWB (ultra wideband) connection, and other currently known or future-developed wireless connection methods.

[0035] Among them, the above image acquisition device may refer to a passenger flow camera. After calibrating the basic parameters, the passenger flow camera can be installed at the corresponding positions in the above-mentioned scenarios such as shopping malls, stores, scenic spots, and stations. Multiple passenger flow cameras can be installed in the same scenario. During the installation process, it is required that the field of view is horizontal, that is, the x-axis in the pixel coordinate system of the passenger flow camera is horizontal with the ground. Specifically, the calibration of the basic parameters of the passenger flow camera includes calibrating the distortion parameters of the camera. Of course, if the passenger flow camera has no distortion, there is no need for calibration. The basic parameters of the above passenger flow camera also include the internal parameter matrix and the external parameter matrix of the passenger flow camera. The internal parameter matrix and the external parameter matrix can be used for converting the two-dimensional pixel coordinate system to the three-dimensional world coordinate system.

[0036] S102. Obtain the two-dimensional key point coordinates of the person included in the person image from the person image. The two-dimensional key point coordinates include the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates.

[0037] Among them, image acquisition can be performed through a passenger flow camera. During the image acquisition process, it is possible to automatically identify whether there is a human image in the recognition range in the image. Specifically, recognition can be performed based on the characteristics of the human body. For example, it can be recognized whether there are features such as human faces and human forms in the image. If a human image is captured, a human body frame will be output and marked. The human body frame refers to a target frame that contains the entire human body of a person. It is equivalent to inputting a picture and, after recognizing a human body, the output is a human body frame. Moreover, a head frame can be further framed from the human body frame. The head frame refers to a target frame that contains the entire head of a person. Then, through key point detection, the head and two feet of the person are found, and based on the human body frame and the head frame, the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates are calculated respectively. Among them, the human body frame can be expressed as person(x, y, w, h); the head frame coordinates can be expressed as headRect(x, y, w, h); the two-dimensional head center point coordinates can be expressed as headCenter(x, y), and the two-dimensional foot center point coordinates as footCenter(x, y). The content within the above parentheses represents coordinates, where x and y correspond to the x-coordinate and y-coordinate respectively, and w and h correspond to the width of the x-coordinate and the height of the y-coordinate respectively, with the unit being pixels.

[0038] S103. Convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates through the conversion from the two-dimensional pixel coordinate system to the three-dimensional world coordinate system to obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates.

[0039] Among them, based on the method of converting the two-dimensional pixel coordinate system (2D (dimensions) pixel coordinate system) to the three-dimensional world coordinate system (3D world coordinate system), it is possible to convert the two-dimensional head center point coordinates headCenter(x, y) in the 2D pixel coordinate system to the three-dimensional head center point coordinates 3Dhead(x, y, z) in the 3D world coordinate system, and to convert the two-dimensional foot center point coordinates footCenter(x, y) to the three-dimensional foot center point coordinates 3Dfoot(x, y, z).

[0040] S104. Obtain the auxiliary point coordinates of the reference plane, form an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculate the estimated distance from the three-dimensional foot center point coordinates to the auxiliary line.

[0041] A reference plane can be taken within the shooting range of the passenger flow camera. In this embodiment, the above-mentioned reference plane may refer to the ceiling plane above the heads of people, and an auxiliary point 3DheadTmp(x+1, y, z) is taken on the ceiling plane. Connect the auxiliary point 3DheadTmp(x+1, y, z) and the three-dimensional head center point coordinates 3Dhead(x, y, z) to form the above-mentioned auxiliary line L. By calculating the distance from a point to a line, calculate the distance from the three-dimensional foot center point coordinates 3Dfoot(x, y, z) to the auxiliary line L. This distance is the estimated distance headFootHeight from the three-dimensional foot center point coordinates 3Dfoot(x, y, z) to the three-dimensional head center point coordinates 3Dhead(x, y, z).

[0042] S105. Calculate the height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the human body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image, and the two-dimensional foot center point coordinates.

[0043] Among them, the specific calculation formula for the height of the person is as follows:[[]]END]]

[0044] height = headFootHeight + headRect.h * fy + (person.y + person.h - footCenter.y) * fy.

[0045] Among them: headFootHeight represents the estimated distance; headRect.h represents the height in the head frame coordinates; fy belongs to the basic parameters of the camera and refers to the actual distance of one pixel in the y direction, with the unit of mm; person.y represents the y coordinate value of the human body frame coordinates corresponding to the person image; person.h represents the height in the human body frame coordinates; footCenter.y represents the y coordinate value in the two-dimensional foot center point coordinates.

[0046] Specifically, according to the above calculation formula, the height of the person in the person image collected by the passenger flow camera can be calculated. This height of the person is not necessarily the same as the actual height of the person, but can be used as the optimal height estimate recognized by the passenger flow camera.

[0047] In an embodiment of the present invention, by obtaining a personnel image collected by an image acquisition device; obtaining two-dimensional key point coordinates of the personnel included in the personnel image from the personnel image, the two-dimensional key point coordinates including two-dimensional head center point coordinates and two-dimensional foot center point coordinates; converting the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates through a conversion from a two-dimensional pixel coordinate system to a three-dimensional world coordinate system to obtain three-dimensional head center point coordinates and three-dimensional foot center point coordinates; obtaining auxiliary point coordinates of a reference plane, forming an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculating an estimated distance from the three-dimensional foot center point coordinates to the auxiliary line; calculating the height of the personnel included in the personnel image based on the estimated distance, the basic parameters of the image acquisition device, the body frame coordinates of the personnel included in the personnel image, the head frame coordinates of the personnel included in the personnel image, and the two-dimensional foot center point coordinates. It can be seen that in this application, the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates are obtained through the conversion method of converting from a 2D pixel coordinate system to a 3D world coordinate system. Converting to three-dimensional coordinates can reduce visual differences, and by selecting the auxiliary point coordinates to calculate the estimated distance and then calculating the personnel height, the accuracy of the personnel height calculated by such an estimation method is higher.

[0048] Please refer to Figure 2 , Figure 2 which is a flowchart of another method for calculating the height of a person provided by an embodiment of the present invention. As Figure 2 shown, it includes the steps:

[0049] S201. Obtain a personnel image collected by an image acquisition device.

[0050] S202. Detect the body frame and head frame of the personnel included in the personnel image based on a human detection model. The body frame has corresponding body frame coordinates, and the head frame has corresponding head frame coordinates.

[0051] Among them, after installing and starting the passenger flow camera for image acquisition, image tracking can be combined with a target tracking algorithm. Human detection is performed through the human detection model PersonDetect of the passenger flow camera. Input a picture, and the output is the body frame, which frames the human body. The body frame is a rectangular frame, which is beneficial for coordinate positioning. The human detection model PersonDetect can detect whether there are human body features in the picture, such as: whether features such as the head, body, legs, shoes, arms, clothes, etc. are recognized. After recognizing it as a human body, the human body will be marked with a body frame, and the body frame coordinates person(x, y, w, h) will be obtained.

[0052] S203. Perform two-dimensional human key point detection on the body frame and head frame through a human key point detection model, and calculate the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates.

[0053] Among them, the human key point detection model is a pre-obtained human key point model KeyPoint. The input of this key point detection model is the human body frame, and the output is the two-dimensional foot center point coordinates footCenter(x, y) of the human body. During the above two-dimensional human key point detection process, the target tracking algorithm can also be used to track personnel. After calibrating the human body frame, human key point detection can be performed to obtain the coordinates of the two feet, and the midpoint between the two feet can be calculated based on the coordinates of the two feet, that is, the two-dimensional foot center point coordinates. In addition, after calibrating the human body frame, human head detection can be performed from the human body frame to obtain and calibrate the head frame headRect(x, y, w, h), calculate the center position of the head frame, and obtain the two-dimensional head center point coordinates headCenter(x, y).

[0054] The above human key point detection model can detect the head, left ankle, right ankle, nose, left eye, right eye, left ear, right ear, left wrist, right wrist, left elbow, right elbow, left shoulder, right shoulder, left hip, right hip, left knee, right knee, etc. of the personnel. According to the obtained two-dimensional key point coordinates, human body estimation can be performed, that is, the posture of the target personnel in the current state can be estimated. It should be noted that during the detection process, when the human body is detected to be in a squatting state, height detection is not performed, and when the human body is detected to be in a standing state, detection will be performed, that is, human body posture estimation will be performed.

[0055] S204. Based on the internal parameter matrix of the image acquisition device, convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional pixel coordinate system to the two-dimensional image coordinate system.

[0056] Among them, before installing the passenger flow camera, the basic parameters of the passenger flow camera have been set, and the above basic parameters of the passenger flow camera include the internal parameter matrix and the external parameter matrix of the passenger flow camera. Therefore, based on the internal parameter matrix of the passenger flow camera, according to the conversion relationship between the 2D pixel coordinate system and the 2D image coordinate system, the two-dimensional head center point coordinates in the 2D pixel coordinate system can be converted to the 2D image coordinate system, and the two-dimensional foot center point coordinates in the 2D pixel coordinate system can be converted to the 2D image coordinate system. Specifically, the conversion relationship between the 2D pixel coordinate system (u, v) and the 2D image coordinate system (x, y) is as follows:

[0057]

[0058] Among them, the left side of the equal sign is the 2D pixel coordinate system (u, v), and the right side of the equal sign is the product of the internal parameter matrix and the 2D image coordinate system (x, y). The f x and f y respectively refer to the actual distance of one pixel in the x direction and the y direction, with the unit of mm; c x and cy is the offset distance from the center of the 2D image coordinate system (x, y) to the 2D pixel coordinate system (u, v), with the unit of mm.

[0059] Specifically, in this embodiment, the 2D pixel coordinate system (u, v) is converted into a 3D world coordinate system (x c , y c , z c ). Therefore, the conversion from the above 2D pixel coordinate system (u, v) to the 2D image coordinate system (x, y) can be obtained through the inverse matrix. The internal parameter matrix of the passenger flow camera is known. Substituting the two-dimensional head center point coordinates in the 2D pixel coordinate system (u, v) can convert to obtain the two-dimensional head center point coordinates headCenter(x, y) in the 2D image coordinate system (x, y), and substituting the two-dimensional foot center point coordinates in the 2D pixel coordinate system (u, v) can convert to obtain the two-dimensional foot center point coordinates footCenter(x, y) in the 2D image coordinate system (x, y).

[0060] S205. Based on the external parameter matrix of the image acquisition device, convert the two-dimensional head center point coordinates and two-dimensional foot center point coordinates in the two-dimensional image coordinate system to the three-dimensional world coordinate system to obtain the three-dimensional head center point coordinates and three-dimensional foot center point coordinates.

[0061] Among them, the conversion relationship between the 2D image coordinate system (x, y) and the 3D world coordinate system (x c , y c , z c ) is as follows:

[0062]

[0063] Among them, the left side of the equal sign is the 2D image coordinate system (x, y), and the right side of the equal sign is the product of the external parameter matrix and the 3D world coordinate system (x c , y c , z c ). In the external parameter matrix, f represents the focal length of the passenger flow camera, and Zc represents the depth of the camera.

[0064] The above external parameter matrix of the passenger flow camera is known data. Therefore, substituting the two-dimensional foot center point coordinates footCenter(x, y) and two-dimensional head center point coordinates headCenter(x, y) in the 2D image coordinate system (x, y) into the conversion formula between the above 2D image coordinate system (x, y) and the 3D world coordinate system (x, y, z), the three-dimensional head center point coordinates 3Dhead(x, y, z) and three-dimensional foot center point coordinates 3Dfoot(x, y, z) in the 3D world coordinate system (x, y, z) can be calculated through the inverse matrix.

[0065] S206. Determine the reference plane and select auxiliary points from the reference plane.

[0066] S207. Connect the coordinates of the auxiliary points and the coordinates of the three-dimensional head center point to form an auxiliary line, and the auxiliary line is perpendicular to the ground.

[0067] Among them, take the ceiling of the human head as the reference plane, and take the auxiliary point 3DheadTmp(x + 1, y, z) on the reference plane, and connect the coordinates of the three-dimensional head center point 3Dhead(x, y, z) to obtain the auxiliary line L. Among them, the auxiliary line L formed by the auxiliary point 3DheadTmp(x + 1, y, z) and the coordinates of the three-dimensional head center point 3Dhead(x, y, z) is perpendicular to the ground. Therefore, in the 3D world coordinate system, there is no deviation in the x direction and the y direction, and only assistance is provided in the z direction.

[0068] S208. Calculate the distance from the coordinates of the three-dimensional foot center point to the auxiliary line, and the distance is the estimated distance from the coordinates of the three-dimensional head center point to the coordinates of the three-dimensional foot center point.

[0069] Through the distance calculation method from a point to a line, the distance from the coordinates of the three-dimensional foot center point 3Dfoot(x, y, z) to the auxiliary line L can be calculated, and the distance is used as the estimated distance between the three-dimensional head center point 3Dhead(x, y, z) and the three-dimensional foot center point 3Dfoot(x, y, z).

[0070] S209. Calculate the height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image, and the two-dimensional foot center point coordinates.

[0071] In the embodiment of the present invention, human detection is achieved through the human detection model PersonDetect, and the human body is marked with a human body box and the head box is marked. The two-dimensional foot center point coordinates footCenter(x, y) can be obtained from the human body box through the human key point model KeyPoint, and the two-dimensional head center point coordinates headCenter(x, y) can be obtained from the head box through the human key point model KeyPoint. Then, based on the conversion relationship between the 2D pixel coordinate system and the 2D image coordinate system, and the conversion relationship between the 2D image coordinate system and the 3D world coordinate system, the two-dimensional foot center point coordinates footCenter(x, y) and the two-dimensional head center point coordinates headCenter(x, y) are respectively converted into the three-dimensional foot center point 3Dfoot(x, y, z) and the three-dimensional head center point 3Dhead(x, y, z) by means of the inverse matrix, so as to realize the coordinate conversion between the 2D pixel coordinate system and the 3D world coordinate system. The obtained three-dimensional coordinate points are more conducive to height estimation. In addition, auxiliary lines are made by means of obtaining auxiliary points with the help of a reference plane, the estimated distance is calculated, and the height of the person is calculated by combining the estimated distance and the internal parameter matrix and external parameter matrix of the above passenger flow camera, the human body box coordinates, head box coordinates and two-dimensional foot center point coordinates of the person image. Such an estimation method has a small visual difference and the calculated height of the person is more accurate.

[0072] Please refer to Figure 3 , Figure 3 is the Figure 2 flow chart of step S203 provided by the embodiment of the present invention, which specifically includes:

[0073] S2031. Detect the foot key points of the human body box through the human key point detection model, and respectively obtain the first foot coordinates and the second foot coordinates.

[0074] Among them, after calibrating the human body box, the foot key points of the human body are detected. The above first foot coordinates and second foot coordinates respectively represent the left foot coordinates and right foot coordinates in the human body box. The left foot coordinates can be represented by the left ankle coordinates leftankle(x, y), and the right foot coordinates are represented by the right ankle coordinates rightAnkle(x, y).

[0075] S2032. Calculate the mean coordinates based on the first foot coordinates and the second foot coordinates, and use the mean coordinates as the two-dimensional foot center point coordinates.

[0076] Among them, based on the left ankle coordinates leftankle(x, y) and the right ankle coordinates rightAnkle(x, y), the two-dimensional foot center point coordinates footCenter(x, y) can be obtained by calculating the mean value.

[0077] As a possible implementation manner, a compensation value can be preset. When two-foot coordinates are obtained but only one foot is detected, the coordinates of the detected single foot combined with the compensation value can be used as the two-dimensional foot center point coordinates. For example: when only the left foot / right foot is detected, after positioning the coordinates of the left foot / right foot, the coordinates of the left foot / right foot are increased or decreased in the horizontal x-axis direction by combining the compensation value, and the calculated result is used as the above two-dimensional foot center point coordinates.

[0078] S2033. Perform head key point detection on the human body frame through the human key point detection model, mark the head frame coordinates, calculate the center position of the head frame according to the head frame coordinates, and use the center position as the two-dimensional head center point coordinates.

[0079] In the process of obtaining the two-dimensional foot center point coordinates footCenter(x, y), the head frame in the human body frame can also be marked, and at the same time, the head frame coordinates headRect(x, y, w, h) are obtained. The head frame can be a rectangular frame. Based on the width w and height h of the head frame coordinates headRect(x, y, w, h), the center point of the rectangular frame can be calculated, and the coordinates of this center point are the two-dimensional head center point coordinates headCenter(x, y).

[0080] In this embodiment, through the human key point detection of the human key point detection model, the coordinate mean value of the left foot ankle coordinates leftankle(x, y) and the right foot ankle coordinates rightAnkle(x, y) is used as the two-dimensional foot center point coordinates footCenter(x, y), and the two-dimensional head center point coordinates headCenter(x, y) are calculated according to the head frame coordinates through head frame calibration. In this way, the obtained data is more accurate.

[0081] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a device for calculating the height of a person provided by an embodiment of the present invention. As Figure 4 shown, the device 400 includes:

[0082] The first acquisition module 401 is used to acquire the person image collected by the image acquisition device;

[0083] The second acquisition module 402 is used to acquire the two-dimensional key point coordinates of the person included in the person image from the person image. The two-dimensional key point coordinates include the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates;

[0084] The conversion module 403 is used to convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates through the conversion from the two-dimensional pixel coordinate system to the three-dimensional world coordinate system, and obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates;

[0085] The first calculation module 404 is configured to obtain the coordinates of the auxiliary points of the reference plane, form an auxiliary line based on the coordinates of the auxiliary points and the coordinates of the three-dimensional head center point, and calculate the estimated distance from the coordinates of the three-dimensional foot center point to the auxiliary line;

[0086] The second calculation module 405 is configured to calculate the height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the coordinates of the human body frame of the person included in the person image, the coordinates of the head frame of the person included in the person image, and the coordinates of the two-dimensional foot center point.

[0087] Optionally, as Figure 5 shown, Figure 5 FIG. is a schematic structural diagram of another device for calculating the height of a person provided by an embodiment of the present invention. Among them, the second acquisition module 402 includes:

[0088] The first detection unit 4021 is configured to detect the human body frame and the head frame of the person included in the person image based on the human body detection model. The human body frame has corresponding human body frame coordinates, and the head frame has corresponding head frame coordinates;

[0089] The second detection unit 4022 is configured to perform two-dimensional human key point detection on the human body frame and the head frame through the human key point detection model, and calculate the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates.

[0090] Optionally, as Figure 6 shown, Figure 6 FIG. is a schematic structural diagram of another device for calculating the height of a person provided by an embodiment of the present invention. Among them, the second detection unit 4022 includes:

[0091] The first detection subunit 40221 is configured to perform foot key point detection on the human body frame through the human key point detection model, and respectively obtain the first foot coordinates and the second foot coordinates;

[0092] The first calculation subunit 40222 is configured to calculate the mean coordinates based on the first foot coordinates and the second foot coordinates, and use the mean coordinates as the two-dimensional foot center point coordinates;

[0093] The second calculation subunit 40223 is configured to perform head key point detection on the human body frame through the human key point detection model, mark the head frame coordinates, calculate the center position of the head frame according to the head frame coordinates, and use the center position as the two-dimensional head center point coordinates.

[0094] Optionally, as Figure 7 shown, Figure 7 FIG. is a schematic structural diagram of another device for calculating the height of a person provided by an embodiment of the present invention. Among them, the device 400 further includes:

[0095] A setting module 406 is configured to set the basic parameters of an image acquisition device. The basic parameters of the image acquisition device include the internal parameter matrix and the external parameter matrix of the image acquisition device;

[0096] And the above conversion module 403 includes:

[0097] A first calculation unit 4031 is configured to convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional pixel coordinate system to the two-dimensional image coordinate system based on the internal parameter matrix of the image acquisition device;

[0098] A second calculation unit 4031 is configured to convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional image coordinate system to the three-dimensional world coordinate system based on the external parameter matrix of the image acquisition device, so as to obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates.

[0099] As Figure 8 shown, Figure 8 is a schematic structural diagram of another device for calculating the height of a person provided by an embodiment of the present invention. Among them, the first calculation module 404 includes:

[0100] A selection unit 4041 is configured to determine a reference plane and select auxiliary points from the reference plane;

[0101] A connection unit 4042 is configured to connect the coordinates of the auxiliary points and the three-dimensional head center point coordinates to form an auxiliary line, and the auxiliary line is perpendicular to the ground;

[0102] A third calculation unit 4043 is configured to calculate the distance from the three-dimensional foot center point coordinates to the auxiliary line, and the distance is the estimated distance from the three-dimensional head center point coordinates to the three-dimensional foot center point coordinates.

[0103] The device for calculating the height of a person provided by the embodiment of the present invention can implement Figures 1-3 each implementation manner in the embodiment of the method for calculating the height of a person in , and the corresponding beneficial effects. To avoid repetition, they are not described here again.

[0104] As Figure 9 shown, Figure 9 is a structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 9 shown, it includes: a memory 902, a processor 901, a network interface 903, and a computer program stored on the memory 902 and executable on the processor 901. Among them:

[0105] The processor 901 is configured to call the computer program stored in the memory 902 and execute the following steps:

[0106] Obtain a person image collected by an image acquisition device;

[0107] Obtain the two-dimensional key point coordinates of the person included in the person image, where the two-dimensional key point coordinates include the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates;

[0108] Convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates through the conversion from the two-dimensional pixel coordinate system to the three-dimensional world coordinate system to obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates;

[0109] Obtain the auxiliary point coordinates of the reference plane, form an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculate the estimated distance from the three-dimensional foot center point coordinates to the auxiliary line;

[0110] Calculate the person height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the human body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image, and the two-dimensional foot center point coordinates.

[0111] Optionally, the obtaining of the two-dimensional key point coordinates of the person included in the person image executed by the processor 901 includes:

[0112] Detect the human body frame and the head frame of the person included in the person image based on the human body detection model. The human body frame has corresponding human body frame coordinates, and the head frame has corresponding head frame coordinates;

[0113] Perform two-dimensional human key point detection on the human body frame and the head frame through the human key point detection model, and calculate and obtain the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates.

[0114] Optionally, the performing of two-dimensional human key point detection on the human body frame and the head frame through the human key point detection model and calculating and obtaining the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates executed by the processor 901 includes:

[0115] Perform foot key point detection on the human body frame through the human key point detection model, and respectively obtain the first foot coordinates and the second foot coordinates;

[0116] Calculate the mean coordinates based on the first foot coordinates and the second foot coordinates, and use the mean coordinates as the two-dimensional foot center point coordinates;

[0117] Perform head key point detection on the human body frame through the human key point detection model, mark the head frame coordinates, calculate the center position of the head frame according to the head frame coordinates, and use the center position as the two-dimensional head center point coordinates.

[0118] Optionally, the processor 901 is further configured to execute:

[0119] Set the basic parameters of the image acquisition device, where the basic parameters of the image acquisition device include the internal parameter matrix and the external parameter matrix of the image acquisition device.

[0120] Optionally, the conversion of the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates by the processor 901 through the two-dimensional pixel coordinate system to the three-dimensional world coordinate system includes:

[0121] Based on the internal parameter matrix of the image acquisition device, convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional pixel coordinate system to the two-dimensional image coordinate system;

[0122] Based on the external parameter matrix of the image acquisition device, convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional image coordinate system to the three-dimensional world coordinate system, and obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates.

[0123] Optionally, when the processor 901 executes to obtain the auxiliary point coordinates of the reference plane, form an auxiliary line based on the auxiliary point coordinates and the three-dimensional head center point coordinates, and calculate the estimated distance from the three-dimensional foot center point coordinates to the auxiliary line, including:

[0124] Determine the reference plane and select auxiliary points from the reference plane;

[0125] Connect the coordinates of the auxiliary points and the three-dimensional head center point coordinates to form an auxiliary line, and the auxiliary line is perpendicular to the ground;

[0126] Calculate the distance from the three-dimensional foot center point coordinates to the auxiliary line, and the distance is the estimated distance from the three-dimensional head center point coordinates to the three-dimensional foot center point coordinates.

[0127] The electronic device provided by the embodiments of the present invention can implement various implementation manners and corresponding beneficial effects in the embodiments of the calculation method based on the height of a person. To avoid repetition, they will not be described here again.

[0128] It should be noted that only components 901-903 are shown in the figure. However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of the present technology can understand that the electronic device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0129] The electronic device 900 can be a computing device such as a desktop computer, a notebook, a handheld computer, and a cloud server. The electronic device 900 can interact with a customer through a keyboard, a mouse, a remote control, a touchpad, a voice control device, or other means.

[0130] The memory 902 includes at least one type of readable storage medium. The readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 901 can be an internal storage unit of the electronic device, such as the hard disk or the memory of the electronic device. In other embodiments, the memory 901 can also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Of course, the memory 901 can also include both the internal storage unit and the external storage device of the electronic device. In this embodiment, the memory 901 is generally used to store the operating system and various application software installed on the electronic device, such as the program code of the method for calculating the height of a person. In addition, the memory 901 can also be used to temporarily store various types of data that have been output or will be output.

[0131] In some embodiments, the processor 901 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 902 is generally used to control the overall operation of the electronic device. In this embodiment, the processor 901 is used to run the program code stored in the memory 901 or process data, such as running the program code of the method for calculating the height of a person.

[0132] The network interface 903 can include a wireless network interface or a wired network interface. The network interface 903 is generally used to establish a communication connection between the electronic device 900 and other electronic devices.

[0133] The embodiment of the present invention also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor 901, it implements each process of the embodiment of the method for calculating the height of a person provided by the embodiment of the present invention, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0134] Those of ordinary skill in the art can understand that all or part of the process of implementing the method for calculating the height of a person can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the various methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory 902 (Random Access Memory, abbreviated as RAM), etc.

[0135] The above-disclosed are only the preferred embodiments of the present invention. Of course, the scope of the rights of the present invention cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. A method for calculating a person's height, characterized in that: The method comprises the following steps: Acquiring an image of a person captured by an image acquisition device; Acquire two-dimensional key point coordinates of a person included in the person image from the person image, the two-dimensional key point coordinates including two-dimensional head center point coordinates and two-dimensional foot center point coordinates; Converting the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates by converting the two-dimensional pixel coordinate system to a three-dimensional world coordinate system to obtain three-dimensional head center point coordinates and three-dimensional foot center point coordinates; Using the ceiling above the person's head as a reference surface, select auxiliary points from the reference surface; connect the coordinates of the auxiliary points with the coordinates of the three-dimensional head center point to form an auxiliary line, with the auxiliary line perpendicular to the ground; calculate the distance from the three-dimensional foot center point coordinate to the auxiliary line, where the distance is the estimated distance from the three-dimensional head center point coordinate to the three-dimensional foot center point coordinate; The height of the person included in the person image is calculated based on the estimated distance, the basic parameters of the image acquisition device, the body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image and the two-dimensional foot center point coordinates, wherein the person height specific The calculation formula is as follows: height=headFootHeight+headRect.h*fy+(person.y+person.hfootCenter.y)*fy Among them: headFootHeight represents the estimated distance; headRect.h represents the height in the head frame coordinates; fy is a basic camera parameter, which refers to the actual distance of one pixel in the y direction, in mm; person.y represents the y coordinate value of the body frame coordinates corresponding to the person image; person.h represents the height in the body frame coordinates; footCenter.y represents the y coordinate value of the two-dimensional foot center coordinates.

2. The method according to claim 1, wherein The acquiring, from the person image, the two-dimensional key point coordinates of the person contained in the person image includes: Detecting a human body frame and a head frame of a person contained in a person image based on a human body detection model, wherein the human body frame has corresponding human body frame coordinates, and the head frame has corresponding head frame coordinates; The human body frame and the head frame are subjected to two-dimensional human body key point detection by a human body key point detection model, and the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates are obtained by calculation.

3. The method according to claim 2, wherein The performing two-dimensional human key point detection on the human body frame and the head frame by using a human key point detection model, and calculating the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates, includes: Perform foot key point detection on the human body frame using the human body key point detection model to obtain a first foot coordinate and a second foot coordinate respectively; Calculating mean coordinates based on the first foot coordinates and the second foot coordinates, and using the mean coordinates as the two-dimensional foot center point coordinates; The human body frame is subjected to head key point detection by the human body key point detection model, the head frame coordinates are marked, the center position of the head frame is calculated according to the head frame coordinates, and the center position is used as the two-dimensional head center point coordinates.

4. The method according to claim 1, wherein The method further comprises: Setting basic parameters of the image acquisition device, wherein the basic parameters of the image acquisition device include an intrinsic parameter matrix and an extrinsic parameter matrix of the image acquisition device; The converting of the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates by converting the two-dimensional pixel coordinate system to the three-dimensional world coordinate system includes: Based on the intrinsic parameter matrix of the image acquisition device, the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional pixel coordinate system are converted into the two-dimensional image coordinate system; Based on the extrinsic parameter matrix of the image acquisition device, the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates in the two-dimensional image coordinate system are converted to the three-dimensional world coordinate system to obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates.

5. A device for calculating a person's height, characterized in that: include: A first acquisition module is used to acquire a person image captured by an image acquisition device; a second acquisition module, configured to acquire, from the person image, two-dimensional key point coordinates of the person contained in the person image, the two-dimensional key point coordinates including two-dimensional head center point coordinates and two-dimensional foot center point coordinates; a conversion module, configured to convert the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates by converting the two-dimensional pixel coordinate system into a three-dimensional world coordinate system to obtain the three-dimensional head center point coordinates and the three-dimensional foot center point coordinates; The first calculation module is configured to use the ceiling above the person's head as a reference surface and select auxiliary points from the reference surface; connect the coordinates of the auxiliary points with the coordinates of the three-dimensional head center point to form an auxiliary line, wherein the auxiliary line is perpendicular to the ground; Calculating the distance between the three-dimensional foot center coordinate and the auxiliary line, where the distance is an estimated distance between the three-dimensional head center coordinate and the three-dimensional foot center coordinate; The second calculation module is used to calculate the height of the person included in the person image based on the estimated distance, the basic parameters of the image acquisition device, the body frame coordinates of the person included in the person image, the head frame coordinates of the person included in the person image and the two-dimensional foot center point coordinates, wherein the person height specific The calculation formula is as follows: height=headFootHeight+headRect.h*fy+(person.y+person.hfootCenter.y)*fy Among them: headFootHeight represents the estimated distance; headRect.h represents the height in the head frame coordinates; fy is a basic camera parameter, which refers to the actual distance of one pixel in the y direction, in mm; person.y represents the y coordinate value of the body frame coordinates corresponding to the person image; person.h represents the height in the body frame coordinates; footCenter.y represents the y coordinate value of the two-dimensional foot center coordinates.

6. The device according to claim 5, characterized in that The acquisition module includes: A first detection unit is configured to detect a human body frame and a head frame of a person contained in a person image based on a human body detection model, wherein the human body frame has corresponding human body frame coordinates, and the head frame has corresponding head frame coordinates; The second detection unit is used to perform two-dimensional human key point detection on the human body frame and the head frame through a human key point detection model, and calculate the two-dimensional head center point coordinates and the two-dimensional foot center point coordinates.

7. The device according to claim 6, characterized in that The second detection unit includes: a first detection subunit, configured to perform foot key point detection on the human body frame using the human body key point detection model, and obtain first foot coordinates and second foot coordinates respectively; a first calculating subunit, configured to calculate mean coordinates based on the first foot coordinates and the second foot coordinates, and use the mean coordinates as the two-dimensional foot center point coordinates; The second calculation subunit is used to perform head key point detection on the human body frame through the human body key point detection model, mark the head frame coordinates, calculate the center position of the head frame according to the head frame coordinates, and use the center position as the two-dimensional head center point coordinates.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for calculating a person's height as claimed in any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for calculating a person's height according to any one of claims 1 to 4.

Images