Method and apparatus for measuring a human parameter

Abstract

The application relates to the technical field of human parameter measurement, and provides a human parameter measurement method and device, which are characterized in that according to the mapping relationship of pixel points in a human RGB image and a human depth image synchronously collected by a depth camera, a human mask graph is accurately segmented from the depth image, a target human point cloud is extracted according to the human mask graph, the real height of the human body is confirmed, a human parameterized model generated by a single human RGB image is scaled according to the real height of the human body, the scaled human parameterized model is consistent with the real body shape of the human body, the scaled human parameterized model is cut through a preset part to be measured, and the three-dimensional coordinates of the intersection point of the tangent line passing through the vertex of the cutting plane and the vertex connecting line can be used to quickly and accurately determine the human parameters of each preset part, and the application is suitable for human parameter measurement in various postures and has higher robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human parameter measurement, and provides a human parameter measurement method and device. BACKGROUND

[0002] With the improvement of the technology level, the research on human body shape gradually deepens. At present, the human parameter measurement methods mainly include measuring human parameters by scanning equipment and measuring human parameters by vision.

[0003] The method for measuring human parameters by scanning equipment scans the human body in all directions by means of scanning equipment to obtain human parameters. This method has high measurement accuracy, but the scanning equipment has high cost and cannot be widely used.

[0004] The method for measuring human parameters by vision is divided into a single-image-based measurement method and a multi-image-based measurement method. The single-image-based measurement method generates a human parameterized model by using a single image, and measures human parameters based on the human parameterized model. However, the human parameterized model generated by the single image cannot be aligned with the real human body, resulting in unreal human parameters. The multi-image-based measurement method extracts human point clouds from human images taken at different angles by a depth camera to reconstruct a high-precision human parameterized model. However, the human point clouds and the human parameterized model need to be continuously optimized and aligned by Iterated Closest Points (ICP), resulting in a long time consumption of human parameter measurement.

[0005] Therefore, it is an urgent problem to provide a fast, accurate and low-cost human parameter measurement method. SUMMARY

[0006] The present application provides a human parameter measurement method and device for improving the accuracy and speed of human parameter measurement.

[0007] In one aspect, the present application provides a human parameter measurement method, comprising:

[0008] receiving a human RGB image and a target human point cloud sent by a client, wherein the target human point cloud is extracted from a human mask image obtained from a synchronous acquisition human depth image according to a first human bounding box, and the first human bounding box is determined according to the mapping relationship of pixel points in the human RGB image and the human depth image;

[0009] converting the human RGB image into a fixed-size target RGB image, and generating a human parameterized model according to the target RGB image;

[0010] determine a real human height according to the target human point cloud, and scale the human parametric model according to the real human height;

[0011] determine human parameters of each preset part according to three-dimensional coordinates of each vertex in the scaled human parametric model, and send the human parameters to the client for display.

[0012] In another aspect, the present application provides a human parameter measurement method, comprising:

[0013] obtaining a human RGB image and a human depth image synchronously captured by a depth camera;

[0014] determining a first human bounding box in the human depth image according to a mapping relationship of pixel points in the human RGB image and the human depth image;

[0015] obtaining a human mask image from the human depth image according to the first human bounding box, and extracting a target human point cloud according to the human mask image;

[0016] sending the human RGB image and the target human point cloud to a server for human parameter measurement;

[0017] receiving and displaying a human parameter measurement result sent by the server.

[0018] In another aspect, the present application provides a human parameter measurement method, comprising:

[0019] obtaining a human RGB image and a human depth image synchronously captured by a depth camera;

[0020] determining a first human bounding box in the human depth image according to a mapping relationship of pixel points in the human RGB image and the human depth image;

[0021] obtaining a human mask image from the human depth image according to the first human bounding box, and extracting a target human point cloud according to the human mask image;

[0022] converting the human RGB image into a target RGB image with a fixed size, and generating a human parametric model according to the target RGB image;

[0023] determining a real human height according to the target human point cloud, and scaling the human parametric model according to the real human height;

[0024] determining human parameters of each preset part according to three-dimensional coordinates of each vertex in the scaled human parametric model, and displaying the human parameters through a display.

[0025] In another aspect, the present application provides a server device, comprising a processor, a memory and a communication interface, the communication interface, the memory and the processor are connected through a bus, the memory comprises a data storage unit and a program storage unit, the program storage unit stores a computer program, and the processor executes the following operations according to the computer program:

[0026] The communication interface receives the human body RGB image and the target human body point cloud sent by the client and stores them in the data storage unit, wherein the target human body point cloud is extracted from a human body mask image according to a first human body bounding box from a synchronously collected human body depth image, and the first human body bounding box is determined according to the mapping relationship of the pixel points in the human body RGB image and the human body depth imagev

[0027] The human body RGB image is converted into a fixed-size target RGB image, and a human body parameterized model is generated according to the target RGB image;

[0028] The real human body height is determined according to the target human body point cloud, and the human body parameterized model is scaled according to the real human body height;

[0029] The human body parameters of each preset part are determined according to the three-dimensional coordinates of each vertex in the scaled human body parameterized model, and the human body parameter measurement result is sent to the client for display through the communication interface.

[0030] In another aspect, the present application provides a client device, comprising a processor, a memory, a display screen and at least one communication interface, the communication interface, the display screen, the memory and the processor are connected through a bus, the memory comprises a data storage unit and a program storage unit, the program storage unit stores a computer program, and the processor executes the following operations according to the computer program:

[0031] The communication interface acquires a human body RGB image and a human body depth image synchronously collected by a depth camera and stores them in the data storage unit;

[0032] The first human body bounding box in the human body depth image is determined according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image;

[0033] The human body mask image is obtained from the human body depth image according to the first human body bounding box, and the target human body point cloud is extracted according to the human body mask image;

[0034] The communication interface sends the human body RGB image and the target human body point cloud to the server for human body parameter measurement;

[0035] The communication interface receives the human parameter measurement result sent by the server and displays the result through the display screen.

[0036] In another aspect, the present application provides a human parameter measurement device, comprising a processor, a memory and a communication interface, the communication interface, the memory and the processor are connected through a bus, the memory comprises a data storage unit and a program storage unit, the program storage unit stores a computer program, and the processor executes the following operations according to the computer program:

[0037] The communication interface acquires a frame of human RGB image and human depth image synchronously collected by a depth camera and stores them into the data storage unit;

[0038] According to the mapping relationship of the pixel points in the human RGB image and the human depth image, a first human boundary box in the human depth image is determined;

[0039] A human mask image is obtained from the human depth image according to the first human boundary box, and a target human point cloud is extracted according to the human mask image;

[0040] The human RGB image is converted into a target RGB image with a fixed size, and a human parameterized model is generated according to the target RGB image;

[0041] The real human height is determined according to the target human point cloud, and the human parameterized model is scaled according to the real human height;

[0042] The human parameters of each preset part are determined according to the three-dimensional coordinates of each vertex in the scaled human parameterized model, and the human parameters are displayed through a display, wherein the display comprises a built-in display screen or an external display screen.

[0043] In another aspect, the present application provides a computer readable storage medium, which stores computer executable instructions for causing a computer device to execute the human parameter measurement method provided by the embodiments of the present application.

[0044] The method and device for measuring human body parameters provided in the embodiments of the present application, the client acquires a frame of human body RGB image and human body depth image synchronously collected, in the case of only one frame of human body depth image, the accuracy of foreground and background segmentation of the human body depth image through the network is poor, and due to the different resolutions of the human body RGB image and the human body depth image, the human body depth image cannot be directly segmented by using the bounding box in the human body RGB image, therefore, according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image, the first human body bounding box in the human body depth image is determined and the human body mask image is acquired, so as to accurately segment the human body in the depth image, and then the target human body point cloud is extracted according to the human body mask image, and the human body RGB image and the target human body point cloud are sent to the server. The server converts the human body RGB image into a target RGB image with a fixed size, generates a human body parameterized model according to the target RGB image, and determines the real human body height according to the target human body point cloud, and then scales the human body parameterized model according to the real human body height, the scaled human body parameterized model is consistent with the real body shape of the human body, so that the human body parameters of each preset part can be quickly and accurately determined according to the three-dimensional coordinates of each vertex in the scaled human body parameterized model. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0046] Figure 1A The system architecture diagram for measuring human body parameters provided in the embodiments of the present application;

[0047] Figure 1B Another system architecture diagram for measuring human body parameters provided in the embodiments of the present application;

[0048] Figure 1C Another system architecture diagram for measuring human body parameters provided in the embodiments of the present application;

[0049] Figure 2 The flowchart of the method for measuring human body parameters on the client side provided in the embodiments of the present application;

[0050] Figure 3 The flowchart of the method for segmenting the human body mask image from the human body depth image provided in the embodiments of the present application;

[0051] Figure 4 The flowchart of the method for measuring human body parameters on the server side provided in the embodiments of the present application;

[0052] Figure 5 A method flowchart for generating a human parameterized model based on a single RGB image is provided for the embodiments of the present application;

[0053] Figure 6 An effect diagram of a human parameterized model generated based on a single RGB image is provided for the embodiments of the present application;

[0054] Figure 7 A human real height diagram is provided for the embodiments of the present application;

[0055] Figure 8 An effect diagram of an adjusted human parameterized model is provided for the embodiments of the present application;

[0056] Figure 9 A process diagram for measuring a human height parameter is provided for the embodiments of the present application;

[0057] Figure 10 A transverse section diagram of a plurality of vertices associated with a selected waist circumference is provided for the embodiments of the present application;

[0058] Figure 11 A diagram of a plurality of vertices associated with a plurality of selected parts is provided for the embodiments of the present application;

[0059] Figure 12 A method flowchart for determining a human parameter of a preset part is provided for the embodiments of the present application;

[0060] Figure 13 A diagram of a planar cut human parameterized model is provided for the embodiments of the present application;

[0061] Figure 14 A diagram of a planar cut surface passing through a patch is provided for the embodiments of the present application;

[0062] Figure 15 A diagram of an intersection of a tangent of a planar cut surface and a patch is provided for the embodiments of the present application;

[0063] Figure 16 A diagram of a length ratio of two vertices of a line segment where an intersection point is located is provided for the embodiments of the present application;

[0064] Figure 17 A method flowchart for measuring another human parameter on a server side is provided for the embodiments of the present application;

[0065] Figure 18 A method flowchart for measuring another human parameter on a client side is provided for the embodiments of the present application;

[0066] Figure 19 A client device result diagram is provided for the embodiments of the present application;

[0067] Figure 20 A server device result diagram provided for an embodiment of the present application;

[0068] Figure 21 A human parameter measurement device result diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments described in the present application document, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application technical scheme.

[0070] Human parameters are closely related to various aspects of people's life, and are the basic technical basis for measuring the production capacity of a country. By understanding the body parameters of most people, size standards suitable for different age stages will be obtained, and the clothes and objects used by people will also be more suitable. Therefore, human parameters are one of the important information for enterprises to design and produce products.

[0071] For example, what shape of mask and clothing is most suitable for consumers, how high a desk and chair is most suitable for students, and how architects design the spacing and height of the guardrails of a pedestrian overpass to ensure the safety of pedestrians, etc. These common objects all need related human parameters as the basis for production design.

[0072] The traditional manual measurement method of human parameters mostly uses tools such as a flexible ruler, an altimeter, a range finder, an angle meter, etc. to directly measure the surface length of each part of the human body in the horizontal, vertical, and oblique directions. This method is quick and simple, and the measuring tools are easy to use, but it takes too long to measure, has large errors, and the measurement results are affected by the skill level of the measurer, which is not conducive to large-scale and rapid human parameter measurement.

[0073] With the continuous improvement of technology, people's research on human body shape has gradually deepened, and human databases have been established at home and abroad to save human information for production design. For example, the human database is used for the establishment and repair of clothing standard sizes, as well as clothing design and customization.

[0074] With the development of virtual reality, computer vision, computer graphics technology, three-dimensional reconstruction technology gradually becomes the popular research content in the field of VR, three-dimensional display and remote three-dimensional communication. Three-dimensional reconstruction refers to the process of reconstructing three-dimensional information according to single view or multi-view images, including static three-dimensional reconstruction and dynamic three-dimensional reconstruction, and the method of measuring human body parameters based on human body three-dimensional reconstruction model can solve the problems in traditional manual measurement method, and gradually becomes the research hotspot in various fields, for example, virtual online fitting based on human body model becomes a new shopping way.

[0075] Human feature point position and feature parameter reflect important information of human basic shape, and human parameter measurement provides basic data for individual morphological study through visual principle and fitting method, which is the core technology of human body shape research. Therefore, the realization of convenient, accurate and low-cost human measurement method has become the key of human measurement technology.

[0076] At present, the non-contact human parameter measurement method is divided into active measurement method and passive measurement method.

[0077] The active measurement method is to determine the human body parameters by scanning the human body in all directions through the scanning device, wherein the scanning method includes laser three-dimensional scanning method, moire fringe scanning method and infrared measurement method. Since this method is human body scanning by using special scanning equipment, the measurement accuracy is high, but the scanning equipment is expensive, and it is difficult to be widely popularized, and when scanning, a specific scanning scene needs to be arranged to collect different parts of the human body, so that the accurate scanning data can be obtained to accurately solve the human body parameters, and at the same time, the arrangement of the specific scanning scene also increases the measurement time.

[0078] The passive measurement method is to measure human body parameters through vision, including monocular vision method, binocular vision method and multi-view vision method, which has lower equipment cost than the active measurement method, but there are some problems in measuring human body parameters through vision.

[0079] For example, the monocular vision method is superior to the single view image without depth information, and the person needs to stand at the camera focus position, which is not easy to determine and brings large measurement error; the binocular vision method and the multi-view vision method can measure the depth information of the human body, but in order to measure the three-dimensional parameters of the human body, multiple groups of data need to be collected, which requires the person to stand still at the same position, and the collection conditions are relatively strict.

[0080] Among them, monocular vision method, binocular vision method and multi-view vision method are divided into ordinary camera and depth camera (such as TOF camera, structured light camera, Kinect camera, binocular camera, etc.) according to different collection devices. Since the ordinary camera cannot obtain the depth information of the human body, the person must stand at a specific position, that is, the distance or angle of this position from the camera is fixed, that is, the fixed depth value, otherwise the human body parameters cannot be accurately measured. The human body point cloud extracted from the human body images of different angles taken by the depth camera can reconstruct a high-precision human body parameterized model, but the generated human body point cloud and human body parameterized model need to be continuously optimized and aligned by non-rigid iterated closest points (ICP), resulting in a long time consumption of human body parameter measurement.

[0081] In view of the problems of the existing human body parameter measurement method, such as complex measurement environment, low measurement accuracy, slow measurement speed and the like, the embodiment of the present application provides a human body parameter measurement method and device. For a human body depth image and a human body RGB image collected by a depth camera at the same time, first, a human body parameterized model aligned with the human body in the image is generated according to the human body RGB image, then the human body in the human body depth image is segmented, and the human body parameterized model is scaled according to the point cloud of the segmented human body part, so that the human body parameterized model is consistent with the size, posture and proportion of the real human body, and finally the parameters of each part of the human body are measured through the human body parameterized model. The method has the characteristics of low cost of collection equipment, simple collection scene, fast measurement speed and high accuracy.

[0082] Referring to Figure 1A The system architecture diagram for measuring human body parameters provided by the embodiment of the present application is shown in Figure 1A The whole system is divided into a front end and a service end. Among them:

[0083] The front end mainly includes a client and a depth camera. The depth camera is used as a collection device to collect human body RGB images and human body depth images. The client has data processing capability and display function, can extract target human body point cloud without background interference according to the human body RGB images and human body depth images collected by the depth camera, and send the target human body point cloud and human body RGB image to the service end in a wired or wireless manner to request the service end to measure human body parameters and display the measurement results returned by the service end.

[0084] The service end is composed of a server, can respond to the request of the client, generate a human body parameterized model according to a single human body RGB image, adjust the human body parameterized model through the real height determined by the target human body point cloud to obtain a human body parameterized model consistent with the real human body, and then measure the parameters of each part of the human body through the real human body parameterized model and return the measurement results to the client for display.

[0085] In some embodiments, the front end can be used only for image acquisition, and image processing, human parameterized model generation and human parameter measurement can all be performed by the service end, so as to fully utilize the powerful data processing capability of the service end. As shown in Figure 1B Fig. 2, another system architecture diagram for measuring human parameters provided by an embodiment of the present application is shown, as shown in Figure 1B Fig. 3, the system includes a front end and a service end, the front end mainly includes a display and a depth camera, and the service end mainly includes a server. In the system, the depth camera is used to acquire human RGB images and human depth images. The service end extracts target human point cloud without background interference according to the human RGB images and the human depth images acquired by the depth camera, and generates a human parameterized model according to a single human RGB image, so as to adjust the human parameterized model through the real height determined by the target human point cloud, obtain a human parameterized model consistent with the real human, and then measure parameters of each part of the human through the real human parameterized model, and return the human parameter measurement result to the display for display.

[0086] In other embodiments, when the data processing capability of the client is strong, the client can directly generate a human parameterized model according to the human depth images and the human RGB images, and measure human parameters based on the human parameterized model. As shown in Figure 1C Fig. 4, another system architecture diagram for measuring human parameters provided by an embodiment of the present application is shown, as shown in Figure 1C Fig. 5, the system only includes a front end, wherein the depth camera is used to acquire human RGB images and human depth images, the client extracts target human point cloud without background interference according to the human RGB images and the human depth images acquired by the depth camera, and generates a human parameterized model according to a single human RGB image, so as to adjust the human parameterized model through the real height determined by the target human point cloud, obtain a human parameterized model consistent with the real human, and then measure parameters of each part of the human through the real human parameterized model, and display the human parameter measurement result.

[0087] Optionally, Figure 1A , Figure 1B and Figure 1C As shown in Fig. 6, the depth camera in the front end can be a TOF camera, and can also be a structured light camera, a Kinect camera, an RGBD camera, etc. The client can be a personal computer (PC), a desktop computer, a smart phone, a tablet computer, a smart television, a vehicle-mounted device, a wearable device, etc. The server in the service end can be one server, and can also be a server cluster, and can also be a general server, a micro server or a cloud server.

[0088] In the following,Figure 1A The system architecture shown is an example, and the specific implementation of the embodiments of the present application is described in detail.

[0089] Referring to Figure 2 A flowchart of a human parameter measurement method provided by the embodiments of the present application is executed by the front end, specifically by the client in the front end, and mainly includes the following steps:

[0090] S201: Obtain a frame of human RGB image and human depth image synchronously collected by the depth camera.

[0091] Before the depth camera collects the image, the depth camera is fixed, and the depth camera is set to be horizontally viewable to collect the complete human body from head to foot. After the user to be measured walks into the field of view of the depth camera, the depth camera synchronously collects a frame of human RGB image and depth image. After the depth camera completes the collection, the client reads the human RGB image and human depth image collected by the depth camera for image processing.

[0092] S202: Determine the first human body bounding box in the human depth image according to the mapping relationship of the pixel points in the human RGB image and the human depth image.

[0093] Currently, there are mainly two methods for segmenting the human mask image from the depth image: the first method is to segment the foreground and background by collecting the background image, specifically: before the user to be measured enters the scene, a background depth image is collected, and after the user to be measured enters the scene, a current depth image is collected, and the current depth image is subtracted from the background depth image, thereby separating the human body, that is, this method requires two times of image collection, and thus it is not applicable to scenes with more changes. The second method is to separate the human body from a single depth image by using a network, but the human mask image separated by this method is not accurate enough, and the network has a high requirement for the performance of the graphics card.

[0094] To improve the convenience and accuracy of human segmentation, in S202, the boundary box of the human body in the human depth image is determined according to the mapping relationship of the pixel points in the human RGB image and the human depth image, thereby the human body is segmented. For the specific process, refer to Figure 3 , which mainly includes the following steps:

[0095] S2021: Perform human recognition on the human RGB image to obtain a second human body bounding box.

[0096] In S2021, a second human body bounding box in a rectangle is obtained by human body recognition on the human body RGB image. There are many human body recognition methods based on the RGB image, and common algorithms include a support vector machine (SVM) algorithm, a convolutional neural network (CNN), a YOLO (You Only Look Once) algorithm, an SSD (Single Shot MultiBox Detector) algorithm, and the like, and the present application does not make a restrictive requirement.

[0097] Generally, the resolutions of the human body RGB image and the human body depth image collected by the depth camera are inconsistent, so the second human body bounding box in the human body RGB image cannot be directly used for human body segmentation on the human body depth image. However, the pixel points in the human body RGB image and the human body depth image synchronously collected by the depth camera have a mapping relationship, and therefore, the first human body bounding box in the human body depth image can be determined by means of the mapping relationship. For details, refer to S2022-S2025.

[0098] S2022: For each of the four corner points of the second human body bounding box, the current pixel point in the preset neighborhood of the corner point is mapped to the human body depth image according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image.

[0099] Because the resolutions of the human body RGB image and the human body depth image are inconsistent, not every pixel point in the images has a one-to-one mapping relationship. Therefore, in S2022, the pixel points in the preset neighborhood of the four corner points of the second human body bounding box in the human body RGB image are mapped, and specifically, for each of the four corner points of the second human body bounding box, the following operation is performed: the current pixel point in the preset neighborhood of the corner point is mapped to the human body depth image according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image, so as to determine the first human body bounding box in the human body depth image according to the correspondence of the current pixel point in the human body depth image.

[0100] S2023: It is determined whether there is a corresponding pixel point of the current pixel point in the human body depth image, if yes, S2024 is performed, and if not, S2025 is performed.

[0101] In S2023, after the current pixel point is mapped to the human body depth image, it is determined whether there is a corresponding pixel point of the current pixel point in the human body depth image. When there is a corresponding pixel point, it indicates that the corresponding pixel point and the current pixel point satisfy the correspondence of the pixel points in the human body RGB image and the human body depth image.

[0102] S2024: Corresponding pixel points are taken as one corner point of the first human body bounding box.

[0103] In S2024, the corresponding pixel points having a mapping relationship with the current pixel point are taken as the corner points of the first human body bounding box.

[0104] S2025: The next pixel point in the preset neighborhood of the corner point is read, and S2023 is executed.

[0105] In S2025, when there is no corresponding pixel point having a mapping relationship with the current pixel point, the next pixel point in the preset neighborhood of the corner point is continuously read until the corresponding pixel point having a mapping relationship is obtained.

[0106] In S202, after the steps of S2022-S2025 are executed for all four corner points of the second human body bounding box, the four corner points of the first human body bounding box can be obtained, thereby determining the first human body bounding box in the human body depth image.

[0107] S203: Obtain a human body mask image from the human body depth image according to the first human body bounding box, and extract a target human body point cloud according to the human body mask image.

[0108] In S203, the least square method is performed on the pixel points in the first human body bounding box in the human body depth image to segment the foreground human body and obtain the human body mask image. At the same time, the human body mask image is converted into the target human body point cloud according to the parameters of the depth camera.

[0109] S204: Send the human body RGB image and the target human body point cloud to the server for human body parameter measurement.

[0110] In S204, an optional embodiment is that after the client obtains the target human body point cloud, the human body RGB image and the target human body point cloud are packaged, and a human body parameter measurement request is sent to the server, and the packaged data is also sent.

[0111] S205: Receive the human body parameter measurement result sent by the server and display.

[0112] In S205, the client intuitively displays the human body parameter measurement result through the user interface, so that subsequent staff can design and produce products according to the human body parameter measurement result.

[0113] The server responds to the request sent by the client and performs human body parameter measurement according to the received data. Referring to Figure 4 The server side human body parameter measurement method flowchart provided by the embodiment of the present application mainly includes the following steps:

[0114] S401: Receive the human body RGB image and the target human body point cloud sent by the client.

[0115] wherein the target human point cloud is extracted from a human mask map obtained from the synchronous acquisition of the human depth image according to the first human bounding box, and the first human bounding box is determined according to the mapping relationship of the pixel points in the human RGB image and the human depth image. For details, refer to the description of the client side, which is not repeated here.

[0116] S402: Convert the human RGB image into a target RGB image of a fixed size, and generate a human parametric model according to the target RGB image.

[0117] In S402, the size of the target image can be set according to actual conditions, and the embodiments of the present application do not have restrictive requirements. After obtaining the target RGB image, the target RGB image is input into the human reconstruction network based on a single image to generate a human parametric model consistent with the human pose and body shape in the human RGB image. For details, refer to Figure 5 , which mainly includes the following steps:

[0118] S4021: Extract human shape parameters and human pose parameters according to the target RGB image.

[0119] The human parametric model is a human shape reconstructed based on a statistical method, which describes the human shape through a set of low-dimensional vectors (i.e. human shape (shape) parameters and human pose (pose) parameters). Common human parametric models include but are not limited to SMPL model, SCAPE model and DECA model, etc.

[0120] In three-dimensional space, a human parametric model can be directly synthesized by giving a set of human shape parameters and human pose parameters. Therefore, in S4021, the shape parameters and pose parameters of the human parametric model are extracted and generated according to the target RGB image.

[0121] S4022: Drive the standard parametric model according to the human shape parameters and human pose parameters to obtain a human parametric model consistent with the human pose and body shape in the target RGB image.

[0122] In the embodiment of the present application, the network for generating a human parameterized model based on a single human RGB image needs to drive the standard parameterized model and map the vertices of the standard parameterized model to a two-dimensional image, that is, drive the standard parameterized model to be consistent with the size and posture of the human body in the target RGB image, no matter whether it is through parameter optimization or through a regression network, according to the human shape parameters and the human posture parameters. However, the number of vertices of the commonly used human parameterized model is generally more than 10,000. If the above operation is performed every time the optimization or regression is performed, the driving operation consumes about 10 ms of time each time, and in addition to the mapping process from three-dimensional vertices to a two-dimensional image, this seriously causes time consumption.

[0123] To solve the above problem, in S4022, at least one uniform downsampling of the vertices in the standard parameterized model is performed when the standard parameterized model is loaded in the initialization driving process. Because the vertices of the human parameterized model are continuously arranged, two consecutive vertices can be collected as one vertex, or three consecutive vertices can be collected as one vertex when downsampling. In this way, after at least one uniform downsampling, the number of vertices of the human parameterized model is reduced, thereby improving the reconstruction efficiency and saving time cost.

[0124] The number and times of uniform sampling can be set according to actual needs, and the embodiment of the present application does not have a restrictive requirement.

[0125] For example, the embodiment of the present application downsamples more than 10,000 vertices of the human parameterized model to more than 1,000, thereby reducing the operation time.

[0126] In the embodiment of the present application, because the size of the target RGB image is fixed, the size of the human parameterized model generated each time also changes little. As shown in FIG. 4, it is an effect diagram of the human parameterized model generated based on a single RGB image provided by the embodiment of the present application. Figure 6

[0127] S403: Determine the real human height according to the target human point cloud, and scale the human parameterized model according to the real human height.

[0128] Because the human parameterized model generated based on a single RGB image is relative to the size of the human body in the human RGB image, and is not the size of the real human body in space, if the generated human parameterized model is directly used for human parameter measurement, the measured human parameters will not conform to the actual human parameters.

[0129] ​For example, if a three-year-old child stands close to a depth camera, the child's size will appear larger in the captured RGB image, resulting in a larger error in the measured human body parameters. For instance, the child's height might be measured as 180cm, which is seriously inconsistent with reality.

[0130] To address the impact of the distance between the user and the depth camera on the measurement results, in S403, the actual human height is determined based on the target human point cloud. Specifically, the actual human height can be obtained by subtracting the minimum value from the maximum value in the vertical direction (e.g., the Y-axis) of the target human point cloud. Figure 7 As shown, the actual height of a human body is represented by h.

[0131] After obtaining the actual human height, the parametric human body model is scaled according to the ratio between the actual human height and the model height of the parametric human body model, aligning the parametric human body model with the actual human body to obtain a parametric human body model with the same proportions as the actual human body. For example... Figure 8 As shown, this is the adjusted human body parameterization model provided in an embodiment of this application.

[0132] S404: Determine the human body parameters for each preset part based on the three-dimensional coordinates of each vertex in the scaled human body parametric model.

[0133] In S404, after obtaining a parametric human body model consistent with the real human body, human body parameters are measured based on the parametric human body model, mainly including the following two methods:

[0134] Method 1

[0135] Since the vertex positions of the parametric human body model remain unchanged (i.e., the vertex indices in the model remain unchanged), some human body parameters (such as height, arm length, leg length, neck length, etc.) can be determined based on the coordinates of the vertices in three-dimensional space. Specifically, according to the national standard measurement method, multiple vertices associated with preset body parts are selected from the scaled parametric human body model, and the human body parameters of the preset body parts are determined based on the three-dimensional coordinates of the selected multiple vertices.

[0136] like Figure 9 As shown, taking the determination of height parameters as an example, the distance between the maximum value of vertex A in the vertical direction (i.e., the y-axis) and the minimum value of vertex B in the vertical direction in the scaled human body parameterized model is determined as the human body height parameter.

[0137] Still Figure 9 As shown, taking the determination of arm length parameters as an example, the sum of the distance between the shoulder vertex C and the elbow vertex D, and the distance between the elbow vertex D and the wrist vertex E in the scaled human parametric model, is determined as the human arm length parameter.

[0138] Similarly, according to the national standard measurement method, the multiple vertices associated with each part can be used to determine the leg length, neck length, shoulder width, and other parameters.

[0139] It should be noted that the multiple vertices associated with each part can be manually selected or pre-set.

[0140] Method two

[0141] When measuring human body parameters based on the selected multiple vertices, the selected vertices are not uniform and may not be on the same plane, which reduces the accuracy of the calculated vertex-to-vertex distance and results in a large measurement error.

[0142] For example, taking the measurement of the waist circumference as an example, as shown in Figure 10 , it can be seen that the selected vertices are not uniform.

[0143] For example, taking the measurement of the chest circumference, waist circumference, and hip circumference as an example, as shown in Figure 11 , the multiple vertices associated with each of the three parts are not uniform and are not on the same plane.

[0144] To improve the accuracy of human body parameter measurement, in method two, first, determine the plane sheet through which the tangent plane at the standard measurement position point of the scaled human body parameterization model of the preset part passes, and then determine the human body parameter of the preset part according to the three-dimensional coordinates of the vertices included in the plane sheet. For details of the measurement process, see Figure 12 , which mainly includes the following steps:

[0145] S4041: Determine the tangent plane at the standard measurement position point of the scaled human body parameterization model of the preset part, and extract the plane sheet through which the tangent plane passes.

[0146] In S4041, first, determine the standard measurement position point of the preset part to be measured in the scaled human body parameterization model according to the national standard measurement method of the human body parameter, then establish a tangent plane at the standard measurement position point, and extract the plane sheet through which the tangent plane passes. Each plane sheet includes three vertices, and the tangent plane passing through the model plane sheet includes the tangent plane passing through the inside of the plane sheet or the tangent plane passing through one vertex of the plane sheet.

[0147] For example, taking the measurement of the waist circumference as an example, at the standard measurement position point of the waist, the XZ plane of the point is used to tangent the human body parameterization model (i.e., the value of the Y axis is fixed, and the human body parameterization model is tangent by the XZ plane), and the tangent result is as shown in Figure 13 . The plane sheet through which the tangent plane passes is as shown in Figure 14 , which is smoother and more uniform than Figure 11 .

[0148] S4042: respectively determine the intersection point of the tangent of the tangent plane and the line between the two vertices of the face sheet passed through, and determine the length proportion of each intersection point to the corresponding two vertices.

[0149] In S4042, when the tangent plane passes through the inside of the face sheet, the tangent of the tangent plane and the line between the two vertices of the face sheet exist intersection points, as shown in Figure 15 The horizontal line is the tangent, and the round dot is the intersection point of the tangent and the line between the two vertices. One tangent can pass through the inside of multiple face sheets, that is, there are multiple intersection points. For each intersection point, according to the distance between the two vertices of the line where the intersection point is located, the length proportion of the intersection point to the two vertices of the line where the intersection point is located can be determined.

[0150] For example, as shown in Figure 16 The length proportion of the intersection point O to the vertex P and the vertex Q is k: 1-k.

[0151] S4043: for each intersection point, according to the three-dimensional coordinates of the two vertices of the line where the intersection point is located and the length proportion, determine the three-dimensional coordinates of the intersection point.

[0152] In the reconstruction of the three-dimensional model, the three-dimensional coordinates of each vertex included in the model are already determined, so in S4043, for each intersection point, the three-dimensional coordinates of the intersection point can be determined according to the three-dimensional coordinates of the two vertices of the line where the intersection point is located and the length proportion.

[0153] S4044: the vertex of the face sheet passed through by the tangent of the tangent plane is taken as the candidate vertex.

[0154] In S4044, when the tangent plane passes through the vertex included in the face sheet, the vertex of the face sheet passed through by the tangent of the tangent plane is taken as the candidate vertex. Still as shown in Figure 15 The vertex included in the face sheet is represented by a square dot. It can be seen that the tangent also passes through part of the vertex.

[0155] S4045: according to the three-dimensional coordinates of the candidate vertex and the three-dimensional coordinates of each intersection point, determine the human body parameter of the preset part.

[0156] In S4045, according to the three-dimensional coordinates of the candidate vertex and each intersection point, the sum of the distances between all points (including intersection points and candidate vertices) passed through by the tangent is determined, so as to take the sum as the human body parameter of the preset part to be measured.

[0157] S405: send the measurement result of the human body parameter to the client for display.

[0158] In Figure 12In the process of measuring the human body parameter, for each preset part to be measured, the length proportions corresponding to each alternative vertex and each intersection point associated with the preset part are recorded, and subsequent measurement does not need to be calculated again, and the three-dimensional parameter value of the human body can be directly determined through the index of the recorded point.

[0159] In the embodiments of the present application, the vertex index and the length proportion corresponding to each intersection point of the model are unchanged regardless of the change of the human body parameterization model, and therefore, the human body parameter measurement method provided in the embodiments of the present application can be applied to human body parameter measurement in various postures.

[0160] In some embodiments, the human body parameter measurement method can also be based on Figure 1B The system architecture shown in FIG. 1 implements the human body parameter measurement method, in which the front end is only used for image acquisition and data display, and the data processing process is entirely performed by the server, as shown in FIG. 2. Figure 17 The process mainly includes the following steps:

[0161] S1701: The server acquires a frame of human body RGB image and human body depth image synchronously collected by the depth camera.

[0162] S1702: The server determines the first human body bounding box in the human body depth image according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image.

[0163] S1703: The server obtains the human body mask image from the human body depth image according to the first human body bounding box, and extracts the target human body point cloud according to the human body mask image.

[0164] S1704: The server converts the human body RGB image into a target RGB image with a fixed size, and generates a human body parameterization model according to the target RGB image.

[0165] S1705: The server determines the real human body height according to the target human body point cloud, and scales the human body parameterization model according to the real human body height.

[0166] S1706: The server determines the human body parameter of each preset part according to the three-dimensional coordinates of each vertex in the scaled human body parameterization model.

[0167] S1707: The server displays the human body parameter measurement result through the display of the front end.

[0168] In the embodiments of the present application, the vertex index and the length proportion corresponding to each intersection point of the model are unchanged regardless of the change of the human body parameterization model, and therefore, the human body parameter measurement method provided in the embodiments of the present application can be applied to human body parameter measurement in various postures. Figure 17 The specific implementation of each step in FIG. 1 is described in the foregoing embodiments, which will not be repeated here.

[0169] In some embodiments, the human body parameter measurement method can also be based on Figure 1C The system architecture shown in FIG. 1 implements the human body parameter measurement method, in which the front end is only used for image acquisition and data display, and the data processing process is entirely performed by the server, as shown in FIG. 2.Figure 18 and mainly includes the following steps:

[0170] S1801: The client acquires a frame of human RGB image and human depth image synchronously collected by a depth camera.

[0171] S1802: The client determines a first human bounding box in the human depth image according to the mapping relationship of the pixel points in the human RGB image and the human depth image.

[0172] S1803: The client obtains a human mask image from the human depth image according to the first human bounding box, and extracts a target human point cloud according to the human mask image.

[0173] S1804: The client converts the human RGB image into a target RGB image with a fixed size, and generates a human parameterized model according to the target RGB image.

[0174] S1805: The client determines a real human height according to the target human point cloud, and scales the human parameterized model according to the real human height.

[0175] S1806: The client determines human parameters of each preset part according to the three-dimensional coordinates of each vertex in the scaled human parameterized model.

[0176] S1807: The client displays the human parameter measurement result.

[0177] In the method, the steps are implemented as described in the foregoing embodiments, which will not be repeated here. Figure 18

[0178] In the method for measuring human parameters provided by the embodiments of the present application, a frame of human RGB image and human depth image synchronously collected by a depth camera are acquired, a first human bounding box in the human depth image is determined according to the mapping relationship of the pixel points in the human RGB image and the human depth image, and a human mask image is obtained, so that the human in the depth image is accurately segmented, and then a target human point cloud is extracted according to the human mask image. Meanwhile, the human RGB image is converted into a target RGB image with a fixed size, a human parameterized model is generated according to the target RGB image, a real human height is determined according to the target human point cloud, and then the human parameterized model is scaled according to the real human height. The scaled human parameterized model is consistent with the real body shape of the human, so that the scaled human parameterized model is cut according to the preset part to be measured, and the three-dimensional coordinates of the intersection of the tangent line passing through the vertex and the vertex connecting line can be quickly and accurately determined to determine the human parameters of each preset part, and the method is suitable for human parameter measurement in various postures and has higher robustness.

[0179] ​Based on the same technical concept, the embodiment of the present application provides a client device which can implement the steps of the method for measuring the human body parameter of the client in the above-mentioned embodiment and achieve the same technical effects.

[0180] Referring to Figure 19 The client device comprises a processor 1901, a memory 1902, a display screen 1903 and at least one communication interface 1904, the communication interface 1904, the display screen 1903, the memory 1902 and the processor 1901 are connected through a bus 1905, the memory 1902 comprises a data storage unit and a program storage unit, the program storage unit stores a computer program, and the processor 1901 executes the following operations according to the computer program:

[0181] acquire a frame of human body RGB image and human body depth image synchronously collected by the depth camera through the communication interface 1904 and store them to the data storage unit;

[0182] determine a first human body bounding box in the human body depth image according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image;

[0183] obtain a human body mask image from the human body depth image according to the first human body bounding box and extract a target human body point cloud according to the human body mask image;

[0184] send the human body RGB image and the target human body point cloud to a server for human body parameter measurement through the communication interface 1904;

[0185] receive the human body parameter measurement result sent by the server through the communication interface 1904 and display it through the display screen 1903.

[0186] Optionally, the processor 1901 determines the first human body bounding box in the human body depth image according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image, and the specific operation is as follows:

[0187] perform human body recognition on the human body RGB image to obtain a second human body bounding box;

[0188] for each corner point in the second human body bounding box, map the current pixel point in the preset neighborhood of the corner point to the human body depth image according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image;

[0189] if there is a corresponding pixel point of the current pixel point in the human body depth image, take the corresponding pixel point as a corner point of the first human body bounding box.

[0190] Based on the same technical concept, the embodiment of the present application provides a server device, which can implement the steps of the method for measuring human body parameters on the server side in the above-mentioned embodiment and achieve the same technical effects.

[0191] With reference to Figure 20 The server device comprises a processor 2001, a memory 2002 and a communication interface 2003, the communication interface 2003, the memory 2002 and the processor 2001 are connected through a bus 2004, the memory 2002 comprises a data storage unit and a program storage unit, the program storage unit stores a computer program, and the processor 2001 performs the following operations according to the computer program:

[0192] The communication interface 2003 receives the human body RGB image and the target human body point cloud sent by the client and stores them in the data storage unit, wherein the target human body point cloud is extracted from a human body mask image according to a first human body bounding box from a synchronously collected human body depth image, and the first human body bounding box is determined according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image;

[0193] The human body RGB image is converted into a target RGB image with a fixed size, and a human body parameterized model is generated according to the target RGB image;

[0194] The real human body height is determined according to the target human body point cloud, and the human body parameterized model is scaled according to the real human body height;

[0195] The human body parameters of each preset part are determined according to the three-dimensional coordinates of each vertex in the scaled human body parameterized model;

[0196] The communication interface 2003 sends the human body parameter measurement result to the client for display.

[0197] Optionally, the processor 2001 generates a human body parameterized model according to the target RGB image, and the specific operation is as follows:

[0198] The human body shape parameters and the human body posture parameters are extracted according to the target RGB image;

[0199] The standard parameterized model is driven according to the human body shape parameters and the human body posture parameters to obtain a human body parameterized model consistent with the human body posture and the body shape in the target RGB image, wherein the vertices in the standard parameterized model are uniformly down-sampled at least once in the initialization driving process.

[0200] Optionally, the processor 2001 determines the human parameter of each preset part according to the three-dimensional coordinates of each vertex in the scaled human parameterized model, and specifically, the operation is as follows:

[0201] For each preset part, the following operations are performed:

[0202] According to the national standard measurement method, a plurality of vertices associated with the preset part are selected from the scaled human parameterized model, and the human parameter of the preset part is determined according to the three-dimensional coordinates of the selected plurality of vertices; or

[0203] The tangent plane of the preset part at the standard measurement position point of the scaled human parameterized model is determined, and the human parameter of the preset part is determined according to the three-dimensional coordinates of the vertices contained in the face sheet.

[0204] Optionally, the processor 2001 determines the human parameter of the preset part according to the three-dimensional coordinates of each vertex in the scaled human parameterized model, and specifically, the operation is as follows:

[0205] The tangent plane of the preset part at the standard measurement position point of the scaled human parameterized model is determined, and the human parameter of the preset part is determined according to the three-dimensional coordinates of the vertices contained in the face sheet.

[0206] The intersection points of the tangent line of the tangent plane and the line segment connecting the two vertices of the face sheet are determined respectively, and the length proportions of each intersection point to the corresponding two vertices are determined.

[0207] For each intersection point, the three-dimensional coordinates of the intersection point are determined according to the three-dimensional coordinates of the two vertices of the line segment where the intersection point is located and the length proportion.

[0208] The vertices of the face sheet through which the tangent line of the tangent plane passes are taken as the alternative vertices.

[0209] The human parameter of the preset part is determined according to the three-dimensional coordinates of the alternative vertices and the three-dimensional coordinates of each intersection point.

[0210] Based on the same technical concept, the embodiment of the present application provides a human parameter measurement device, which can implement the steps of the human parameter measurement method in the above-mentioned embodiments and achieve the same technical effects.

[0211] Referring to Figure 21The device comprises a processor 2101, a memory 2102, and a communication interface 2103, the communication interface 2103, the memory 2102, and the processor 2101 are connected through a bus 2104, the memory 2102 comprises a data storage unit and a program storage unit, the program storage unit stores a computer program, and the processor 2101 executes the following operations according to the computer program:

[0212] A frame of human body RGB image and human body depth image synchronously collected by the depth camera are acquired through the communication interface 2103 and stored in the data storage unit;

[0213] A first human body bounding box in the human body depth image is determined according to the mapping relationship of the pixel points in the human body RGB image and the human body depth image;

[0214] A human body mask image is obtained from the human body depth image according to the first human body bounding box, and a target human body point cloud is extracted according to the human body mask image;

[0215] The human body RGB image is converted into a target RGB image with a fixed size, and a human body parameterized model is generated according to the target RGB image;

[0216] The real human body height is determined according to the target human body point cloud, and the human body parameterized model is scaled according to the real human body height;

[0217] The human body parameters of each preset part are determined according to the three-dimensional coordinates of each vertex in the scaled human body parameterized model, and the human body parameters are displayed through a display 2105, wherein the display 2105 is a built-in display or an external display.

[0218] Optionally, the processor 2101 generates a human body parameterized model according to the target RGB image, and the specific operation is as follows:

[0219] Human body shape parameters and human body posture parameters are extracted according to the target RGB image;

[0220] A standard parameterized model is driven according to the human body shape parameters and the human body posture parameters, to obtain a human body parameterized model consistent with the human body posture and body shape in the target RGB image, wherein the vertices in the standard parameterized model are uniformly down-sampled at least once in the initialization driving process.

[0221] Optionally, the processor 2101 determines the human body parameters of each preset part according to the three-dimensional coordinates of each vertex in the scaled human body parameterized model, and the specific operation is as follows:

[0222] For each preset part, the following operations are performed:

[0223] According to the national standard measurement method, a plurality of vertices associated with the preset part are selected from the scaled human parameterized model, and a human parameter of the preset part is determined according to three-dimensional coordinates of the selected plurality of vertices; or

[0224] A tangent plane of the preset part at a standard measurement position point of the scaled human parameterized model is determined, and a face sheet through which the tangent plane passes is determined, and a human parameter of the preset part is determined according to three-dimensional coordinates of vertices contained in the face sheet.

[0225] Optionally, the processor 2101 determines a tangent plane of the preset part at a standard measurement position point of the scaled human parameterized model, and a face sheet through which the tangent plane passes is determined, and a human parameter of the preset part is determined according to three-dimensional coordinates of vertices contained in the face sheet, and the specific operation is:

[0226] A tangent plane of the preset part at a standard measurement position point of the scaled human parameterized model is determined, and a face sheet through which the tangent plane passes is determined;

[0227] Intersections of tangent lines of the tangent plane and lines connecting two vertices of the face sheet are determined respectively, and length proportions of each intersection point to the corresponding two vertices are determined;

[0228] For each intersection point, three-dimensional coordinates of the intersection point are determined according to three-dimensional coordinates of two vertices of the line on which the intersection point is located and the length proportion;

[0229] Vertices of the face sheet through which the tangent line of the tangent plane passes are taken as candidate vertices;

[0230] A human parameter of the preset part is determined according to three-dimensional coordinates of the candidate vertices and three-dimensional coordinates of each intersection point.

[0231] Optionally, the processor determines a first human bounding box in the human depth image according to a mapping relationship of pixel points in the human RGB image and the human depth image, and the specific operation is:

[0232] Human body recognition is performed on the human RGB image to obtain a second human bounding box;

[0233] For each of four corner points contained in the second human bounding box, a current pixel point in a preset neighborhood of the corner point is mapped into the human depth image according to a mapping relationship of pixel points in the human RGB image and the human depth image;

[0234] If there is a corresponding pixel point of the current pixel point in the human depth image, the corresponding pixel point is taken as a corner point of the first human bounding box.

[0235] It should be noted that, Figure 19-21As an example, the hardware necessary for implementing the method steps of measuring the human body parameter according to the embodiments of the present application is given.

[0236] The embodiments of the present application Figure 19-21 The processor involved in the embodiments of the present application can be a central processing unit (CPU), a general-purpose processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.

[0237] The embodiments of the present application further provide a computer readable storage medium for storing some instructions, which can complete the method for measuring the human body parameter in the foregoing embodiments when executed.

[0238] The embodiments of the present application further provide a computer program product for storing a computer program, which is used for executing the method for measuring the human body parameter in the foregoing embodiments.

[0239] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0240] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0241] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.

[0242] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.

[0243] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for measuring human body parameters, characterized in that, include: The system receives a human RGB image and a target human point cloud sent by the client. The target human point cloud is extracted from a human mask image obtained from a synchronously acquired human depth image based on a first human bounding box. The first human bounding box is determined based on the mapping relationship between the pixels in the human RGB image and the human depth image. The human body RGB image is converted into a target RGB image of a fixed size, and a human body parametric model is generated based on the target RGB image; Based on the target human body point cloud, the actual human body height is determined, and the human body parameterization model is scaled according to the actual human body height. Based on the three-dimensional coordinates of each vertex in the scaled human body parametric model, the human body parameters of each preset part are determined and sent to the client for display.

2. The method as described in claim 1, characterized in that, The step of generating a human body parameterized model based on the target RGB image includes: Extract human body shape parameters and human body posture parameters from the target RGB image; The standard parametric model is driven based on the human body shape parameters and human body posture parameters to obtain a human body parametric model that is consistent with the human body posture and body shape in the target RGB image. During the initialization driving process, the vertices in the standard parametric model are uniformly downsampled at least once.

3. The method as described in claim 1, characterized in that, The step of determining the human body parameters for each preset part based on the three-dimensional coordinates of each vertex in the scaled human body parameterized model includes: For each preset part, perform the following operations: According to the national standard measurement method, multiple vertices associated with the preset body part are selected from the scaled parametric human body model, and the human body parameters of the preset body part are determined based on the three-dimensional coordinates of the selected multiple vertices; or The planar plane passing through the preset body part at the standard measurement position point of the scaled human body parametric model is determined, and the human body parameters of the preset body part are determined based on the three-dimensional coordinates of the vertices contained in the planar plane.

4. The method as described in claim 1, characterized in that, The step of determining the patch through which the tangent plane of the preset part passes at the standard measurement position point of the scaled human body parametric model intersects, and determining the human body parameters of the preset part based on the three-dimensional coordinates of the vertices contained in the patch, includes: Determine the flat section of the preset part at the standard measurement position point of the scaled human body parameterized model, and extract the surface area through which the flat section passes; Determine the intersection points of the tangent of the flat surface with the line connecting the two vertices of the surface it passes through, and determine the length ratio from each intersection point to the corresponding two vertices; For each intersection point, the three-dimensional coordinates of the intersection point are determined based on the three-dimensional coordinates of the two vertices of the line connecting the intersection point and the length ratio. The vertices of the facets through which the tangent of the flat surface passes are selected as candidate vertices; Based on the three-dimensional coordinates of the candidate vertices and the three-dimensional coordinates of each intersection point, the human body parameters of the preset body part are determined.

5. A method for measuring human body parameters, characterized in that, include: Acquire a single frame of human RGB image and human depth image simultaneously captured by a depth camera; Based on the mapping relationship between pixels in the human body RGB image and the human body depth image, the first human body bounding box in the human body depth image is determined; A human body mask image is obtained from the human body depth image based on the first human body bounding box, and the target human body point cloud is extracted based on the human body mask image; The RGB image of the human body and the point cloud of the target human body are sent to the server for human body parameter measurement. Receive and display the human body parameter measurement results sent by the server.

6. The method as described in claim 5, characterized in that, The step of determining the first human bounding box in the human depth image based on the mapping relationship between pixels in the human RGB image and the human depth image includes: Human body recognition is performed on the RGB image of the human body to obtain a second human body bounding box; For each of the four corner points contained in the second human body bounding box, according to the mapping relationship between the pixels in the human body RGB image and the human body depth image, the current pixel in the preset neighborhood of the corner point is mapped to the human body depth image; If the human body depth image contains a corresponding pixel of the current pixel, then the corresponding pixel is used as a corner point of the first human body bounding box.

7. A method for measuring human body parameters, characterized in that, include: Acquire a single frame of human RGB image and human depth image simultaneously captured by a depth camera; Based on the mapping relationship between pixels in the human body RGB image and the human body depth image, the first human body bounding box in the human body depth image is determined; A human body mask image is obtained from the human body depth image based on the first human body bounding box, and the target human body point cloud is extracted based on the human body mask image; The human body RGB image is converted into a target RGB image of a fixed size, and a human body parametric model is generated based on the target RGB image; Based on the target human body point cloud, the actual human body height is determined, and the human body parameterization model is scaled according to the actual human body height. Based on the 3D coordinates of each vertex in the scaled parametric human body model, the human body parameters of each preset part are determined and displayed on the monitor.

8. A server-side device, characterized in that, The system includes a processor, a memory, and a communication interface. The communication interface, the memory, and the processor are connected via a bus. The memory includes a data storage unit and a program storage unit. The program storage unit stores a computer program. The processor performs the following operations based on the computer program: The communication interface receives and stores the RGB image of the human body and the point cloud of the target human body sent by the client into the data storage unit. The point cloud of the target human body is extracted from the human body mask image obtained from the synchronously acquired human body depth image based on the first human body bounding box. The first human body bounding box is determined based on the mapping relationship of the pixels in the RGB image of the human body and the human body depth image. The human body RGB image is converted into a target RGB image of a fixed size, and a human body parametric model is generated based on the target RGB image; Based on the target human body point cloud, the actual human body height is determined, and the human body parameterization model is scaled according to the actual human body height. Based on the three-dimensional coordinates of each vertex in the scaled human body parametric model, determine the human body parameters of each preset part. The human body parameter measurement results are sent to the client for display via the communication interface.

9. A client device, characterized in that, The system includes a processor, a memory, a display screen, and at least one communication interface. The communication interface, the display screen, the memory, and the processor are connected via a bus. The memory includes a data storage unit and a program storage unit. The program storage unit stores a computer program. The processor performs the following operations according to the computer program: The communication interface is used to acquire a frame of human RGB image and human depth image synchronously captured by the depth camera and store them in the data storage unit. Based on the mapping relationship between pixels in the human body RGB image and the human body depth image, the first human body bounding box in the human body depth image is determined; A human body mask image is obtained from the human body depth image based on the first human body bounding box, and the target human body point cloud is extracted based on the human body mask image; The RGB image of the human body and the point cloud of the target human body are sent to the server through the communication interface for human body parameter measurement. The human body parameter measurement results sent by the server are received through the communication interface and displayed on the display screen.

10. A human body parameter measuring device, characterized in that, The system includes a processor, a memory, and a communication interface. The communication interface, the memory, and the processor are connected via a bus. The memory includes a data storage unit and a program storage unit. The program storage unit stores a computer program. The processor performs the following operations based on the computer program: The communication interface is used to acquire a frame of human RGB image and human depth image synchronously captured by the depth camera and store them in the data storage unit. Based on the mapping relationship between pixels in the human body RGB image and the human body depth image, the first human body bounding box in the human body depth image is determined; A human body mask image is obtained from the human body depth image based on the first human body bounding box, and the target human body point cloud is extracted based on the human body mask image; The human body RGB image is converted into a target RGB image of a fixed size, and a human body parametric model is generated based on the target RGB image; Based on the target human body point cloud, the actual human body height is determined, and the human body parameterization model is scaled according to the actual human body height. Based on the three-dimensional coordinates of each vertex in the scaled human body parametric model, the human body parameters of each preset part are determined and displayed on a monitor, wherein the monitor includes a built-in display screen or an external display screen.

Images