Intelligent body measurement method and device based on deep learning mapping

Abstract

The invention relates to an intelligent body type measurement method and device based on deep learning mapping. The method comprises the following steps: (1) making a video data set; collecting image data containing a target human body and a reference substance bottle; (2) reference object detection: identifying a reference object in each frame of video by using a deep learning model, and calculating a mapping proportion; (3) key point detection: human body and skeleton key points in the video are detected through a key point detection algorithm, and then related body parameters are calculated; (4) neural network optimization: designing a neural network to optimize and adjust the measurement result, and improving the measurement precision; and (5) applying the completed algorithm model to an actual measurement scene. The method has the characteristics of high robustness and wide applicability, is suitable for multiple fields such as intelligent body type measurement, virtual fitting, health management and the like, and has a remarkable commercial application prospect.

Description

Technical Field

[0001] The present invention relates to the field of computer vision target detection, and in particular to an intelligent body measurement method and device based on deep learning mapping. Background Art

[0002] Existing body measurement technologies have made significant progress in many fields, especially in clothing customization, anthropometry and health management. However, most traditional body measurement methods rely on manual operation or basic two-dimensional measurement, which has certain limitations. First of all, manual measurement is not only time-consuming and labor-intensive, but also the measurement accuracy is easily affected by the operator's experience and the quality of the tool, resulting in large errors in the measurement results. In addition, traditional body measurement methods cannot obtain data in real time, and require a high degree of cooperation from the measured person, causing inconvenience in user experience. With the rapid development of deep learning technology, it is increasingly necessary to design a more convenient and accurate intelligent body measurement method. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides an intelligent body measurement method and device based on deep learning mapping, which is implemented by target detection and deep learning mapping.

[0004] A first aspect of the present invention relates to an intelligent body measurement method based on deep learning mapping, comprising the following steps:

[0005] (1) Video dataset production: Use a mobile device to shoot around the person being tested, who can be standing or sitting. Each frame of the video must capture the entire body of the person being tested and the complete reference object;

[0006] (1.1) During the shooting process, it is necessary to ensure that each frame completely covers the entire body of the person being measured, and clearly captures the complete outline of the reference object, which includes but is not limited to plastic bottles and cans;

[0007] (1.2) After recording is completed, the videos are preliminarily screened to remove clips with blurry image quality or that do not meet the shooting requirements, and only the video set that meets the requirements is retained;

[0008] (1.3) Extract the video set in step (1.2) into an image set {I} by frame, and record the image frame number n.

[0009] (2) Reference object detection: Use the target detection algorithm to process the video data set frame by frame, record the pixel height and corresponding frame number of the reference object, correct the abnormal frames and fill the undetected frames, and calculate the reference object mapping ratio;

[0010] (2.1) Process the image set {I} generated in step (1.3) frame by frame, detect the reference object using the target detection algorithm, and record the pixel height H p(n), where the target detection algorithms include but are not limited to DETR series and YOLO series; if the bottle cannot be detected, it is recorded as a missing value -1, and all detection values ​​are stored as a height sequence {H p};

[0011] (2.2) Repair and fill in the detection results according to the following steps, convert the missing value -1 in the height sequence into None, and convert the remaining values ​​into integers; record the serial number of the frame where the known height value is located {n k} and the corresponding height value If there are missing frames, the missing frames are supplemented, and the supplementation method includes but is not limited to linear interpolation and least squares interpolation;

[0012] (2.3) performing abnormal smoothing on the detected value. If the height value of a certain frame differs from the height value of the previous frame by more than a set threshold, the value is smoothed. The smoothing method includes but is not limited to Gaussian smoothing and local weighted regression.

[0013] (2.4) The processed reference object pixel height values ​​are stored as a new height sequence, combined with the actual height of the reference object, using the reference object pixel height H of each frame ′ p (n) and actual height H r , calculate the ratio of pixels to actual length according to formula (2-1):

[0014]

[0015] (3) Key point detection: Use the key point detection algorithm to detect the key points of the human body, record the pixel data and frame number of each index in each frame, generate a body index pixel data file, and generate a body index measurement file based on the mapping ratio;

[0016] (3.1) For the image set {I} extracted in step (1.3), human key points are detected for each frame using a key point detection algorithm. The key point detection algorithm includes but is not limited to YOLO v8-Pose and PoseResNet;

[0017] (3.2) Based on the detection results of (3.1), output the coordinates and key point information of each detection box (including x-coordinate, y-coordinate and confidence), filter the detection boxes, and retain the results with confidence greater than the set threshold;

[0018] (3.3) Calculate the pixel value of the key indicator according to the Euclidean distance formula between two points:

[0019]

[0020] Among them, x1, y1, x2, y2 are the pixel coordinates of the two key points;

[0021] (3.4) Use the predefined coefficients to perform weighted correction on the distance value:

[0022] L arm =d·k arm (3-2)

[0023] (3.5) Using the key point pairs defined in the skeleton, calculate the human body measurement indicators of each frame one by one, and map the key point pixel distance to the real distance according to the mapping ratio obtained in (2.4) and formula (3-3):

[0024] D s (n) = d s (n)·r(n) (3-3)

[0025] (3.6) All mapped data are checked for consistency, outliers are corrected, and true measurement data are generated.

[0026] (4) Neural network optimization: design a neural network, self-train the optimization model, input the physical index measurement file, optimize each index, and output the final measurement index;

[0027] (4.1) Training neural network to optimize real measurement data

[0028] (4.1.1) Data preprocessing: Store input data and label data in two directories respectively, integrate multiple frames of measurement data into a single input vector, and extract real body data from the corresponding label file to ensure that the input and output

[0029] The data correspond one to one;

[0030] (4.1.2) Design a neural network with a fully connected structure with hidden layers, including but not limited to BP and CNN.

[0031] Learn the relationship between input data and true values ​​through nonlinear mapping;

[0032] (4.2) Perform training according to the training mode in (4.1) and save the model weight W with the best performance during the training process.

[0033] (4.3) Load the best model weight W saved in (4.2) and set the model to evaluation mode for inference;

[0034] (4.4) Read the input data file and preprocess the data to ensure that the input meets the model requirements;

[0035] (4.5) The processed input data is passed into the loaded model, and the optimized value of each indicator is obtained as the final output.

[0036] A second aspect of the present invention relates to an intelligent body measurement device based on deep learning mapping, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement an intelligent body measurement method based on deep learning mapping of the present invention.

[0037] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements an intelligent body measurement method based on deep learning mapping of the present invention.

[0038] The beneficial effect of the present invention is that, by adopting the target detection and key point detection algorithms, combined with the reference object mapping ratio, various measurement indicators of the human body can be accurately calculated, eliminating the errors caused by the operator's experience and tool differences in the traditional manual measurement process, thereby ensuring the high accuracy of the measurement results. Secondly, the method greatly improves the efficiency of body measurement by automatically processing video data without manual intervention, significantly reduces the time and labor intensity required for manual operation, and makes the measurement process more efficient. In addition, the method can obtain and update body measurement data in real time by processing video data frame by frame, avoiding the shortcomings of the traditional body measurement method that cannot obtain data in real time and the high requirements for the cooperation of the measured person, greatly improving the user experience. Finally, based on the optimization of the deep learning model, the method has strong adaptability, can accurately measure according to the physical characteristics of different individuals, and has a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2 It is a video frame image of video splitting by the method of the present invention;

[0041] Figure 3 It is a reference object diagram detected by the method of the present invention;

[0042] Figure 4 It is a method of the present invention to detect a human body and a key point diagram;

[0043] Figure 5 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0044] The core technology of the present invention is to realize intelligent body measurement function by using deep learning mapping.

[0045] Example 1

[0046] like Figure 1 The present invention proposes an intelligent volume measurement method based on deep learning mapping, comprising the following steps:

[0047] Video dataset production: Use a mobile device to shoot around the person being tested, who can be standing or sitting. Each frame of the video must capture the entire body of the person being tested and the complete reference object;

[0048] (1.1) During the shooting process, it is necessary to ensure that each frame completely covers the entire body of the person being measured, and clearly captures the complete outline of the reference object, which includes but is not limited to plastic bottles and cans;

[0049] (1.2) After recording is completed, the videos are preliminarily screened to remove clips with blurry image quality or that do not meet the shooting requirements, and only the video set that meets the requirements is retained;

[0050] (1.3) Extract the video set in step (1.2) into an image set {I} by frame, and record the image frame number n, such as Figure 2 shown.

[0051] Reference object detection: Use the target detection algorithm to process the video data set frame by frame, such as Figure 3 As shown, the pixel height of the reference object and the corresponding frame number are recorded, the abnormal frames are corrected and the undetected frames are filled, and the reference object mapping ratio is calculated;

[0052] (2.1) Process the image set {I} generated in step (1.3) frame by frame, detect the reference object using the target detection algorithm, and record the pixel height H p (n), where the target detection algorithms include but are not limited to DETR series and YOLO series; if the bottle cannot be detected, it is recorded as a missing value -1, and all detection values ​​are stored as a height sequence {H p};

[0053] (2.2) Repair and fill in the detection results according to the following steps, convert the missing value -1 in the height sequence into None, and convert the remaining values ​​into integers; record the serial number of the frame where the known height value is located {n k} and the corresponding height value If there are missing frames, the missing frames are supplemented, and the supplementation method includes but is not limited to linear interpolation and least squares interpolation;

[0054] (2.3) performing abnormal smoothing on the detected value. If the height value of a certain frame differs from the height value of the previous frame by more than a set threshold, the value is smoothed. The smoothing method includes but is not limited to Gaussian smoothing and local weighted regression.

[0055] (2.4) The processed reference object pixel height values ​​are stored as a new height sequence, combined with the actual height of the reference object, using the reference object pixel height H of each frame ′ p (n) and actual height H r, calculate the ratio of pixels to actual length according to formula (2-1):

[0056]

[0057] Key point detection: Use key point detection algorithm to detect key points of human body, such as Figure 4 As shown, the pixel data and frame number of each index of each frame are recorded, a body index pixel data file is generated, and a body index measurement file is generated according to the mapping ratio;

[0058] (3.1) For the image set {I} extracted in step (1.3), human key points are detected for each frame using a key point detection algorithm. The key point detection algorithm includes but is not limited to YOLO v8-Pose and PoseResNet;

[0059] (3.2) Based on the detection results of (3.1), output the coordinates and key point information of each detection box (including x-coordinate, y-coordinate and confidence), filter the detection boxes, and retain the results with confidence greater than the set threshold;

[0060] (3.3) Calculate the pixel value of the key indicator according to the Euclidean distance formula between two points:

[0061]

[0062] Among them, x1, y1, x2, y2 are the pixel coordinates of the two key points;

[0063] (3.4) Use the predefined coefficients to perform weighted correction on the distance value:

[0064] L arm =d·k arm (3-2)

[0065] (3.5) Using the key point pairs defined in the skeleton, calculate the human body measurement indicators of each frame one by one, and map the key point pixel distance to the real distance according to the mapping ratio obtained in (2.4) and formula (3-3):

[0066] D s (n) = d s (n)·r(n) (3-3)

[0067] (3.6) All mapped data are checked for consistency, outliers are corrected, and true measurement data are generated.

[0068] Neural network optimization: design a neural network, self-train the optimization model, input the physical index measurement file, optimize each index, and output the final measurement index;

[0069] (4.1) Training neural network to optimize real measurement data

[0070] (4.1.1) Data preprocessing: Store input data and label data in two directories respectively, integrate multiple frames of measurement data into a single input vector, and extract real body data from the corresponding label file to ensure one-to-one correspondence between input and output data;

[0071] (4.1.2) Design a neural network with a fully connected structure with hidden layers, including but not limited to BP and CNN, to learn the relationship between input data and true values ​​through nonlinear mapping;

[0072] (4.2) Perform training according to the training mode in (4.1) and save the model weight W with the best performance during the training process.

[0073] (4.3) Load the best model weight W saved in (4.2) and set the model to evaluation mode for inference;

[0074] (4.4) Read the input data file and preprocess the data to ensure that the input meets the model requirements;

[0075] (4.5) The processed input data is passed into the loaded model, and the optimized value of each indicator is obtained as the final output, as shown in Table 1.

[0076] Table 1

[0077] Measuring indicators height Upper arm length Forearm length Long thighs Calf length Head height sit high Measurement value (unit: cm) 168.88 29.93 25.07 37.80 51.26 25.75 43.65

[0078] Example 2

[0079] like Figure 5 This embodiment relates to an intelligent body measurement device based on deep learning mapping, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement an intelligent body measurement method based on deep learning mapping of the present invention.

[0080] Example 3

[0081] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, an intelligent body measurement method based on deep learning mapping of the present invention is implemented.

[0082] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to the equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. An intelligent volume measurement method based on deep learning mapping, characterized in that: The steps include: (1) Video dataset production: Use a mobile device to shoot around the person being tested, who can be standing or sitting. Each frame of the video must capture the entire body of the person being tested and the complete reference object; (2) Reference object detection: Use the target detection algorithm to process the video data set frame by frame, record the pixel height and corresponding frame number of the reference object, correct the abnormal frames and fill the undetected frames, and calculate the reference object mapping ratio; (3) Key point detection: Use the key point detection algorithm to detect the key points of the human body, record the pixel data and frame number of each index in each frame, generate a body index pixel data file, and generate a body index measurement file based on the mapping ratio; (4) Neural network optimization: Design a neural network, self-train the optimization model, input the physical index measurement file, optimize each index, and output the final measurement index.

2. The intelligent body measurement method based on deep learning mapping according to claim 1, characterized in that: The video data set preparation described in step (1) includes the following sub-steps: (1.1) During the shooting process, it is necessary to ensure that each frame completely covers the entire body of the person being measured, and clearly captures the complete outline of the reference object, which includes but is not limited to plastic bottles and cans; (1.2) After recording is completed, the videos are preliminarily screened to remove clips with blurry image quality or that do not meet the shooting requirements, and only the video set that meets the requirements is retained; (1.3) Extract the video set in step (1.2) into an image set {I} by frame, and record the image frame number n.

3. The intelligent body measurement method based on deep learning mapping according to claim 1, characterized in that: The reference object detection described in step (2) includes the following sub-steps: (2.1) Process the image set {I} generated in step (1.3) frame by frame, detect the reference object using the target detection algorithm, and record the pixel height H p (n), where the target detection algorithms include but are not limited to DETR series and YOLO series; if the bottle cannot be detected, it is recorded as a missing value -1, and all detection values ​​are stored as a height sequence {H p }; (2.2) Repair and fill in the detection results according to the following steps, convert the missing value -1 in the height sequence into None, and convert the remaining values ​​into integers; record the serial number of the frame where the known height value is located {n k } and the corresponding height value {H pk }; If there are missing frames, the missing frames are supplemented, and the supplementation method includes but is not limited to linear interpolation and least squares interpolation; (2.3) performing abnormal smoothing on the detected value. If the height value of a certain frame differs from the height value of the previous frame by more than a set threshold, the value is smoothed. The smoothing method includes but is not limited to Gaussian smoothing and local weighted regression. (2.4) The processed reference object pixel height values ​​are stored as a new height sequence, combined with the actual height of the reference object, using the reference object pixel height H of each frame ′ p (n) and actual height H r , calculate the ratio of pixels to actual length according to formula (2-1):

4. The intelligent body measurement method based on deep learning mapping according to claim 1, characterized in that: The key point detection described in step (3) includes the following sub-steps: (3.1) For the image set {I} extracted in step (1.3), human key points are detected for each frame using a key point detection algorithm. The key point detection algorithm includes but is not limited to YOLO v8-Pose and PoseResNet; (3.2) Based on the detection results of step (3.1), output the coordinates and key point information of each detection box, filter the detection boxes, and retain the results with confidence greater than the set threshold; (3.3) Calculate the pixel value of the key indicator according to the Euclidean distance formula between two points: Among them, x1, y1, x2, y2 are the pixel coordinates of the two key points; (3.4) Use the predefined coefficients to perform weighted correction on the distance value: L arm =d·k arm (3-2) (3.5) Using the key point pairs defined in the skeleton, calculate the human body measurement indicators of each frame one by one, and map the key point pixel distance to the real distance according to the mapping ratio obtained in step (2.4) and formula (3-3): D s (n)=d s (n)·r(n) (3-3) (3.6) All mapped data are checked for consistency, outliers are corrected, and true measurement data are generated.

5. The intelligent body measurement method based on deep learning mapping according to claim 1, characterized in that: The neural network optimization described in step (4) includes the following sub-steps: (4.1) Training a neural network to optimize real measurement data; (4.1.1) Data preprocessing: Store input data and label data in two directories respectively, integrate multiple frames of measurement data into a single input vector, and extract real body data from the corresponding label file to ensure one-to-one correspondence between input and output data; (4.1.2) Design a neural network with a fully connected structure with hidden layers, including but not limited to BP and CNN, to learn the relationship between input data and true values ​​through nonlinear mapping; (4.2) Perform training according to the training mode in step (4.1), and save the model weight W with the best performance during the training process; (4.3) Load the best model weight W saved in step (4.2) and set the model to evaluation mode for inference; (4.4) Read the input data file and preprocess the data to ensure that the input meets the model requirements; (4.5) The processed input data is passed into the loaded model, and the optimized value of each indicator is obtained as the final output.

6. An intelligent body measurement device based on deep learning mapping, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement an intelligent body measurement method based on deep learning mapping as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, an intelligent body measurement method based on deep learning mapping as described in any one of claims 1 to 5 is implemented.

Images