An intelligent body measurement method and device based on deep learning mapping

Abstract

The present invention relates to an intelligent body shape measurement method and device based on deep learning mapping, which includes: (1) video data set production: collecting image data containing a target human body and a reference bottle; (2) reference object detection: applying a deep learning model to identify the reference object in each frame of video and calculating the mapping ratio; (3) key point detection: detecting the key points of the human body and bones in the video through a key point detection algorithm, and then calculating related body parameters; (4) neural network optimization: designing a neural network to optimize and adjust the measurement results to improve measurement accuracy; (5) applying the completed algorithm model to actual measurement scenarios. The present invention has the characteristics of strong robustness and wide applicability, and is suitable for multiple fields such as intelligent body shape measurement, virtual fitting, and health management, and has significant commercial application prospects.

Description

Technical Field

[0001] The present invention relates to the field of computer vision target detection, and in particular to an intelligent body measurement method and device based on deep learning mapping. Background Art

[0002] Existing body measurement technologies have made significant progress in multiple fields, and have been widely used in clothing customization, anthropometrics, and health management. However, traditional body measurement methods mostly rely on manual operations or basic two-dimensional measurements, which have certain limitations. First, manual measurement is not only time-consuming and labor-intensive, but the measurement accuracy is easily affected by the operator's experience and the quality of the tools, resulting in large errors in the measurement results. In addition, traditional body measurement methods cannot obtain data in real time and require a high degree of cooperation from the measured person, causing inconvenience in the user experience. With the rapid development of deep learning technology, it is increasingly necessary to design a more convenient and accurate intelligent body measurement method. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides an intelligent body measurement method and device based on deep learning mapping, which is implemented by target detection and deep learning mapping.

[0004] A first aspect of the present invention relates to an intelligent body measurement method based on deep learning mapping, comprising the following steps:

[0005] (1) Video dataset production: Use a mobile device to shoot around the subject, who can be standing or sitting. Each frame of the video must capture the subject's entire body and the complete reference object;

[0006] (1.1) During the shooting process, ensure that each frame fully covers the entire body of the person being measured and clearly captures the complete outline of the reference object, including but not limited to plastic bottles and cans;

[0007] (1.2) After recording is complete, the videos will be preliminarily screened to remove blurry clips or those that do not meet the shooting requirements, and only the videos that meet the requirements will be retained;

[0008] (1.3) Extract the video set in step (1.2) into an image set {I} by frame, and record the image frame number n.

[0009] (2) Reference object detection: Use the target detection algorithm to process the video data set frame by frame, record the pixel height of the reference object and the corresponding frame number, correct the abnormal frames and fill the undetected frames, and calculate the reference object mapping ratio;

[0010] (2.1) Process the image set {I} generated in step (1.3) frame by frame, detect the reference object using the target detection algorithm, and record the pixel height H p(n), where the target detection algorithm includes but is not limited to the DETR series and the YOLO series; if the bottle cannot be detected, it is recorded as a missing value -1, and all detection values ​​are stored as a height sequence {H p};

[0011] (2.2) Repair and fill in the test results according to the following steps, convert the missing value -1 in the height sequence into None, and convert the remaining values ​​into integers; record the serial number of the frame where the known height value is located {n k} and the corresponding height value If there are missing frames, the missing frames are supplemented, and the supplementation method includes but is not limited to linear interpolation and least squares interpolation;

[0012] (2.3) Perform abnormal smoothing on the detected value. If the height value of a certain frame differs from the height value of the previous frame by more than a set threshold, the value is smoothed. The smoothing method includes but is not limited to Gaussian smoothing and local weighted regression.

[0013] (2.4) The processed reference pixel height values ​​are stored as a new height sequence, and the reference pixel height H of each frame is used in combination with the actual height of the reference object. ′ p (n) and actual height H r , calculate the ratio of pixels to actual length according to formula (2-1):

[0014]

[0015] (3) Key point detection: Use the key point detection algorithm to detect key points of the human body, record the pixel data and frame number of each indicator in each frame, generate the body indicator pixel data file, and generate the body indicator measurement file according to the mapping ratio;

[0016] (3.1) For the image set {I} extracted in step (1.3), perform human key point detection on each frame using a key point detection algorithm. The key point detection algorithm may include but is not limited to YOLO v8-Pose and PoseResNet.

[0017] (3.2) Based on the detection results of (3.1), output the coordinates and key point information of each detection box (including x-coordinate, y-coordinate and confidence level), filter the detection boxes, and retain the results with confidence levels greater than the set threshold;

[0018] (3.3) Calculate the pixel value of the key indicator according to the Euclidean distance formula between two points:

[0019]

[0020] Among them, x1, y1, x2, y2 are the pixel coordinates of the two key points;

[0021] (3.4) Use predefined coefficients to perform weighted correction on the distance value:

[0022] L arm =d·k arm (3-2)

[0023] (3.5) Using the key point pairs defined in the skeleton, calculate the human body measurement indicators of each frame one by one, and map the key point pixel distance to the real distance according to the mapping ratio obtained in (2.4) and formula (3-3):

[0024] D s (n) = d s (n)·r(n) (3-3)

[0025] (3.6) All mapped data are checked for consistency, outliers are corrected, and true measurement data are generated.

[0026] (4) Neural network optimization: design a neural network, self-train the optimization model, input the body index measurement file, optimize each index, and output the final measurement index;

[0027] (4.1) Training neural networks to optimize real measurement data

[0028] (4.1.1) Data preprocessing: Store input data and label data in two directories respectively, integrate multiple frames of measurement data into a single input vector, and extract real body data from the corresponding label file to ensure that the input and output

[0029] The data correspond one to one;

[0030] (4.1.2) Design a neural network with a fully connected structure with hidden layers, including but not limited to BP and CNN.

[0031] Learn the relationship between input data and true values ​​through nonlinear mapping;

[0032] (4.2) Perform training according to the training mode in (4.1), and save the model weight W with the best performance during the training process.

[0033] (4.3) Load the best model weight W saved in (4.2) and set the model to evaluation mode for inference;

[0034] (4.4) Read the input data file and preprocess the data to ensure that the input meets the model requirements;

[0035] (4.5) The processed input data is passed into the loaded model, and the optimized value of each indicator is obtained as the final output.

[0036] A second aspect of the present invention relates to an intelligent body measurement device based on deep learning mapping, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement an intelligent body measurement method based on deep learning mapping of the present invention.

[0037] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements an intelligent body measurement method based on deep learning mapping of the present invention.

[0038] The beneficial effect of the present invention is that, by adopting target detection and key point detection algorithms, combined with the reference object mapping ratio, it is possible to accurately calculate various measurement indicators of the human body, eliminating the errors caused by operator experience and tool differences in the traditional manual measurement process, thereby ensuring the high accuracy of the measurement results. Secondly, the method greatly improves the efficiency of measurement by automatically processing video data without manual intervention, significantly reduces the time and labor intensity required for manual operation, and makes the measurement process more efficient. In addition, the method can obtain and update body measurement data in real time by processing video data frame by frame, avoiding the shortcomings of traditional measurement methods such as the inability to obtain data in real time and the high requirements for the cooperation of the subject, greatly improving the user experience. Finally, based on the optimization of the deep learning model, the method has strong adaptability, can perform accurate measurements based on the physical characteristics of different individuals, and has a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2 It is a video frame image obtained by video splitting using the method of the present invention;

[0041] Figure 3 It is a reference map for detection by the method of the present invention;

[0042] Figure 4 It is a method of the present invention to detect the human body and key point diagram;

[0043] Figure 5 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0044] The core technology of this invention is to realize intelligent body measurement function by using deep learning mapping.

[0045] Example 1

[0046] like Figure 1 The present invention proposes an intelligent body measurement method based on deep learning mapping, comprising the following steps:

[0047] Video dataset production: Use a mobile device to film the subject, who can be standing or sitting. Each frame of the video must capture the subject's entire body and the reference object.

[0048] (1.1) During the shooting process, ensure that each frame fully covers the entire body of the person being measured and clearly captures the complete outline of the reference object, including but not limited to plastic bottles and cans;

[0049] (1.2) After recording is complete, the videos will be preliminarily screened to remove blurry clips or those that do not meet the shooting requirements, and only the videos that meet the requirements will be retained;

[0050] (1.3) Extract the video set in step (1.2) into an image set {I} by frame, and record the image frame number n, such as Figure 2 shown.

[0051] Reference object detection: Use target detection algorithms to process video data sets frame by frame, such as Figure 3 As shown, the pixel height of the reference object and the corresponding frame number are recorded, the abnormal frames are corrected and the undetected frames are filled, and the reference object mapping ratio is calculated;

[0052] (2.1) Process the image set {I} generated in step (1.3) frame by frame, detect the reference object using the target detection algorithm, and record the pixel height H p (n), where the target detection algorithm includes but is not limited to the DETR series and the YOLO series; if the bottle cannot be detected, it is recorded as a missing value -1, and all detection values ​​are stored as a height sequence {H p};

[0053] (2.2) Repair and fill in the test results according to the following steps, convert the missing value -1 in the height sequence into None, and convert the remaining values ​​into integers; record the serial number of the frame where the known height value is located {n k} and the corresponding height value If there are missing frames, the missing frames are supplemented, and the supplementation method includes but is not limited to linear interpolation and least squares interpolation;

[0054] (2.3) Perform abnormal smoothing on the detected value. If the height value of a certain frame differs from the height value of the previous frame by more than a set threshold, the value is smoothed. The smoothing method includes but is not limited to Gaussian smoothing and local weighted regression.

[0055] (2.4) The processed reference pixel height values ​​are stored as a new height sequence, and the reference pixel height H of each frame is used in combination with the actual height of the reference object. ′ p (n) and actual height H r, calculate the ratio of pixels to actual length according to formula (2-1):

[0056]

[0057] Key point detection: Use key point detection algorithm to detect key points of human body, such as Figure 4 As shown, the pixel data and frame number of each indicator of each frame are recorded to generate a body indicator pixel data file, and a body indicator measurement file is generated according to the mapping ratio;

[0058] (3.1) For the image set {I} extracted in step (1.3), perform human key point detection on each frame using a key point detection algorithm. The key point detection algorithm may include but is not limited to YOLO v8-Pose and PoseResNet.

[0059] (3.2) Based on the detection results of (3.1), output the coordinates and key point information of each detection box (including x-coordinate, y-coordinate and confidence level), filter the detection boxes, and retain the results with confidence levels greater than the set threshold;

[0060] (3.3) Calculate the pixel value of the key indicator according to the Euclidean distance formula between two points:

[0061]

[0062] Among them, x1, y1, x2, y2 are the pixel coordinates of the two key points;

[0063] (3.4) Use predefined coefficients to perform weighted correction on the distance value:

[0064] L arm =d·k arm (3-2)

[0065] (3.5) Using the key point pairs defined in the skeleton, calculate the human body measurement indicators of each frame one by one, and map the key point pixel distance to the real distance according to the mapping ratio obtained in (2.4) and formula (3-3):

[0066] D s (n) = d s (n)·r(n) (3-3)

[0067] (3.6) All mapped data are checked for consistency, outliers are corrected, and true measurement data are generated.

[0068] Neural network optimization: design a neural network, self-train the optimization model, input the body index measurement file, optimize each indicator, and output the final measurement indicator;

[0069] (4.1) Training neural networks to optimize real measurement data

[0070] (4.1.1) Data preprocessing: Store input data and label data in two separate directories, integrate multiple frames of measurement data into a single input vector, and extract real body data from the corresponding label files to ensure a one-to-one correspondence between input and output data;

[0071] (4.1.2) Design a neural network using a fully connected structure with hidden layers, including but not limited to BP and CNN, to learn the relationship between input data and true values ​​through nonlinear mapping;

[0072] (4.2) Perform training according to the training mode in (4.1), and save the model weight W with the best performance during the training process.

[0073] (4.3) Load the best model weight W saved in (4.2) and set the model to evaluation mode for inference;

[0074] (4.4) Read the input data file and preprocess the data to ensure that the input meets the model requirements;

[0075] (4.5) The processed input data is passed into the loaded model, and the optimized value of each indicator is obtained as the final output, as shown in Table 1.

[0076] Table 1

[0077] Measurement indicators height Upper arm length Forearm length Long thighs Calf length Head high sit high Measurement value (unit: cm) 168.88 29.93 25.07 37.80 51.26 25.75 43.65

[0078] Example 2

[0079] like Figure 5 This embodiment relates to an intelligent body measurement device based on deep learning mapping, including a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, they are used to implement an intelligent body measurement method based on deep learning mapping of the present invention.

[0080] Example 3

[0081] This embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, an intelligent body measurement method based on deep learning mapping of the present invention is implemented.

[0082] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The scope of protection of the present invention should not be regarded as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. An intelligent body measurement method based on deep learning mapping, characterized in that: The steps include: (1) Video dataset production: Use a mobile device to shoot around the subject, who can be standing or sitting; each frame of the video must capture the subject's entire body and the complete reference object; (2) Reference object detection: Use the target detection algorithm to process the video data set frame by frame, record the pixel height of the reference object and the corresponding frame number, correct the abnormal frames and fill the undetected frames, and calculate the reference object mapping ratio; (3) Key point detection: Use the key point detection algorithm to detect key points of the human body, record the pixel data and frame number of each indicator in each frame, generate the body indicator pixel data file, and generate the body indicator measurement file according to the mapping ratio; (4) Neural network optimization: design a neural network, self-train the optimization model, input the body index measurement file, integrate multiple frames of measurement data into a single input vector, optimize each indicator, and output the final measurement indicator; The video dataset preparation described in step (1) includes the following sub-steps: (1.1) During the shooting process, ensure that each frame fully covers the entire body of the person being measured and clearly captures the complete outline of the reference object, including but not limited to plastic bottles and cans; (1.2) After recording is complete, the videos will be preliminarily screened to remove blurry clips or those that do not meet the shooting requirements, and only the videos that meet the requirements will be retained; (1.3) Extract the video set in step (1.2) into an image set {I} by frame, and record the image frame sequence number n; The reference detection described in step (2) includes the following sub-steps: (2.1) Process the image set {I} generated in step (1.3) frame by frame, detect the reference object using the target detection algorithm, and record the pixel height H p (n), where the target detection algorithm includes but is not limited to the DETR series and the YOLO series; if the bottle cannot be detected, it is recorded as a missing value -1, and all detection values ​​are stored as a height sequence {H p }; (2.2) Repair and fill in the test results according to the following steps, convert the missing value -1 in the height sequence into None, and convert the remaining values ​​into integers; record the serial number of the frame where the known height value is located {n k } and the corresponding height value {Hp k If there are missing frames, the missing frames are supplemented, and the supplementation methods include but are not limited to linear interpolation and least squares interpolation; (2.3) Perform abnormal smoothing on the detected value. If the height value of a certain frame differs from the height value of the previous frame by more than a set threshold, the value is smoothed. The smoothing method includes but is not limited to Gaussian smoothing and local weighted regression. (2.4) The processed reference pixel height values ​​are stored as a new height sequence, and the reference pixel height H of each frame is used in combination with the actual height of the reference object. ′ p (n) and actual height H r , calculate the ratio of pixels to actual length according to formula (2-1): The key point detection described in step (3) includes the following sub-steps: (3.1) For the image set {I} extracted in step (1.3), perform human key point detection on each frame using a key point detection algorithm. The key point detection algorithm may include but is not limited to YOLO v8-Pose and PoseResNet. (3.2) Based on the detection results of step (3.1), output the coordinates and key point information of each detection box, filter the detection boxes, and retain the results with confidence greater than the set threshold; (3.3) Calculate the pixel value of the key indicator according to the Euclidean distance formula between two points: Among them, x1, y1, x2, y2 are the pixel coordinates of the two key points; (3.4) Use predefined coefficients to perform weighted correction on the distance value: L arm =d·k arm (3-2) (3.5) Using the key point pairs defined in the skeleton, calculate the human body measurement indicators of each frame one by one, and map the key point pixel distance to the real distance according to the mapping ratio obtained in step (2.4) and formula (3-3): D s (n)=d s (n)·r(n) (3-3) (3.6) All mapped data are checked for consistency, outliers are corrected, and true measurement data are generated.

2. The intelligent body measurement method based on deep learning mapping according to claim 1, characterized in that: The neural network optimization described in step (4) includes the following sub-steps: (4.1) Training a neural network to optimize real measurement data; (4.1.1) Data preprocessing: Store input data and label data in two separate directories, integrate multiple frames of measurement data into a single input vector, and extract real body data from the corresponding label files to ensure a one-to-one correspondence between input and output data; (4.1.2) Design a neural network using a fully connected structure with hidden layers, including but not limited to BP and CNN, to learn the relationship between input data and true values ​​through nonlinear mapping; (4.2) Training is performed according to the training mode in step (4.1), and the model weight W with the best performance is saved during the training process; (4.3) Load the best model weight W saved in step (4.2) and set the model to evaluation mode for inference; (4.4) Read the input data file and preprocess the data to ensure that the input meets the model requirements; (4.5) The processed input data is passed into the loaded model, and the optimized value of each indicator is obtained as the final output.

3. An intelligent body measurement device based on deep learning mapping, characterized in that: The method comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, the method is used to implement an intelligent body measurement method based on deep learning mapping according to any one of claims 1 to 2.

4. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, an intelligent body measurement method based on deep learning mapping according to any one of claims 1 to 2 is implemented.

Images