Human parameter measurement method based on attention mechanism and corner-based stereo matching

Abstract

The application provides a human parameter measurement method fusing an attention mechanism and a corner point-based stereo matching, and the steps are as follows: a rotating table is rotated at intervals of 90 degrees, and a binocular camera collects four groups of left views and right views of human images at different angles; each group of left views and right views is respectively input into a CBAM-PSPNet semantic segmentation network to extract a human measurement part; a feature point set of the human measurement part is extracted by using a Shi-Tomasi corner point detection algorithm; the feature point set is regionally constrained according to color marker points and a checkerboard system, and the same row feature point two-dimensional coordinates of the left views and the right views are obtained; the three-dimensional space coordinates of the same row feature points are calculated by using binocular stereo calibration parameters; the feature points are reversely rotated back to the initial positions according to the rotating angle of the rotating table, a human girth fitting is realized by using a PIVCF method, and human parameter measurement data are obtained. The application improves the human segmentation precision, simplifies the stereo matching process, and effectively improves the human measurement precision.

Description

Technical Field

[0001] This invention relates to the technical field of human body parameter measurement, and in particular to a human body parameter measurement method that integrates attention mechanism and corner point stereo matching. Background Technology

[0002] Anthropometry data is fundamental to national production and development, playing a crucial role in clothing design, health assessment, and industrial design to ensure a healthy and comfortable user experience. Manual anthropometry relies heavily on the measurer's experience, resulting in fluctuating accuracy and efficiency depending on the measurer. With advancements in information processing technology, 3D human body scanners, such as 3D laser scanners and structured light scanners, have significantly improved the accuracy and efficiency of anthropometry. However, these devices typically extract anthropometry data from thousands of scans, requiring substantial data storage and computation, hindering their widespread application. As device complexity and data volume decrease, the application of optical cameras in anthropometry is gaining increasing attention. Anthropometry devices with optical cameras acquire optical images of the human body and obtain anthropometry data through deep learning-based image processing techniques. Anthropometry methods based on 2D image processing generally obtain intermediate measurement data through 2D image processing and predict anthropometry data by substituting the measurement data into a constructed mathematical equation for the human body. However, due to the lack of three-dimensional spatial information, the measurement accuracy of these 2D image processing-based anthropometry methods is relatively low. Measurement methods based on 3D model reconstruction reconstruct a 3D human body model from point cloud data obtained through multi-view image processing, thereby completing human body measurements. Although these 3D model reconstruction-based human body measurement methods have high measurement accuracy, the process of reconstructing a 3D human body model from multiple images is very complex and time-consuming. Existing human circumference measurement methods based on multi-view vision systems strike a balance between the accuracy and complexity of the two aforementioned human body measurement methods using optical cameras. By combining semantic segmentation and stereo matching, they achieve simple, accurate, and intelligent human body parameter measurement.

[0003] Patent application number 202010441103.7 discloses a method for measuring human circumference based on a multi-view vision system. The steps are as follows: A GSS-PSPNet network model is constructed and trained using a dataset; three sets of binocular stereo vision systems are built to capture color images of the subject; the region to be measured in the color images is segmented; HSV color space analysis is performed on the segmented images to obtain subsets of different color categories; pixels in the subsets are clustered; the SURF method is used to perform stereo matching on the segmented images to obtain the coordinates of the same marker point in the left and right views; the spatial coordinates of the marker points are obtained, resulting in the three-dimensional coordinates of the marker points in a unified coordinate system; all marker points are projected onto the XOZ plane and fitted, and the length of the fitted curve is the circumference of the region being measured. This invention has a simple structure, effectively eliminates measurement errors caused by the movement of the object being measured, greatly simplifies the stereo matching process, and effectively improves matching accuracy. However, the accuracy and efficiency of its human semantic segmentation network still need further optimization, and the accuracy of human parameter measurement needs to be further improved through better feature point design methods. Summary of the Invention

[0004] To address the technical problems of low accuracy and high complexity in existing multi-view vision system methods for measuring human anatomy, this invention proposes a human parameter measurement method that integrates attention mechanisms and corner point stereo matching, thereby further improving the accuracy of human anatomy measurement.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: a method for measuring human body parameters that integrates attention mechanisms and corner-point stereo matching, comprising the following steps:

[0006] Step 1: Using a binocular camera and a turntable, the turntable rotates at 90-degree intervals, and the binocular camera captures four sets of human body images from different angles, including left and right views.

[0007] Step 2: Input the left and right views of each group into the CBAM-PSPNet semantic segmentation network to extract human body measurement parts;

[0008] Step 3: Use the Shi-Tomasi corner detection algorithm to extract the sub-pixel level feature point set of the human body measurement area;

[0009] Step 4: Based on the preset color marker position information and checkerboard pattern, the feature point set is constrained regionally to achieve fine stereo matching at the sub-pixel level and obtain the two-dimensional coordinates of the feature points in the same row of the left and right views.

[0010] Step 5: Calculate the 3D spatial coordinates of feature points in the same row using the binocular stereo calibration parameters;

[0011] Step 6: Rotate the feature points back to their initial positions according to the rotation angle of the turntable, and use the PIVCF method to fit the human body circumference to obtain human body parameter measurement data.

[0012] Preferably, the binocular camera captures human images of the subject wearing a black and white checkered bodysuit, with an adjacent checkerboard spacing of 2.5 cm. The calibrated binocular camera is placed directly in front of a turntable, with the subject standing in the center. The turntable rotates 90° clockwise four times every 7.5 seconds back to its starting position. The binocular camera captures human images at 0°, 90°, 180°, and 270°. A colored marker is affixed to the center point of the turntable. The dataset used for training and testing the CBAM-PSPNet semantic segmentation network includes: 2500 human images with a resolution of 300×600 selected from the public dataset "Look into person"; 2000 human images with a resolution of 2448×2048 captured by a camera; the dataset is expanded to 20308 images, with 15795 human images selected as the training set and 4513 human images as the test set.

[0013] Preferably, the CBAM-PSPNet semantic segmentation network embeds four CBAM attention modules in the feature extraction stage of the ResNet101 backbone network of the PSPNet network; in the bottleneck structure of the ResNet101 backbone network, ordinary convolution is replaced with grouped convolution; the CBAM-PSPNet semantic segmentation network is used to perform semantic segmentation of six regions of the human body: head, chest, waist, hips, thighs and feet in the left and right views.

[0014] Preferably, the intermediate convolutional part of the residual network of the backbone network ResNet101 is composed of residual blocks, and one CBAM module is embedded in the output part of each of the four layers of the intermediate convolutional part.

[0015] The implementation method of the grouped convolution is as follows: For a 256-channel input feature, it is divided into two branches, namely a straight branch and a shortcut branch. Grouped convolution is performed at the second 3×3 convolution layer of the straight branch, and then the outputs of each grouped convolution are concatenated by channel stacking. The number of parameters in the grouped convolution is the same as that in a regular convolution. Where g is the number of groups for each convolutional kernel in the grouped convolutional layer.

[0016] Preferably, the CBAM-PSPNet semantic segmentation network uses an improved backbone network ResNet101 with four CBAM modules embedded as feature extractors to extract features from the input image and generate feature maps. Then, a pyramid pooling module is used to extract contextual information from the generated feature maps. The extracted feature maps are fused into global features. The fused global features are then concatenated with the original input feature maps. The pyramid pooling module has four levels: 1×1, 2×2, 3×3, and 6×6, which are used to extract global features and local features of different scales.

[0017] Preferably, the Shi-Tomasi corner detection algorithm achieves accurate positioning of the inner corner point based on the change of gray value in the neighborhood of the corner point; the method for setting the preset color marker points is as follows: red and cyan marker points are set in each 1 / 4 segmented circumference area, and the rectangular area determined by the two marker points contains the same circumference measurement line.

[0018] Preferably, the method for defining the region is as follows:

[0019] The Shi-Tomasi corner detection algorithm accurately locates feature points in the human body segmentation regions of the left and right views, obtaining sub-pixel corner set sets for the segmented regions of the left and right views respectively. and Where, x l_i y l_i These represent the x and y coordinates of the corner pixels within the segmented area of ​​the left view, respectively; x r_j y r_j These represent the x and y coordinates of the corner pixels within the segmented region of the right view, respectively; M and N represent the number of corner pixels detected within the segmented regions of the left and right views, respectively.

[0020] Obtain the set S of corner points S in the left and right views of the same row using the marker point constraint. l_RC and S r_RC ,in, and

[0021] For the set of corner points S of the constrained region l_RC and S r_RC S is obtained by sorting the x-coordinates. l_RC 'and S r_RC This allows for a one-to-one correspondence between the pixel coordinates of the same corner point in the left and right views, resulting in matching point pairs of the same row of feature points in the left and right views, thus achieving sub-pixel level fine stereo matching.

[0022] Based on the characteristics of the chessboard, and using the center coordinates of the red and cyan markers as constraints, the steps are adjusted according to the y-coordinates of the markers with a step size N. stepMove up or down, where N step The difference in the number of pixels between adjacent chessboard corner points; and the shooting distance D and the step size N step The relationship is: N step =7.02D 2 -45.18D+93.43.

[0023] Preferably, the method for implementing the marker point constraint is as follows:

[0024] All pixels in the segmented image constitute the dataset Z = {z1, z2, ... z} i ...}, i = 1, 2, ... N, where each pixel z... i In the HSV color space, the component is z. i (H i ,S i V i The two-dimensional coordinates of the segmented image are z. i (x i ,y i );

[0025] Based on the HSV color range, the pixel set M of the red and cyan markers in the left view is obtained for each segmented region. l_R M l_C The set of pixels M marked with red and cyan colors in the right view r_R M r_C The center point coordinates of the red and cyan markers within the segmented area of ​​the left view are obtained by averaging the pixel coordinates within the pixel set. The coordinates of the center points of the red and cyan markers within the area divided by the right view

[0026] Mark points and As constraints, obtain the set of corner pixel points within the constraint box defined by the red and cyan markers in the left view segmentation region:

[0027]

[0028] in, and M1 represents the number of corner points within the left view constraint box;

[0029] Mark points and As constraints, obtain the set of corner pixel points within the constraint box defined by the red and cyan markers in the right view segmentation region:

[0030]

[0031] in, and N1 represents the number of corner points within the constraint box of the right view, and N1 = M1.

[0032] Preferably, the marker point and The ordinate is expressed with a step size N step Progressing upwards or downwards, we obtain... and Complete the accurate matching of pixel information across multiple rows in the segmented region.

[0033] Preferably, the implementation method of the PIVCF method is as follows: the three-dimensional coordinates of the rotated feature point are rotated back to the initial position, and the rotation axis is the Y-axis perpendicular to the turntable center point O(x0,y0,z0); the contour map of the fitted gravitational dimension is projected onto the XOZ plane, and the feature point (x,y,z) is rotated back to the initial position (x′,y′,z′) according to the turntable rotation angle θ and the turntable center point O(x0,y0,z0):

[0034]

[0035] The maximum circumference data is used as the final circumference output for the bust, hips, and upper thigh, while the minimum circumference data is used as the final circumference output for the waist. Based on the rows containing the bust and upper thigh circumference, two rows of checkerboard patterns are moved down, and the data from the third row is used as the final circumference output for the underbust and mid-thigh circumference.

[0036] Compared with existing technologies, the beneficial effects of this invention are as follows: Using a set of binocular cameras and a precision turntable, rotating at 90-degree intervals, four sets of human body images from different angles are acquired; the human body measurement parts are extracted from each set of left and right views using the CBAM-PSPNet semantic segmentation network; the Shi-Tomasi corner detection algorithm is used to extract the sub-pixel level feature point set of the human body measurement parts; the feature point set is constrained according to preset color marker point position information and a checkerboard pattern to obtain the two-dimensional planar coordinates of the same row of feature points in the left and right views; the three-dimensional spatial coordinates of the same row of feature points are calculated using binocular stereo calibration parameters; the feature points are rotated back to their initial positions according to the turntable rotation angle, and the human body circumference is fitted using the PIVCF method to calculate and output human body parameter measurement data. The acquisition system designed in this invention, using a combination of binocular cameras and a turntable, has a simple structure, low cost, and convenient operation. The human body segmentation algorithm with an attention mechanism and the feature point measurement method based on corner sub-pixel stereo matching proposed in this invention improve the accuracy of human body segmentation, simplify the stereo matching process, and effectively improve the accuracy of human body parameter measurement. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of the present invention;

[0039] Figure 2 The human body image acquisition system of the present invention includes (a) a three-dimensional spatial view and (b) a top view;

[0040] Figure 3 This is a structural diagram of the backbone network in the CBAM-PSPNet network of the present invention;

[0041] Figure 4 The image shows a comparison of the feature visualization results of the CBAM-PSPNet backbone network of the present invention and the original PSPNet backbone network, where (a) is the visualization result of the original PSPNet backbone network ResNet101, and (b) is the visualization result of the CBAM-PSPNet backbone network ResNet101+CBAM.

[0042] Figure 5 This is a schematic diagram of the improved ResNet101 bottleneck structure of the CBAM-PSPNet network of the present invention;

[0043] Figure 6 This is a schematic diagram comparing the parameters of grouped convolution and ordinary convolution according to the present invention, wherein (a) is ordinary convolution and (b) is grouped convolution;

[0044] Figure 7 This is a flowchart of the CBAM-PSPNet network of the present invention;

[0045] Figure 8 This is a model architecture diagram of the CBAM-PSPNet network of the present invention;

[0046] Figure 9 This is a magnified view of a corner point of the checkerboard pattern of the present invention;

[0047] Figure 10 This is a schematic diagram of the corner detection results of the present invention;

[0048] Figure 11 This is a schematic diagram of the pre-set color marker scheme of the present invention, wherein (a) is a left view and (b) is a right view;

[0049] Figure 12This is a schematic diagram of the marker point constraint matching of the present invention, wherein (a) is the left view and (b) is the right view.

[0050] Figure 13 The graph shows the curve fitting results for a female subject in this invention, where (a) is the bust circumference, (b) is the underbust circumference, (c) is the waist circumference, (d) is the hip circumference, (e) is the upper thigh circumference, and (f) is the mid-thigh circumference.

[0051] Figure 14 The graph shows the curve fitting results for a male subject in this invention, where (a) is the chest circumference, (b) is the underchest circumference, (c) is the waist circumference, (d) is the hip circumference, (e) is the upper thigh circumference, and (f) is the mid-thigh circumference. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, the present invention provides a method for measuring human body parameters that integrates attention mechanisms and corner-point stereo matching. The method mainly includes human body part segmentation optimization, feature point extraction and stereo matching optimization, and circumference curve fitting. The steps are as follows:

[0054] Step 1: Using a set of binocular cameras and a turntable, rotate the camera at 90-degree intervals to acquire four sets of human images from different angles.

[0055] like Figure 2As shown, this invention uses two Hikvision MV-CA050-11UC industrial cameras and a precision turntable to form a human image acquisition system to collect human image data. The cameras use an IMX264 sensor with a resolution of 2448×2048. The human image acquisition system also includes a turntable supporting 360-degree rotation without blind spots, and a PC equipped with an Intel i7-10750H CPU, 16GB RAM, and an Nvidia RTX2060 graphics card with 6GB of video memory. The PC communicates with the cameras via a USB 3.0 interface. A new checkerboard corner point design was used, and the subject wore a black and white checkerboard bodysuit for measurement, with an adjacent checkerboard spacing of 2.5cm. The calibrated binocular camera was placed directly in front of the turntable. The subject stood in the center of the turntable. The turntable was connected to the PC via a drive motor. The PC controlled the turntable to rotate 90° clockwise every 7.5 seconds, rotating 4 times to return to the starting position. The turntable acquired binocular image data every 90 degrees, and acquired images at 0 degrees, 90 degrees, 180 degrees, and 270 degrees, for a total of four sets of human images.

[0056] Step 2: Input each set of left and right views into the CBAM-PSPNet semantic segmentation network to extract human body measurement parts.

[0057] The CBAM-PSPNet semantic segmentation network incorporates an attention mechanism for human body segmentation in human images. This invention embeds four CBAM attention modules into the feature extraction stage of the ResNet101 backbone of the PSPNet network, and one CBAM module into the output of each of the four intermediate convolutional layers. This achieves better spatial and channel feature fusion, enhances model generalization ability, and strengthens semantic information, optimizing the extraction of human contours and semantic information for refined human body segmentation. Furthermore, in the bottleneck structure of the ResNet101 backbone, ordinary convolutions are replaced with grouped convolutions to achieve high-efficiency human body segmentation. The CBAM-PSPNet network model is trained using the constructed dataset. Figure 3 The diagram shows the structure of the backbone network in the CBAM-PSPNet semantic segmentation network of this invention. The residual network of the backbone network ResNet101 includes three parts: an input part, an intermediate convolutional part (Layer 1-4), and an output part. The intermediate convolutional part is composed of residual blocks. Figure 3 This shows the embedding positions of the CBAM attention module in the intermediate convolutional layers of the ResNet101 backbone network, with the embedding positions set in the output portion of each layer. For example... Figure 4The image shows a comparison of the feature visualization results between the backbone network of the semantic segmentation network CBAM-PSPNet of this invention and the original PSPNet backbone network. The comparison mainly focuses on six feature maps in the feature extraction stage, corresponding to Conv1, MaxPool, and Layers 1-4 of the ResNet101 backbone network. (a) is the visualization result of the original PSPNet backbone network ResNet101, and (b) is the visualization result of the CBAM-PSPNet semantic segmentation network backbone network ResNet101+CBAM. Through visual comparison, during feature extraction in Conv1, MaxPool, Layer 1, and Layer 2, the CBAM-PSPNet backbone network demonstrates significantly better performance in extracting low-level edge information, particularly in human contour extraction. During feature extraction in Layers 3 and 4, it can be observed that the CBAM-PSPNet backbone network acquires richer high-level semantic information.

[0058] To reduce the computational cost of the network model, this invention replaces ordinary convolutions in the ResNet101 residual blocks of the backbone network with grouped convolutions. Specific improvements are as follows: Figure 5 As shown. First, for a 256-channel input feature, it is divided into two branches: a straight branch and a shortcut branch. At the second layer of the 3×3 convolution in the straight branch, a group convolution operation is performed. Four branch convolutions are used to further reduce the network model parameters. Then, the outputs of each group convolution are concatenated and connected by channel stacking.

[0059] Figure 6 The parameters of grouped convolution and ordinary convolution are compared. Figure 6 (a) is a standard convolution operation. Assuming the input feature map size is H×W×c1, there are c2 convolution kernels of size h1×w1×c1. After the convolution operation, the output is H×W×c2. The number of parameters P1 in a standard convolutional layer is:

[0060] P1=h1×w1×c1×c2 (1)

[0061] Figure 6 (b) Represents grouped convolution operation. Assuming the input feature map is divided into g groups, the corresponding convolution kernels are also divided into g groups. The convolution operation is performed within each group, and the parameter count P2 of the grouped convolutional layer is:

[0062]

[0063] Therefore, the number of parameters in grouped convolution is the same as that in ordinary convolution. The model parameters can be further compressed to improve segmentation efficiency.

[0064] Figure 7 The flowchart of the entire CBAM-PSPNet network is shown. First, the improved backbone network ResNet101 embeds four CBAM modules as feature extractors to extract human features from the input image, such as contour features and positional features of human body parts, generating feature maps that include channel and spatial attention to improve segmentation accuracy. Then, the generated feature maps are processed by a pyramid pooling module to extract contextual information. The pyramid pooling kernel is divided into four levels: 1×1, 2×2, 3×3, and 6×6, to extract global and local features at different scales. Next, the feature maps extracted from the four levels are fused into global features. Finally, the fused global features are concatenated with the original input feature map, so that the extracted feature map carries both local and global contextual information.

[0065] Figure 8 The diagram shows the architecture of the CBAM-PSPNet network, which mainly consists of three parts: ResNet residual blocks, pyramid pooling modules, and the CBAM attention module. The CBAM attention module includes a channel attention module and a spatial attention module, set sequentially. The channel attention module calculates the weights of each channel in the input image to represent the relevance of each channel to key information; a higher weight indicates a higher relevance. The spatial attention module, based on the channel attention, learns a weight for each pixel in a two-dimensional plane according to the channel orientation to represent the importance of spatial location information. Finally, the weights learned by the channel and spatial attention modules are combined onto the input feature map to strengthen effective features and weaken ineffective features, achieving feature selection and enhancement. A database for training and testing the CBAM-PSPNet semantic segmentation network was constructed: 2500 human images meeting the requirements of this invention were selected from the public dataset Look into Person (LIP), with an image resolution of 300×600; 2000 images were directly acquired using a Hikvision MV-CA050-11UC camera, with an image resolution of 2448×2048. Based on this, the dataset was expanded to 20308 images, and ultimately 15795 human images were selected as the training set and 4513 human images as the test set.

[0066] This invention utilizes an improved CBAM-PSPNet semantic segmentation network for semantic segmentation of six regions of the human body: head, chest, waist, hips, thighs, and feet. To verify the effectiveness of this invention, the same embedding method is used in the ResNet101 backbone network. Performance is compared by adding CBAM modules at different locations. As shown in Table 1, for feature maps with an input size of 224×224, the ResNet101 backbone network has 42.50M parameters and 7.84G computation. By adding a CBAM attention module after each bottleneck structure in the ResNet101 backbone network, the model has 47.24M parameters and 7.85G computation, denoted as ResNet101(bottleneck)+CBAM. By adding the CBAM attention mechanism to the first and last convolutional layers of the ResNet101 backbone network, the model has 43.00M parameters and 7.84G computation, denoted as ResNet101(first-last)+CBAM. The method used in this invention has a model parameter size of 32.52M and a computational cost of 5.94G, denoted as ResNet101(layer)+CBAM+group. From the above data, it can be seen that ResNet101(Layer)+CBAM+group has the smallest parameter size and computational cost. Compared to the original ResNet101 backbone network, the parameter size is reduced by 9.98M and the computational cost by 1.9G. Compared to ResNet101(bottleneck)+CBAM, the parameter size is reduced by 14.72M and the computational cost by 1.91G. Compared to ResNet101(first-last)+CBAM, the parameter size is reduced by 10.48M and the computational cost by 1.9G.

[0067] Table 1 Comparison of parameter quantity and computational cost of different backbone network structures

[0068]

[0069] Table 2 shows a comparison of performance metrics for different methods. The average pixel accuracy (MPA) of the CBAM-PSPNet semantic segmentation network of this invention is 92.28%, which is 4.03% higher than the baseline network PSPNet. The pixel accuracy (PA) is 0.03% higher than PSPNet, and the average intersection-union ratio (MIOU) is 0.81% higher than the baseline network PSPNet. Meanwhile, compared with PSPNet (bottleneck) + CBAM, the pixel accuracy (PA) is improved by 0.03%, the MPA by 2.4%, and the MIOU by 0.73%. Compared with PSPNet (first-last) + CBAM, the pixel accuracy (PA) is improved by 0.06%, the average pixel accuracy (MPA) by 1.35%, and the MIOU by 0.65%.

[0070] Table 2 Comparison of performance indicators of different methods

[0071]

[0072] In this invention, a human segmentation algorithm based on an improved fusion attention mechanism is developed. Four CBAM modules are embedded in the four intermediate convolutional layers of the ResNet101 backbone network to achieve better fusion of spatial and channel features. At the same time, grouped convolutions are used instead of ordinary convolutions in the ResNet101 residual blocks to reduce model parameters and computational costs, thereby optimizing the efficiency of the PSPNet-based human segmentation algorithm.

[0073] Step 3: Use the Shi-Tomasi corner detection algorithm to extract the sub-pixel level feature point set of the human body measurement parts.

[0074] The design of feature points directly affects the accuracy of matching, which in turn directly impacts the accuracy of anthropometric measurements. For example... Figure 9 The image shown is a magnified view of the corner points of the checkerboard pattern of the present invention. The present invention is based on the feature point design of the corner points and adopts the checkerboard corner point design scheme. It can improve the matching accuracy to the sub-pixel level by using the region constraint stereo matching based on Shi-Tomasi corner point detection. At the same time, the region constraint of the checkerboard corner points reduces the matching complexity. The search range of its matching feature points, i.e., corner points, is within 4-5 pixels, which can greatly narrow the matching range and achieve fast and accurate matching.

[0075] The purpose of corner detection is to obtain feature point information of the human body's circumference region, and then use this information for human body measurement. Based on the characteristics of checkerboard corners, the Shi-Tomasi corner detection algorithm is adopted. This method achieves accurate localization of inner corners based on the changes in grayscale values ​​within the corner's neighborhood, with detection accuracy reaching the sub-pixel level, meeting the requirements for improving matching accuracy in human body measurement. Figure 10The diagram shows the corner detection results of the segmented region using the Shi-Tomasi corner detection algorithm, where the dots represent the detected feature points.

[0076] In the improved feature point design method for subpixel stereo matching based on corners, the subjects wear black and white checkerboard tights for measurement, with an adjacent checkerboard spacing of 2.5cm. The Shi-Tomasi corner detection algorithm is used to obtain the set of feature points within the human body segmentation area. Then, the obtained feature point set is constrained according to the preset color marker position information and checkerboard pattern, thereby obtaining matching point pairs of the same feature point in the left and right views, achieving subpixel-level fine stereo matching.

[0077] Step 4: Based on the preset color marker position information and checkerboard pattern, perform region constraints on the feature point set in the segmented image, obtain the two-dimensional coordinates of the feature points in the same row of the left and right views, and perform sub-pixel level fine stereo matching.

[0078] After extracting the sub-pixel level feature point coordinates of the segmented region, the complexity of stereo matching is reduced by using region constraints based on the characteristics of the checkerboard corner points. This invention achieves pixel information constraint by pre-setting marker points at the measurement location. Figure 11 This is a schematic diagram of the pre-set color marker scheme of the present invention. In this diagram, 1, 2, 3, and 4 represent four different turntable angles. L1, L2, L3, and L4 correspond to the left view of the waist segmentation area, and R1, R2, R3, and R4 correspond to the right view of the waist segmentation area. It can be observed that red and cyan markers are set in each 1 / 4 segmented circumference area. In the waist area, based on the characteristics of the human body, markers are pre-set from L1, L2, L3, and L4 at horizontal intervals of 8, 7, 8, and 7 checkerboard squares, and vertical intervals of 1 checkerboard square, for a total of 4 markers. This ensures that both red and cyan markers are captured in each shooting area, and that the rectangular area defined by the two markers includes the same circumference measurement line.

[0079] Taking a pair of lumbar segmentation regions L1 and R1 as an example, the matching of left and right views is achieved by constraining the pixel information of the corner points of the lumbar region using marked points: First, the segmented image contains four color categories: red, cyan, black, and white. All pixels in the segmented image constitute the dataset Z = {z1, z2, ... z...} i ...}, i = 1, 2, ... N, where each pixel z... i In the HSV color space, the component is z. i (H i ,S i V i The two-dimensional coordinates of the segmented image are z. i (xi ,y i Table 3 shows the HSV color space ranges for the four colors. If component V... i Greater than 46 and component S i Greater than 43, component H i Greater than 0 and less than 10 or component H i If the value is greater than 156 and less than 180, it is judged as red; if component V i Greater than 46 and component S i Greater than 43, component H i If the value is greater than 78 and less than 99, it is judged as cyan; if the component V i Greater than 221 and component S i If the value is less than 30, it is judged as white; if the component V i If the value is less than 46, it is judged as black. This yields the set M of pixels marked with red in the segmented image. R ={z R ∈Z|V R >46&S R >43&(0<H R <10||156<H R <180)};The set of pixels M marked with cyan C ={z C ∈Z|V C >46&S C >43&(78<H C <99)}. For the waist segmentation regions L1 and R1, the above judgment method is used to obtain a set of four pixels, denoted as M, representing the red and cyan markers in the left and right views. l_R M r_R M l_C and M r_C The center coordinates of the red and cyan markers within the waist segmentation regions L1 and R1 are obtained by averaging the pixel coordinates within the set. and in:

[0080]

[0081]

[0082]

[0083]

[0084] Table 3 HSV ranges for the four colors

[0085] Hmin Hmax Smin Smax Vmin Vmax red 0 / 156 10 / 180 45 255 46 255 blue 78 99 43 255 46 255 black 0 180 0 255 0 46 White 0 180 0 30 221 225

[0086] The Shi-Tomasi corner detection algorithm is used to accurately locate feature points in the waist segmentation region, obtaining the sub-pixel corner point sets of the L1 and R1 segmentation regions of the waist segmentation region, denoted as […]. Figure 12 This is a schematic diagram of the marker point constraint matching of the present invention. A rectangular region can be determined based on the center point coordinates of the red and cyan marker points. In the left view L1, the marker points... and As a constraint, obtain the set S of corner pixels within the constraint box defined by the red and cyan markers in the waist segmentation region L1. l_RC :

[0087]

[0088] in, and Similarly, in the right view R1, the marked points are... and As a constraint, obtain the set S of corner pixels within the constraint box defined by the red and cyan markers in the waist segmentation region R1. r_RC :

[0089]

[0090] in, and And N1 = M1.

[0091] By constraining the marker points, obtain the set S of corner points in the left and right views of the same row. l_RC and S r_RC ,in, and Based on the characteristics of the chessboard grid, the x-coordinates of the corner pixels within the same row are strictly sequentially increasing. Therefore, for the set of corner pixels S of the constrained region... l_RC and S r_RC S is obtained by sorting the x-coordinates. l_RC 'and S r_RC This allows for a one-to-one correspondence between the pixel coordinates of the left and right views of the same corner point, achieving sub-pixel level fine-grained stereo matching and reducing the complexity of stereo matching.

[0092]

[0093]

[0094] Through the above stereo matching process, matching point pairs of feature points in the same row are obtained in the left and right views, realizing sub-pixel level fine stereo matching and reducing the complexity of stereo matching scheme.

[0095] The above process only completes the precise matching of corner points within the constrained area. To achieve precise matching of corner points across multiple rows within the segmented area, based on the characteristics of the chessboard grid, the center coordinates of the red and cyan markers are used as constraints, and the matching is performed according to the y-coordinates of the markers with a step size N. step Move up or down, where N step This represents the pixel difference between adjacent chessboard corner points. In actual shooting, due to varying shooting distances D (in meters), N... step The value will change; the closer the shooting distance, the higher the value of N. step The larger the distance, the greater the shooting distance D and the step size N. step The following relationship exists:

[0096] N step =7.02D 2 -45.18D+93.43 (11)

[0097] In this embodiment, D = 2.4m, N step =25. Marker point and The vertical axis is set with a step size N. step Progressing upwards or downwards, we obtain... and Complete the accurate matching of pixel information across multiple rows in the segmented region.

[0098] Step 5: Calculate the three-dimensional spatial coordinates of feature points in the same row using the binocular stereo calibration parameters.

[0099] By utilizing corner detection to obtain sub-pixel-level information of feature points in the human body circumference region, sub-pixel coordinates of feature points in the same row in both left and right views are obtained, enabling row-by-row stereo matching. Using binocular stereo calibration parameters, the three-dimensional spatial coordinates of feature points in the same row are further calculated, realizing the conversion from two-dimensional planar information to three-dimensional spatial information, providing three-dimensional spatial data for the circumference fitting in step six.

[0100] Step 6: Rotate the feature points back to their initial positions according to the rotation angle of the turntable, use the PIVCF method to fit the human body circumference, calculate and output the human body parameter measurement data.

[0101] To obtain the initial coordinates of the entire human body circumference, this invention requires rotating the three-dimensional coordinates of the rotated feature points back to their initial positions. The axis of rotation is the Y-axis, perpendicular to the center point O(x0, y0, z0) of the turntable. The center point of the turntable is marked with colored points for easy coordinate value extraction. Since the connecting line of the feature points for circumference measurement is parallel to the ground, and the axis of rotation is also perpendicular to the ground, to improve measurement efficiency during the experiment, the contour map of the fitted circumference is projected onto the XOZ plane. Based on the turntable rotation angle θ and the turntable center point O(x0, y0, z0), the feature points (x, y, z) are rotated back to their initial positions (x′, y′, z′).

[0102]

[0103] This invention employs the PIVCF method, a polynomial fitting method with intermediate variables, to fit the girth curve. Figure 13 and Figure 14 The curve fitting results for six dimensions are presented separately for one female and one male subject. Figure 13 and Figure 14 It can be seen that the PIVCF fitting results used in this invention are highly consistent with the original point connection results, and are more in line with the contour features of the human body than the original point connection results.

[0104] According to GB / T 16160-2017 "Definition and Method of Clothing Human Measurement", the maximum circumference data is used as the final circumference output for the chest, hips and upper thigh, and the minimum circumference data is used as the final circumference output for the waist. Based on the rows containing the chest and upper thigh circumference, two rows of checkerboard patterns are moved down, and the data in the third row is used as the final circumference output for the underbust and mid-thigh circumference.

[0105] Based on actual testing and verification on multiple male (M) and female (F) subjects, as shown in Table 4, compared with the invention patent with application number 202010441103.7, the mean absolute error (MAD) value of the girth measurement of the present invention is significantly smaller than the MAD value of the girth measurement of the invention patent, which can significantly improve the accuracy of human body measurement.

[0106] Table 4 Comparison of circumference measurement errors

[0107]

[0108] This invention utilizes a set of synchronously triggered binocular cameras and a precision turntable to acquire full-dimensional images, resulting in a simple system structure and low cost. The human segmentation algorithm incorporating an attention mechanism and the feature point design based on sub-pixel-level stereo matching of corner points proposed in this invention can significantly improve the accuracy of human parameter measurements.

[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for measuring human body parameters that integrates attention mechanisms and corner-point stereo matching, characterized in that, The steps are as follows: Step 1: Using a binocular camera and a turntable, the turntable rotates at 90-degree intervals, and the binocular camera captures four sets of human body images from different angles, including left and right views. Step 2: Input the left and right views of each group into the CBAM-PSPNet semantic segmentation network to extract human body measurement parts; The CBAM-PSPNet semantic segmentation network embeds four CBAM attention modules in the feature extraction stage of the ResNet101 backbone network of the PSPNet network; in the bottleneck structure of the ResNet101 backbone network, ordinary convolutions are replaced with grouped convolutions. Step 3: Use the Shi-Tomasi corner detection algorithm to extract the sub-pixel level feature point set of the human body measurement area; The Shi-Tomasi corner detection algorithm accurately locates inner corners based on the changes in grayscale values ​​within the corner's neighborhood. Step 4: Based on the preset color marker position information and checkerboard pattern, the feature point set is constrained regionally to achieve fine stereo matching at the sub-pixel level and obtain the two-dimensional coordinates of the feature points in the same row of the left and right views. The method for setting the preset color markers is as follows: red and cyan markers are set in each 1 / 4 division circumference area, and the rectangular area defined by the two markers contains the same circumference measurement line. Step 5: Calculate the 3D spatial coordinates of feature points in the same row using the binocular stereo calibration parameters; Step 6: Rotate the feature points back to their initial positions according to the rotation angle of the turntable, and use the PIVCF method to fit the human body circumference to obtain human body parameter measurement data; The method for defining the region constraint is as follows: The Shi-Tomasi corner detection algorithm accurately locates feature points in the human body segmentation regions of the left and right views, obtaining sub-pixel corner set sets for the segmented regions of the left and right views respectively. and Where, x l_i y l_i These represent the x and y coordinates of the corner pixels within the segmented area of ​​the left view, respectively. r_j y r_j These represent the x and y coordinates of the corner pixels within the segmented area of ​​the right view, respectively, while M and N represent the number of corner pixels detected within the segmented areas of the left and right views, respectively. Obtain the set S of corner points S in the left and right views of the same row using the marker point constraint. l_RC and S r_RC ,in, and For the set of corner points S of the constrained region l_RC and S r_RC S is obtained by sorting the x-coordinates. l_RC 'and S r_RC This allows for a one-to-one correspondence between the pixel coordinates of the same corner point in the left and right views, resulting in matching point pairs of the same row of feature points in the left and right views, thus achieving sub-pixel level fine stereo matching. Based on the characteristics of the chessboard, and using the center coordinates of the red and cyan markers as constraints, the steps are adjusted according to the y-coordinates of the markers with a step size N. step Move up or down, where N step The difference in the number of pixels between adjacent chessboard corner points; and the shooting distance D and the step size N step The relationship is: N step =7.02D 2 -45.18D +93.43; The method for implementing the marker point constraint is as follows: All pixels in the segmented image constitute the dataset Z = {z1, z2, ... z} i ...}, i = 1, 2, ... N, where each pixel z... i In the HSV color space, the component is z. i (H i ,S i V i The two-dimensional coordinates of the segmented image are z. i (x i ,y i ); Based on the HSV color range, the pixel set M of the red and cyan markers in the left view is obtained for each segmented region. l_R M l_C The set of pixels M marked with red and cyan colors in the right view r_R M r_C The center point coordinates of the red and cyan markers within the segmented area of ​​the left view are obtained by averaging the pixel coordinates within the pixel set. The coordinates of the center points of the red and cyan markers within the area divided by the right view Mark points and As constraints, obtain the set of corner pixel points within the constraint box defined by the red and cyan markers in the left view segmentation region: in, and M1 represents the number of corner points within the left view constraint box; Mark points and As constraints, obtain the set of corner pixel points within the constraint box defined by the red and cyan markers in the right view segmentation region: in, and N1 represents the number of corner points within the constraint box of the right view, and N1 = M1.

2. The method for measuring human body parameters based on corner-point stereo matching, according to claim 1, is characterized in that... The binocular camera captures human images of the subject wearing a black and white checkered bodysuit, with a spacing of 2.5 cm between adjacent checkered patterns. The calibrated binocular camera is placed directly in front of the turntable, and the subject stands in the center of the turntable. The turntable rotates 90° clockwise 4 times every 7.5 seconds to return to the starting position. The binocular camera captures human images once each at 0 degrees, 90 degrees, 180 degrees, and 270 degrees. The center point of the turntable is marked with a colored dot; The datasets used for training and testing the CBAM-PSPNet semantic segmentation network include: 2500 human images with a resolution of 300×600 selected from the public dataset Look into person; 2000 human images with a resolution of 2448×2048 captured by a camera; the dataset was expanded to 20308 images, with 15795 human images selected as the training set and 4513 human images selected as the test set.

3. The fusion attention mechanism and human parameter measurement method based on corner-point stereo matching according to claim 1 or 2, characterized in that, The CBAM-PSPNet semantic segmentation network was used to perform semantic segmentation of six regions of the human body in the left and right views: head, chest, waist, hips, thighs, and feet.

4. The method for measuring human body parameters based on corner-point stereo matching, according to claim 3, is characterized in that... The intermediate convolutional part of the residual network of the backbone network ResNet101 is composed of residual blocks, and one CBAM module is embedded in the output part of each of the four layers of the intermediate convolutional part. The implementation method of the grouped convolution is as follows: For a 256-channel input feature, it is divided into two branches, namely a straight branch and a shortcut branch. Grouped convolution is performed at the second 3×3 convolution layer of the straight branch, and then the outputs of each grouped convolution are concatenated by channel stacking. The number of parameters in the grouped convolution is the same as that in a regular convolution. Where g is the number of groups for each convolutional kernel in the grouped convolutional layer.

5. The method for measuring human body parameters based on corner-point stereo matching, according to claim 4, is characterized in that... The CBAM-PSPNet semantic segmentation network uses an improved backbone network ResNet101 with four CBAM modules embedded as feature extractors to extract features from the input image and generate feature maps. Then, a pyramid pooling module is used to extract contextual information from the generated feature maps. The extracted feature maps are fused into global features. Finally, the fused global features are concatenated with the original input feature map. The pyramid pooling module has a pyramid pooling kernel divided into four levels: 1×1, 2×2, 3×3 and 6×6, which are used to extract global and local features of different scales.

6. The method for measuring human body parameters based on corner-point stereo matching according to claim 5, characterized in that, The marker point and The ordinate is expressed with a step size N step Progressing upwards or downwards, we obtain... and Complete the accurate matching of pixel information across multiple rows in the segmented region.

7. The fusion attention mechanism and the human body parameter measurement method based on corner-point stereo matching according to any one of claims 1, 4, 5 or 6, characterized in that, The implementation method of the PIVCF method is as follows: the three-dimensional coordinates of the rotated feature point are rotated back to the initial position, and the rotation axis is the Y-axis perpendicular to the turntable center point O(x0,y0,z0); the contour map of the fitted gravitational dimension is projected onto the XOZ plane, and the feature point (x,y,z) is rotated back to the initial position (x′,y′,z′) according to the turntable rotation angle θ and the turntable center point O(x0,y0,z0): The maximum circumference data is used as the final circumference output for the bust, hips, and upper thigh, while the minimum circumference data is used as the final circumference output for the waist. Based on the rows containing the bust and upper thigh circumference, two rows of checkerboard patterns are moved down, and the data from the third row is used as the final circumference output for the underbust and mid-thigh circumference.

Images