A low-contrast human body analysis method based on frequency domain enhancement and structured feature guidance

Abstract

The application provides a low-contrast human body analysis method based on frequency domain enhancement and structured feature guidance, comprising: constructing a multi-scale feature extraction backbone network, performing multi-scale feature extraction on an input low-contrast human body image to obtain multi-level spatial features; building an edge detection branch, inputting the multi-level spatial features into the edge detection branch, sequentially performing frequency multi-layer perception machine enhancement, feature fusion module fusion based on edge perception, and high-frequency enhancement module enhancement based on noise perception to obtain high-precision human body edge features; building a human body analysis branch, inputting the multi-level spatial features into the human body analysis branch, and processing the multi-level spatial features through a structured enhancement-based class feature guidance module to obtain human body analysis semantic features; and performing fusion optimization on the high-precision human body edge features and the human body analysis semantic features through a boundary feature fusion module, and then performing processing through a decoder to output a pixel-level analysis result of the low-contrast human body.

Description

Technical Field

[0001] This invention relates to the field of human body analysis technology, and more particularly to a low-contrast human body analysis method based on frequency domain enhancement and structured feature guidance. Background Technology

[0002] With the rapid development of smart terminal devices and multimedia technologies, image and video data with the human body as the primary visual subject has become an important component of digital information. In the field of computer vision, human body analysis technology provides key technical support for applications such as intelligent monitoring, virtual try-on, and human-computer interaction by finely segmenting various anatomical parts of the human body. This technology requires accurate identification and segmentation of semantic parts (such as the head, torso, and limbs) within the human body, and its performance directly affects the effectiveness of downstream tasks. Current mainstream methods rely on deep learning models to extract and analyze human visual features, but they still face significant challenges in real-world scenarios such as complex lighting and low contrast.

[0003] Existing human body parsing techniques can be mainly categorized into four types: visual context-based modeling, human structure prior guidance, multi-task collaborative learning, and probabilistic graph optimization. Visual context-based methods enhance parsing performance by establishing semantic associations between local and global features; however, in low-contrast conditions, the association modeling fails due to the degradation of local features. While human structure-based methods can utilize the topological relationships between body parts, they struggle to distinguish adjacent parts due to the high similarity of features in low-contrast regions. Multi-task learning methods rely on auxiliary supervision signals such as edges or poses, but these signals are severely lacking in low-light environments. Probabilistic graph methods require a clear spatial distribution of body parts as a prior, but errors in part localization in low-contrast images can undermine the effectiveness of the probabilistic graph. These issues collectively lead to defects in existing techniques such as part missegmentation, blurred boundaries, and semantic confusion in low-contrast scenes. Summary of the Invention

[0004] To address the technical problems of blurred edges, weak semantic feature recognition, and insufficient resolution accuracy in low-contrast human images, this invention provides a low-contrast human image resolution method and system based on frequency domain enhancement and structured feature guidance. This invention primarily utilizes a multi-scale feature extraction backbone network combined with a frequency domain enhanced edge detection branch and a structured feature-guided human image resolution branch. A boundary feature fusion module achieves joint optimization of the two types of features, and multiple loss functions are used for joint training. This results in enhanced edge details in low-contrast images, improved semantic representation of human body parts, and significantly improved pixel-level resolution accuracy.

[0005] The technical means employed in this invention are as follows:

[0006] A low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance includes: S1. Construct a multi-scale feature extraction backbone network to extract multi-scale features from the input low-contrast human image and obtain multi-level spatial features. S2. Construct an edge detection branch, input the multi-level spatial features into the edge detection branch, and then enhance them sequentially through a frequency multilayer perceptron, a feature fusion module based on edge perception, and a high-frequency enhancement module based on noise perception to obtain high-precision human edge features. S3. Construct a human body parsing branch, input the multi-level spatial features into the human body parsing branch, and after processing by the structure-enhanced class feature guidance module, obtain the human body parsing semantic features. S4. The high-precision human body edge features and the human body parsing semantic features are fused and optimized through the boundary feature fusion module, and then processed by the decoder to output the pixel-level parsing result of the low-contrast human body.

[0007] Furthermore, the method also includes: S5. Construct a weighted loss function for multiple losses and jointly train the low-contrast human body parsing network based on frequency domain enhancement and structured feature guidance, and iteratively optimize all parameters of the network.

[0008] Further, in step S1, the multi-scale feature extraction backbone network adopts ResNet101, and the specific multi-scale feature extraction steps include: S11, Convert low-contrast human body images Enter to 7 A convolutional layer with 7 kernels and a stride of 2 yields a size of [size missing]. The first feature map; S12. Input the first feature map into the step size of 2, 3. The maximum pooling layer of size 3 yields a size of Second feature map ; S13, Transfer the second feature map The first residual stage is input, and after passing through 3 residual blocks, the output size is... Features ; Features The second residual stage is input, and after passing through 4 residual blocks, the output size is... Features ; Features The input is the third residual stage. After passing through 23 residual blocks, the output size is... Features ; Features The input is the fourth residual stage. After passing through 3 residual blocks, the output size is... Features .

[0009] Further, step S2 includes: S21. Input the multi-level spatial features obtained in step S1 into a frequency multilayer perceptron. The frequency multilayer perceptron processes the multi-level spatial features. , A Fast Fourier Transform (FFT) is performed to map the features to the frequency domain and decompose them into amplitude and phase spectra. A multilayer perceptron is used to specifically enhance the amplitude component, and the enhanced amplitude component is then combined with the original phase component. An Inverse Fast Fourier Transform (IFT) is then performed to map the enhanced features back to the spatial domain, and residual fusion is then performed to obtain the frequency-domain enhanced features. , ; S22. Input the frequency-domain enhanced features into the edge-aware feature fusion module. The feature fusion module performs feature fusion as follows: Low-level features After bilinear interpolation and high-level features Passing through steps of 1 and sizes of 1 respectively The output features are obtained by performing a grouped convolution of 1 and then concatenating the two groups. The output features are passed through two consecutive steps of size 1 and size 3. The convolution of size 3 has a stride of 1 and a size of 1. After a convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The generated weight information is combined with low-level features. and advanced features Multiplication yields the fused features, and cross-scale residual connections are used to obtain a feature of size [value missing]. Features The formula is as follows:

[0010] in, This represents the learnable weight parameters. Represents the attention weight matrix. This represents the matrix dot product operation; Features By using two consecutive steps of size 1 and size 3 The convolution of size 3 has a stride of 1 and a size of 1. After a convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The generated weight information and features and frequency domain enhanced features Multiplication yields the fused features, and cross-scale residual connections are used to obtain a feature of size [value missing]. Multi-scale fusion features The formula is as follows:

[0011] in, This represents the learnable weight parameters. Represents the attention weight matrix. This represents the matrix dot product operation; S23, Fusing features across multiple scales The input is a noise-aware high-frequency enhancement module. The high-frequency enhancement process performed by the module is as follows: Multi-scale fusion features After average pooling and upsampling operations, the multi-scale fused features are... After average pooling and upsampling operations Difference and extraction of high-frequency features The formula is as follows:

[0012] in, This indicates the fusion of multi-scale features. After average pooling and upsampling operations; High frequency features Using steps of 1 and size of 3 respectively A standard convolution with a stride of 2, an inflation rate of 2, and a size of 3. 3-dimensional dilated convolution, simultaneously incorporating high-frequency features We perform standard convolution with a stride of 1 and a size of 1×1, followed by average pooling with a stride of 1 and a size of 1. After a standard convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The guide uses bi-branch convolution to capture multi-scale features, introduces frequency domain channel attention to further optimize high-frequency features, and obtains a size of [size missing] through cross-scale residual connections. Edge features The formula is as follows:

[0013] in, This indicates frequency channel attention operations. Indicates 3 3. Standard convolution operations Indicates 3 3. Dilated convolution operation, This represents the attention weight matrix.

[0014] Further, in step S23, the frequency domain channel attention is modeled using multi-frequency components for refined calibration and enhancement of channel features, including: High frequency features The channel is divided into multiple blocks, and each block is assigned a dedicated two-dimensional discrete cosine transform component to extract the feature frequency information corresponding to each channel subset. This yields features with the same width and height and a channel number of 1. These features are then fed into a fully connected layer for learning and input into a Sigmoid activation function to obtain the frequency channel attention weight matrix.

[0015] Further, step S3 includes: S31. The size extracted from the last layer of the multi-scale feature extraction backbone network is... Features The input is fed into the feature pyramid module, resulting in a size of Enhanced features ; S32, Enhance Features Feed into the vertical convolution branch to enhance features Continuously passing through a step size of 1 and a size of 1 1, step size is 1, size is 3 1, step size is 1, size is 7 A vertical bar convolution of size 1, after vertical pooling, yields a result of size 1. Predicted class distribution map ; S33, Enhance Features Feed into horizontal convolutional branches to enhance features After a step size of 1 and a size of 1 1, step size is 1, size is 3 1, step size is 1, size is 7 A horizontal bar convolution of size 1, after horizontal pooling, yields a result of size 1. Predicted class distribution map ; S34. Spatial attention is added after vertical and horizontal bar convolutions to obtain features. and The formula is as follows:

[0016]

[0017] in, This represents spatial attention operations. Represents the learnable weight coefficients; S35, Features and The summation is used as input features, which are then passed through a standard convolution with a stride of 1 and a size of 3×3, and finally through a compression-excitation channel attention process to obtain the features. The formula is as follows:

[0018] in, This indicates a compression-excitation channel attention operation; S36, Features , , , The concatenation was performed using the Concat operation, and then channel dimensionality reduction was achieved using a standard convolution with a stride of 1 and a size of 3×3, resulting in a size of [size missing]. Human body semantic features The formula is as follows:

[0019] in, This indicates a splicing operation.

[0020] Furthermore, in step S31, the feature pyramid module effectively compensates for the limitations of the deep receptive field of convolutional neural networks through parallel multi-scale pooling and feature fusion strategies, significantly improving the segmentation accuracy of the semantic segmentation model, including: For input size Features Parallel execution 1 1, 2 2, 3 3, 6 6. Adaptive average pooling operations at four scales; Bilinear interpolation is used to upsample all the scaled features after dimensionality reduction to the size of the original input features; The original input features and the four upsampled scale features are concatenated and fused along the channel dimension to obtain a size that includes both global semantics and local details. Enhanced features .

[0021] Further, in step S34, the spatial attention automatically captures important regional features by focusing on the effective information locations on the features, including: The input features are subjected to global max pooling and global average pooling along the channel dimension to obtain two features with the same width and height and a channel number of 1. Perform a Concat concatenation operation on two features with the same width and height and 1 channel to obtain a feature with the same width and height and 2 channels; After performing a standard convolution with a stride of 1 and a size of 7×7 on the concatenated result, the result is input into the Sigmoid activation function to obtain the spatial attention weight matrix.

[0022] Further, step S4 includes: S41. Employ an edge branch decoder for edge features. Decoding operations include: The size is Edge features After a step size of 1 and a size of 3 Standard convolution with a stride of 1 and a size of 1. Channel reduction and bilinear interpolation of standard convolution result in a product of size 1. Features ; S42. Employ a human body parsing branch decoder to parse semantic features of the human body. Decoding operations include: The size is Human body semantic features After a step size of 1 and a size of 1 The channel reduction operation of standard convolution and the bilinear interpolation operation yield a size of 1. Features ; The size is Features After a step size of 1 and a size of 3 The channel reduction operation of a standard convolution with dimensions 3 yields a size of Features; The size is Features and size The features are concatenated using the Concat operation, and then processed with a step size of 1 and a size of 3. A standard convolution with a stride of 1 and a size of 1. The channel reduction operation of a standard convolution of size 1 yields a size of Human body semantic features ; With a step size of 1 and a size of 1 Channel reduction and bilinear interpolation of standard convolution result in a product of size 1. Features ; S43. Human body edge features are fused using the boundary feature fusion module. semantic features of human body analysis The two types of features are initially fused through element-wise summation; the fused features are then input into multiple convolutional layers to further explore the deep correlations between features, and finally, a channel attention operation is performed to obtain a feature of size [size missing]. Features ; S44, with dimensions of Features Input decoder, features After performing a standard convolution with a stride of 1 and a size of 3×3, a Dropout operation, a channel reduction operation of a standard convolution with a stride of 1 and a size of 1×1, and a bilinear interpolation operation, the resulting object has a size of [missing information]. Features .

[0023] Further, step S5 includes: S51. Construct the edge detection cross-entropy loss function The pixel-level cross-entropy loss between the edge map and the edge label map output by the edge detection branch is calculated using the following formula:

[0024] in, Indicates the first edge label in the image. Line number The actual label value of the column; if the pixel belongs to the edge, then... ,otherwise ; Represents the edge graph of the first Line number The probability that a column is predicted to be marginal; S52. Constructing the human body parsing cross-entropy loss function The pixel-level cross-entropy loss between the analytical image and the analytical label image output by the human body analysis branch is calculated using the following formula:

[0025] in, This indicates the parsing of the label graph. Line number The actual label value of the column, if the pixel belongs to the category. ,but ,otherwise ; In the analytical graph, the first Line number Column position belongs to category The probability of; S53. Construct the cross-entropy loss function between the final parsed graph and the parsed label graph. The formula is as follows:

[0026] in, In the analytical graph, the first... Line number Column position belongs to category The probability of; S54. Constructing the Class Distribution Loss Function The loss for calculating the vertical and horizontal predicted class distribution maps and their corresponding label class distribution maps is as follows:

[0027] in, The first level in the horizontal class distribution diagram Line number The predicted value of the column, In the vertical class distribution diagram, the first... Line number The predicted value of the column; The horizontal class distribution diagram shows the first... Line number The actual label value of the column. In the vertical class distribution diagram, the first... Line number The actual label value of the column; S54, Cross-entropy Loss Function Based on Edge Detection Human body analysis cross-entropy loss function Cross-entropy loss function and class distribution loss function Construct the overall network loss function The formula is as follows:

[0028] in, , , All of these represent learnable hyperparameters. , All of these represent weighting coefficients.

[0029] Compared with the prior art, the present invention has the following advantages: 1. This invention uses a frequency domain multilayer perceptron and noise-sensing high-frequency enhancement technology to map spatial domain features to the frequency domain, accurately enhance the high-frequency edge components in the amplitude spectrum, and suppress noise interference. This effectively solves the problem of blurred human body contours and component edges in low-contrast images, making them difficult to identify, and significantly improves the clarity and recognizability of edge features.

[0030] 2. This invention achieves cross-scale weighted fusion of shallow detail features and deep semantic features through edge-aware multi-scale feature fusion technology, avoiding edge breaks or redundancy caused by single-scale features, making edge features more complete and continuous, and providing accurate boundary constraints for subsequent pixel-level parsing.

[0031] 3. This invention uses structured enhancement class feature guidance technology, introduces vertical and horizontal strip convolution and spatial attention mechanism, and specifically extracts the structured distribution features of human body parts (such as the direction and position rules of limbs and torso), which solves the problem of semantic confusion and unclear boundaries of parts in low contrast images and enhances the class distinction of different human body parts.

[0032] 4. This invention uses channel attention and feature pyramid enhancement techniques to finely calibrate features in both channel and spatial dimensions, thereby increasing the weight of key semantic channels and enriching multi-scale semantic information. This enables the human body parsing branch to more accurately perceive human body parts of different sizes and postures, thus improving the robustness of semantic features.

[0033] 5. This invention uses a boundary feature fusion module to perform element-wise weighted fusion of high-precision human body edge features and human body parsing semantic features, achieving complementarity between edge constraints and semantic information. This solves the problems of boundary offset and semantic misclassification caused by separate training branches, making the pixel-level boundary of the final parsing result more closely match the real human body contour.

[0034] 6. This invention uses multi-layer convolution and upsampling techniques in the decoder to further mine deep correlations based on fused features, gradually restoring image resolution and ensuring analytical accuracy at high resolution. At the same time, the Dropout operation improves the model's generalization ability and avoids overfitting.

[0035] 7. This invention constructs a weighted fusion overall loss of edge detection loss, human body parsing loss and class distribution loss through multi-loss function joint training technology, realizing the collaborative optimization of edge detection branch and human body parsing branch, solving the problem that a single loss function cannot take into account both edge accuracy and semantic accuracy, and making the network parameter update direction more in line with the overall parsing goal.

[0036] 8. This invention uses learnable hyperparameter weight allocation technology to assign differentiated weights to different task losses (e.g., the human body analysis loss weight is set to 40), which strengthens the optimization priority of the core analysis task, while ensuring the effective role of edge constraints, and further improves the accuracy and consistency of the overall analysis results.

[0037] In summary, the technical solution of this invention effectively avoids the shortcomings of existing technologies, such as blurred edges in low-contrast human images, weak semantic feature recognition, insufficient analytical accuracy, and inconsistent training objectives across multiple tasks. Through innovative techniques including frequency domain-enhanced edge feature extraction, structured feature-guided semantic parsing, boundary feature fusion, and multi-loss joint training, it achieves synergistic optimization of edge details and semantic information. Therefore, the technical solution of this invention solves the problems of inaccurate recognition of human contours and component edges in low-contrast environments, semantic confusion of human components, boundary offset of analytical results, and difficulty in synergistic optimization of multi-branch training in existing technologies. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of the overall process of the method of the present invention.

[0040] Figure 2 This is a diagram of the overall network structure of the present invention.

[0041] Figure 3 This is a structural diagram of the feature fusion module based on edge perception in this invention.

[0042] Figure 4 This is a structural diagram of the high-frequency enhancement module based on noise perception of the present invention.

[0043] Figure 5 This is a structural diagram of the class feature guidance module based on structured enhancement in this invention.

[0044] Figure 6 The figures shown are partial results provided for embodiments of the present invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product or device.

[0047] This application proposes a low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance, aiming to achieve accurate human body resolution under low-contrast conditions. The overall flowchart of the network is as follows. Figure 1 As shown, the overall structure is as follows Figure 2 As shown, the network mainly consists of two key branches: an edge detection branch and a human body parsing branch. The specific implementation steps are as follows: S1. Construct a multi-scale feature extraction backbone network to extract multi-scale features from the input low-contrast human image and obtain multi-level spatial features. S2. An edge detection branch is constructed. The multi-level spatial features are input into the edge detection branch and sequentially enhanced by a frequency multilayer perceptron, fused by an edge-aware feature fusion module, and enhanced by a noise-aware high-frequency enhancement module to obtain high-precision human edge features. In this embodiment, the edge detection branch achieves high-precision human edge detection by enhancing high-frequency features in the frequency domain, based on fully exploring spatial domain feature representations. First, the multi-scale features output from the backbone network are input into the frequency multilayer perceptron. Leveraging the global perspective of frequency domain analysis, the limitations of the spatial domain model due to low-contrast images are compensated for, thus enhancing the feature representation of human edges. Second, an edge-aware feature fusion module is designed. This module dynamically calibrates the information contribution weights between multi-scale features using an attention mechanism, solving the problems of information redundancy and key region weakening during multi-scale feature fusion in low-contrast scenes. Finally, a noise-aware high-frequency enhancement module is designed. This module, based on noise perception, performs refined enhancement of high-frequency semantic features at the edges, significantly improving the distinguishability of human body part boundaries in the features, providing high-precision feature support for the final pixel-level human body analysis task.

[0048] S3. Construct a human body parsing branch. Input the multi-level spatial features into the human body parsing branch. After processing by the structured enhancement-based class feature guidance module, the semantic features of the human body parsing are obtained. In this embodiment, a structured enhancement-based class feature guidance module is proposed for the human body parsing branch. This module accurately captures the inherent spatial topological constraints and differentiated shape information of each part by introducing a spatial relationship modeling and shape feature extraction branch for human body parts. This guides the model to adaptively learn the positional correlation and shape specificity of different parts, effectively alleviating the category confusion problem caused by similar shapes and adjacent spatial distribution of parts in low-contrast scenes, and improving the structural consistency and classification accuracy of the parsing results.

[0049] S4. The high-precision human body edge features and the human body parsing semantic features are fused and optimized through the boundary feature fusion module, and then processed by the decoder to output the pixel-level parsing result of the low-contrast human body. In this embodiment, the boundary feature fusion module is introduced to further enhance the collaborative optimization of edge details and parsing features, realize the efficient fusion between the boundary features output by the edge detection branch and the semantic features of the human body parsing branch, and ensure that the parsing result has both reliable semantic category discrimination ability and can retain high-precision part boundary details.

[0050] S5. Construct a weighted loss function for multiple losses and jointly train the low-contrast human body resolution network based on frequency domain enhancement and structured feature guidance. Iteratively optimize all parameters of the network to improve the accuracy of low-contrast human body pixel-level resolution results.

[0051] In a specific implementation, as a preferred embodiment of the present invention, in step S1, the multi-scale feature extraction backbone network adopts ResNet101, and the specific multi-scale feature extraction steps include: S11, Convert low-contrast human body images Enter to 7 A convolutional layer with 7 kernels and a stride of 2 yields a size of [size missing]. The first feature map; S12. Input the first feature map into the step size of 2, 3. The maximum pooling layer of size 3 yields a size of Second feature map ; S13, Transfer the second feature map The first residual stage is input, and after passing through 3 residual blocks, the output size is... Features ; Features The second residual stage is input, and after passing through 4 residual blocks, the output size is... Features ; Features The input is the third residual stage. After passing through 23 residual blocks, the output size is... Features ; Features The input is the fourth residual stage. After passing through 3 residual blocks, the output size is... Features .

[0052] In this embodiment, through these four residual stages, the network extracts multi-level spatial feature representations, providing richer hierarchical feature representations for subsequent feature enhancement and fusion.

[0053] In a specific implementation, as a preferred embodiment of the present invention, step S2 includes: S21. Input the multi-level spatial features obtained in step S1 into a frequency multilayer perceptron. The frequency multilayer perceptron processes the multi-level spatial features (the backbone network outputs the first...) Layer characteristics , The fast Fourier transform (FFT) is performed to map the features to the frequency domain and decompose them into amplitude and phase spectra. A multilayer perceptron is used to specifically enhance the amplitude components, and the enhanced amplitude components are then combined with the original phase components. An inverse fast Fourier transform is then performed to map these components back to the spatial domain, and finally, residual fusion is used to obtain the frequency-enhanced features. , ; In this embodiment, the frequency multilayer perceptron compensates for the inherent defects of the spatial domain model, which is limited by the local receptive field, by leveraging the global field-view characteristics of the frequency domain, and directionally enhances the high-frequency features of human body edges in low-contrast images.

[0054] S22. Input the frequency-domain enhanced features into the edge-aware feature fusion module. The overall structure of the edge-aware feature fusion module is as follows: Figure 3 As shown, the feature fusion module performs feature fusion as follows: Low-level features After bilinear interpolation and high-level features Passing through steps of 1 and sizes of 1 respectively The output features are obtained by performing a grouped convolution of 1 and then concatenating the two groups. The output features are passed through two consecutive steps of size 1 and size 3. The convolution of size 3 has a stride of 1 and a size of 1. After a convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The generated weight information is combined with low-level features. and advanced features Multiplication yields the fused features, and cross-scale residual connections are used to obtain a feature of size [value missing]. Features The formula is as follows:

[0055] in, This represents the learnable weight parameters. Represents the attention weight matrix. This represents the matrix dot product operation; Features By using two consecutive steps of size 1 and size 3 The convolution of size 3 has a stride of 1 and a size of 1. After a convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The generated weight information and features and frequency domain enhanced features Multiplication yields the fused features, and cross-scale residual connections are used to obtain a feature of size [value missing]. Multi-scale fusion features The formula is as follows:

[0056] in, This represents the learnable weight parameters. Represents the attention weight matrix. This represents the matrix dot product operation; S23, Fusing features across multiple scales The input is a noise-aware high-frequency enhancement module, the structure of which is as follows: Figure 4 As shown, the high-frequency enhancement module performs high-frequency enhancement as follows: Multi-scale fusion features After average pooling and upsampling operations, the multi-scale fused features are... After average pooling and upsampling operations Difference and extraction of high-frequency features The formula is as follows:

[0057] in, This indicates the fusion of multi-scale features. After average pooling and upsampling operations; High frequency features Using steps of 1 and size of 3 respectively A standard convolution with a stride of 2, an inflation rate of 2, and a size of 3. 3-dimensional dilated convolution, simultaneously incorporating high-frequency features We perform standard convolution with a stride of 1 and a size of 1×1, followed by average pooling with a stride of 1 and a size of 1. After a standard convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The method guides bi-branch convolution to capture multi-scale features, introduces frequency channel attention (FCA) to further optimize high-frequency features, and obtains a size of [size missing] through cross-scale residual connections. Edge features The formula is as follows:

[0058] in, This indicates frequency channel attention operations. Indicates 3 3. Standard convolution operations Indicates 3 3. Dilated convolution operation, This represents the attention weight matrix.

[0059] In a specific implementation, as a preferred embodiment of the present invention, in step S23, the frequency domain channel attention improves upon the limitation of traditional channel attention, which relies solely on global average pooling to extract the lowest frequency component, by modeling through multiple frequency components. This is used for refined calibration and enhancement of channel features, including: High frequency features The channel is divided into multiple blocks, and each block is assigned a dedicated two-dimensional discrete cosine transform component to extract the feature frequency information corresponding to each channel subset. This yields features with the same width and height and a channel number of 1. These features are then fed into a fully connected layer for learning and input into a Sigmoid activation function to obtain the frequency channel attention weight matrix.

[0060] In a specific implementation, as a preferred embodiment of the present invention, in step S3, the overall structure of the class feature guidance module based on structured enhancement is as follows: Figure 5 As shown, the processing procedure includes: S31. The size extracted from the last layer of the multi-scale feature extraction backbone network is... Features The input is fed into the feature pyramid module, resulting in a size of Enhanced features ; S32, Enhance Features Feed into the vertical convolution branch to enhance features Continuously passing through a step size of 1 and a size of 1 1, step size is 1, size is 3 1, step size is 1, size is 7 A vertical bar convolution of size 1, after vertical pooling, yields a result of size 1. Predicted class distribution map ; S33, Enhance Features Feed into horizontal convolutional branches to enhance features After a step size of 1 and a size of 1 1, step size is 1, size is 3 1, step size is 1, size is 7 A horizontal bar convolution of size 1, after horizontal pooling, yields a result of size 1. Predicted class distribution map ; S34. After vertical and horizontal bar convolutions, spatial attention (SA) is added to obtain features. and The formula is as follows:

[0061]

[0062] in, This represents spatial attention operations. Represents the learnable weight coefficients; S35, Features and The summation is used as input features, which are then passed through a standard convolution with a stride of 1 and a size of 3×3, and finally through squeeze-and-excitation (SE) channel attention to obtain the features. The formula is as follows:

[0063] in, This represents the compression-enhancing channel attention operation. In this embodiment, the compression-enhancing channel attention adaptively learns the importance of each channel and weights the channel contributions in the feature map according to the needs of the task, thereby improving model performance. This module is divided into two parts: compression and enhancement. The compression part performs global average pooling on the input features along the channel dimension to obtain features with the same width and height and a channel count of 1. In the enhancement part, the features obtained from the compression part are passed through two fully connected layers and a sigmoid activation function to obtain the channel attention weight matrix.

[0064] S36, Features , , , The concatenation was performed using the Concat operation, and then channel dimensionality reduction was achieved using a standard convolution with a stride of 1 and a size of 3×3, resulting in a size of [size missing]. Human body semantic features The formula is as follows:

[0065] in, This indicates a splicing operation.

[0066] In a specific implementation, as a preferred embodiment of the present invention, in step S31, the feature pyramid module effectively compensates for the limitations of the deep receptive field of convolutional neural networks through parallel multi-scale pooling and feature fusion strategies, significantly improving the segmentation accuracy of the semantic segmentation model, including: For input size Features Parallel execution 1 1, 2 2, 3 3, 6 6. Adaptive average pooling operations at four scales; Bilinear interpolation is used to upsample all the scaled features after dimensionality reduction to the size of the original input features; The original input features and the four upsampled scale features are concatenated and fused along the channel dimension to obtain a size that includes both global semantics and local details. Enhanced features .

[0067] In a specific implementation, as a preferred embodiment of the present invention, in step S34, the spatial attention automatically captures important regional features by focusing on the effective information location of the features, including: The input features are subjected to global max pooling and global average pooling along the channel dimension to obtain two features with the same width and height and a channel number of 1. Perform a Concat concatenation operation on two features with the same width and height and 1 channel to obtain a feature with the same width and height and 2 channels; After performing a standard convolution with a stride of 1 and a size of 7×7 on the concatenated result, the result is input into the Sigmoid activation function to obtain the spatial attention weight matrix.

[0068] In a specific implementation, as a preferred embodiment of the present invention, step S4 includes: S41. Employ an edge branch decoder for edge features. Decoding operations include: The size is Edge features After a step size of 1 and a size of 3 Standard convolution with a stride of 1 and a size of 1. Channel reduction and bilinear interpolation of standard convolution result in a product of size 1. Features ; S42. Employ a human body parsing branch decoder to parse semantic features of the human body. Decoding operations include: The size is Human body semantic features After a step size of 1 and a size of 1 The channel reduction operation of standard convolution and the bilinear interpolation operation yield a size of 1. Features ; The size is Features After a step size of 1 and a size of 3 The channel reduction operation of a standard convolution with dimensions 3 yields a size of Features; The size is Features and size The features are concatenated using the Concat operation, and then processed with a step size of 1 and a size of 3. A standard convolution with a stride of 1 and a size of 1. The channel reduction operation of a standard convolution of size 1 yields a size of Human body semantic features ; With a step size of 1 and a size of 1 Channel reduction and bilinear interpolation of standard convolution result in a product of size 1. Features ; S43. Human body edge features are fused using the boundary feature fusion module. semantic features of human body analysis The two types of features are initially fused through element-wise summation; the fused features are then input into multiple convolutional layers to further explore the deep correlations between features, and finally, a channel attention operation is performed to obtain a feature of size [size missing]. Features ; S44, with dimensions of Features Input decoder, features After performing a standard convolution with a stride of 1 and a size of 3×3, a Dropout operation, a channel reduction operation of a standard convolution with a stride of 1 and a size of 1×1, and a bilinear interpolation operation, the resulting object has a size of [missing information]. Features .

[0069] In a specific implementation, as a preferred embodiment of the present invention, step S5 includes: S51. Construct the edge detection cross-entropy loss function The pixel-level cross-entropy loss between the edge map and the edge label map output by the edge detection branch is calculated using the following formula:

[0070] in, Indicates the first edge label in the image. Line number The actual label value of the column; if the pixel belongs to the edge, then... ,otherwise ; Represents the edge graph of the first Line number The probability that a column is predicted to be marginal; S52. Constructing the human body parsing cross-entropy loss function The pixel-level cross-entropy loss between the analytical image and the analytical label image output by the human body analysis branch is calculated using the following formula:

[0071] in, This indicates the parsing of the label graph. Line number The actual label value of the column, if the pixel belongs to the category. ,but ,otherwise ; In the analytical graph, the first... Line number Column position belongs to category The probability of; S53. Construct the cross-entropy loss function between the final parsed graph and the parsed label graph. The formula is as follows:

[0072] in, In the analytical graph, the first... Line number Column position belongs to category The probability of; S54. Constructing the Class Distribution Loss Function The loss for calculating the vertical and horizontal predicted class distribution maps and their corresponding label class distribution maps is as follows:

[0073] in, The horizontal class distribution diagram shows the first... Line number The predicted value of the column, In the vertical class distribution diagram, the first... Line number The predicted value of the column; The horizontal class distribution diagram shows the first... Line number The actual label value of the column. In the vertical class distribution diagram, the first... Line number The actual label value of the column; S54, Cross-entropy Loss Function Based on Edge Detection Human body analysis cross-entropy loss function Cross-entropy loss function and class distribution loss function Construct the overall network loss function The formula is as follows:

[0074] in, , , All of these represent learnable hyperparameters. , All represent weighting coefficients, which in this embodiment are respectively set to and .

[0075] Example The core technologies of this application are mainly divided into three key modules: First, by designing an edge-aware feature fusion module, efficient fusion of multi-scale features is achieved, solving the problems of low matching degree between shallow details and deep semantic information and high feature redundancy in traditional multi-scale fusion strategies. Second, by designing a noise-aware high-frequency enhancement module, the expressive power of edge features is further enhanced, solving the problem of insufficient spatial domain edge feature representation ability of traditional edge detection networks under low contrast conditions. Finally, based on the existing class feature guidance module, a structured enhancement class feature guidance mechanism is proposed to accurately capture the inherent spatial topological constraints and differentiated shape information of each part, solving the problems of human body part category confusion and insufficient fine-grained structure segmentation accuracy in low-contrast scenes.

[0076] This embodiment verifies the algorithm's effectiveness using the LCHP dataset. This dataset contains 3200 training images and 800 validation images, with a low-contrast image to normal image ratio of 6:4. Related experimental results are as follows: Figure 6As shown in Table 1, the results demonstrate that this algorithm can quickly and accurately segment various parts of the human body for low-contrast human images in different scenarios. Compared with the baseline network CDGNet, this algorithm improves pixel accuracy by 2.03%, average pixel accuracy by 4.24%, and average intersection-union ratio by 3.20%, fully verifying the effectiveness and superiority of the proposed method in low-contrast human body analysis tasks.

[0077] Table 1. Comparison of experimental results between the human body analysis method proposed in this invention and other methods.

[0078] By designing and introducing the above modules, the method of this invention can effectively improve the accuracy of human body resolution under low contrast conditions, and also provides an important reference for solving computer vision tasks such as pedestrian re-identification, human-object interaction detection, and virtual try-on.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance, characterized in that, include: S1. Construct a multi-scale feature extraction backbone network to extract multi-scale features from the input low-contrast human image and obtain multi-level spatial features. S2. Construct an edge detection branch, input the multi-level spatial features into the edge detection branch, and then enhance them sequentially through a frequency multilayer perceptron, a feature fusion module based on edge perception, and a high-frequency enhancement module based on noise perception to obtain high-precision human edge features. S3. Construct a human body parsing branch, input the multi-level spatial features into the human body parsing branch, and after processing by the structure-enhanced class feature guidance module, obtain the human body parsing semantic features. S4. The high-precision human body edge features and the human body parsing semantic features are fused and optimized through the boundary feature fusion module, and then processed by the decoder to output the pixel-level parsing result of the low-contrast human body.

2. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, Also includes: S5. Construct a weighted loss function for multiple losses and jointly train the low-contrast human body parsing network based on frequency domain enhancement and structured feature guidance, and iteratively optimize all parameters of the network.

3. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, In step S1, the multi-scale feature extraction backbone network adopts ResNet101, and the specific multi-scale feature extraction steps include: S11, Convert low-contrast human body images Enter to 7 A convolutional layer with 7 kernels and a stride of 2 yields a size of [size missing]. The first feature map; S12. Input the first feature map into the step size of 2, 3. The maximum pooling layer of size 3 yields a size of Second feature map ; S13, Transfer the second feature map The first residual stage is input, and after passing through 3 residual blocks, the output size is... Features ; Features The second residual stage is input, and after passing through 4 residual blocks, the output size is... Features ; Features The input is the third residual stage. After passing through 23 residual blocks, the output size is... Features ; Features The input is the fourth residual stage. After passing through 3 residual blocks, the output size is... Features .

4. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, Step S2 includes: S21. Input the multi-level spatial features obtained in step S1 into a frequency multilayer perceptron. The frequency multilayer perceptron processes the multi-level spatial features. , A Fast Fourier Transform (FFT) is performed to map the features to the frequency domain and decompose them into amplitude and phase spectra. A multilayer perceptron is used to specifically enhance the amplitude component, and the enhanced amplitude component is then combined with the original phase component. An Inverse Fast Fourier Transform (IFT) is then performed to map the enhanced features back to the spatial domain, and residual fusion is then performed to obtain the frequency-domain enhanced features. , ; S22. Input the frequency-domain enhanced features into the edge-aware feature fusion module. The feature fusion module performs feature fusion as follows: Low-level features After bilinear interpolation and high-level features Passing through steps of 1 and sizes of 1 respectively The output features are obtained by performing a grouped convolution of 1 and then concatenating the two groups. The output features are passed through two consecutive steps of size 1 and size 3. The convolution of size 3 has a stride of 1 and a size of 1. After a convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The generated weight information is combined with low-level features. and advanced features Multiplication yields the fused features, and cross-scale residual connections are used to obtain a feature of size [value missing]. Features The formula is as follows: in, This represents the learnable weight parameters. Represents the attention weight matrix. This represents the matrix dot product operation; Features By using two consecutive steps of size 1 and size 3 The convolution of size 3 has a stride of 1 and a size of 1. After a convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The generated weight information and features and frequency domain enhanced features Multiplication yields the fused features, and cross-scale residual connections are used to obtain a feature of size [value missing]. Multi-scale fusion features The formula is as follows: in, This represents the learnable weight parameters. Represents the attention weight matrix. This represents the matrix dot product operation; S23, Fusing features across multiple scales The input is a noise-aware high-frequency enhancement module. The high-frequency enhancement process performed by the module is as follows: Multi-scale fusion features After average pooling and upsampling operations, the multi-scale fused features are... After average pooling and upsampling operations Difference and extraction of high-frequency features The formula is as follows: in, This indicates the fusion of multi-scale features. After average pooling and upsampling operations; High frequency features Using steps of 1 and size of 3 respectively A standard convolution with a stride of 2, an inflation rate of 2, and a size of 3. 3-dimensional dilated convolution, simultaneously incorporating high-frequency features We perform a standard convolution with a stride of 1 and a size of 1×1, followed by average pooling with a stride of 1 and a size of 1. After a standard convolution of 1, the attention weight matrix is ​​obtained by inputting the sigmoid activation function. The guide uses bi-branch convolution to capture multi-scale features, introduces frequency domain channel attention to further optimize high-frequency features, and obtains a size of [size missing] through cross-scale residual connections. Edge features The formula is as follows: in, This indicates frequency channel attention operations. Indicates 3 3. Standard convolution operations Indicates 3 3. Dilation convolution operation, This represents the attention weight matrix.

5. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 4, characterized in that, In step S23, the frequency domain channel attention is modeled using multiple frequency components for refined calibration and enhancement of channel features, including: High frequency features The channel is divided into multiple blocks, and each block is assigned a dedicated two-dimensional discrete cosine transform component to extract the feature frequency information corresponding to each channel subset. This yields features with the same width and height and a channel number of 1. These features are then fed into a fully connected layer for learning and input into a Sigmoid activation function to obtain the frequency channel attention weight matrix.

6. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, Step S3 includes: S31. The size extracted from the last layer of the multi-scale feature extraction backbone network is... Features The input is fed into the feature pyramid module, resulting in a size of Enhanced features ; S32, Enhance Features Feed into the vertical convolution branch to enhance features Continuously passing through a step size of 1 and a size of 1 1, step size is 1, size is 3 1, step size is 1, size is 7 A vertical bar convolution of size 1, after vertical pooling, yields a result of size 1. Predicted class distribution map ; S33, Enhance Features Feed into horizontal convolutional branches to enhance features After a step size of 1 and a size of 1 1, step size is 1, size is 3 1, step size is 1, size is 7 A horizontal bar convolution of size 1, after horizontal pooling, yields a result of size 1. Predicted class distribution map ; S34. Spatial attention is added after vertical and horizontal bar convolutions to obtain features. and The formula is as follows: in, This represents spatial attention operations. Represents the learnable weight coefficients; S35, Features and The summation is used as input features, which are then passed through a standard convolution with a stride of 1 and a size of 3×3, and finally through a compression-excitation channel attention process to obtain the features. The formula is as follows: in, This indicates a compression-excitation channel attention operation; S36, Features , , , The concatenation was performed using the Concat operation, and then channel dimensionality reduction was achieved using a standard convolution with a stride of 1 and a size of 3×3, resulting in a size of [size missing]. Human body semantic features The formula is as follows: in, This indicates a splicing operation.

7. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, In step S31, the processing procedure of the feature pyramid module includes: For input size Features Parallel execution 1 1, 2 2, 3 3, 6 6. Adaptive average pooling operations at four scales; Bilinear interpolation is used to upsample all the scaled features after dimensionality reduction to the size of the original input features; The original input features and the four upsampled scale features are concatenated and fused along the channel dimension to obtain a size that includes both global semantics and local details. Enhanced features .

8. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, In step S34, the spatial attention automatically captures important regional features by focusing on the effective information locations on the features, including: The input features are subjected to global max pooling and global average pooling along the channel dimension to obtain two features with the same width and height and a channel number of 1. Perform a Concat concatenation operation on two features with the same width and height and 1 channel to obtain a feature with the same width and height and 2 channels; After performing a standard convolution with a stride of 1 and a size of 7×7 on the concatenated result, the result is input into the Sigmoid activation function to obtain the spatial attention weight matrix.

9. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, Step S4 includes: S41. Employ an edge branch decoder for edge features. Decoding operations include: The size is Edge features After a step size of 1 and a size of 3 Standard convolution with a stride of 1 and a size of 1. Channel reduction and bilinear interpolation of standard convolution result in a product of size 1. Features ; S42. Employ a human body parsing branch decoder to parse semantic features of the human body. Decoding operations include: The size is Human body semantic features After a step size of 1 and a size of 1 The channel reduction operation of standard convolution and the bilinear interpolation operation yield a size of 1. Features ; The size is Features After a step size of 1 and a size of 3 The channel reduction operation of a standard convolution with dimensions 3 yields a size of Features; The size is Features and size The features are concatenated using the Concat operation, and then processed with a step size of 1 and a size of 3. A standard convolution with a stride of 1 and a size of 1. The channel reduction operation of a standard convolution of size 1 yields a size of Human body semantic features ; With a step size of 1 and a size of 1 Channel reduction and bilinear interpolation of standard convolution result in a product of size 1. Features ; S43. Human body edge features are fused using the boundary feature fusion module. semantic features of human body analysis The two types of features are initially fused through element-wise summation; the fused features are then input into multiple convolutional layers to further explore the deep correlations between features, and finally, a channel attention operation is performed to obtain a feature of size [size missing]. Features ; S44, with dimensions of Features Input decoder, features After performing a standard convolution with a stride of 1 and a size of 3×3, a Dropout operation, a channel reduction operation of a standard convolution with a stride of 1 and a size of 1×1, and a bilinear interpolation operation, the resulting object has a size of [size missing]. Features .

10. The low-contrast human body resolution method based on frequency domain enhancement and structured feature guidance according to claim 1, characterized in that, Step S5 includes: S51. Construct the edge detection cross-entropy loss function The pixel-level cross-entropy loss between the edge map and the edge label map output by the edge detection branch is calculated using the following formula: in, Indicates the first edge label in the image. Line number The actual label value of the column; if the pixel belongs to the edge, then... ,otherwise ; Represents the edge graph of the first Line number The probability that a column is predicted to be marginal; S52. Constructing the Human Body Parsing Cross-Entropy Loss Function The pixel-level cross-entropy loss between the analytical image and the analytical label image output by the human body analysis branch is calculated using the following formula: in, This indicates the parsing of the label graph. Line number The actual label value of the column, if the pixel belongs to the category. ,but ,otherwise ; In the analytical graph, the first Line number Column position belongs to category The probability of; S53. Construct the cross-entropy loss function between the final parsed graph and the parsed label graph. The formula is as follows: in, In the analytical graph, the first Line number Column position belongs to category The probability of; S54. Constructing the Class Distribution Loss Function The loss for calculating the vertical and horizontal predicted class distribution maps and their corresponding label class distribution maps is as follows: in, The horizontal class distribution diagram shows the first... Line number The predicted value of the column, In the vertical class distribution diagram, the first Line number The predicted value of the column; The horizontal class distribution diagram shows the first... Line number The actual label value of the column. In the vertical class distribution diagram, the first Line number The actual label value of the column; S54, Cross-entropy Loss Function Based on Edge Detection Human body analysis cross-entropy loss function Cross-entropy loss function and class distribution loss function Construct the overall network loss function The formula is as follows: in, , , All of these represent learnable hyperparameters. , All of these represent weighting coefficients.

Images