Multi-dimensional visual semantic perception and target classification method and system for complex scene

Abstract

The application discloses a multi-dimensional visual semantic perception and target classification method and system for complex scenes, extracts multi-scale features through a shared backbone network, avoids repeated feature calculation of two independent models of face anti-counterfeiting and scene classification, and significantly improves the system processing capacity under high concurrency scenes; a dual-domain cross analysis mechanism of spatial domain texture features and frequency domain high-frequency residual features is introduced in the local face anti-counterfeiting perception branch, which can effectively capture the high-frequency microscopic counterfeit traces unique to AI generated images, and the detection ability for deep counterfeit images is significantly enhanced compared with the traditional scheme which only relies on spatial domain features; the global multi-dimensional semantic perception branch effectively suppresses the interference of complex background noise on the classification result through context aggregation processing of channel attention and spatial attention, and improves the scene classification robustness of low-quality images such as children's paintings and life photos; the application significantly improves the comprehensive judgment accuracy of multi-dimensional visual perception.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a method and system for multidimensional visual semantic perception and target classification in complex scenes. Background Technology

[0002] With the rapid development of generative artificial intelligence (AIGC) and multimodal large models, computer vision technology is playing an increasingly important role in image recognition and content auditing in daily applications. However, current technology still has significant shortcomings when facing these two highly challenging image recognition needs.

[0003] The first type of demand is for high-precision local biometric recognition and authentication. In recent years, online fraud and deepfake technology have become increasingly rampant, with AI-generated fake faces and adversarial examples emerging one after another. Traditional face recognition algorithms mainly rely on spatial domain texture and geometric features for identity matching, lacking the ability to perceive pixel-level forgery traces and high-frequency microscopic features. For example, AI-generated face images may be highly similar to real faces in the spatial domain, but they often have specific high-frequency noise distribution patterns in the frequency domain. Traditional methods struggle to capture such frequency domain anomalies, resulting in low detection accuracy for deepfake images. Although some existing anti-spoofing solutions introduce frequency domain analysis, it is usually only used as an independent post-processing module and is not used for joint reasoning with global semantic understanding. Therefore, it cannot effectively distinguish between composite fraudulent images such as "real face + fake background" and "fake face + real background" in complex scenarios.

[0004] The second type of requirement is multi-category, cross-scenario general semantic understanding and content compliance auditing. In interactive scenarios such as children's social interaction and AI creation, images uploaded by users (especially young children) often have complex backgrounds, varying lighting, and cluttered objects. Existing scene classifiers are easily affected by background noise and prone to misjudgment when processing such images. At the same time, existing solutions usually only output a single scene category label, failing to perform multi-label joint judgment on multiple objects, scene types, and emotional tendencies in the image, and even more so failing to address the security auditing needs of specific subjects (such as faces) in the image.

[0005] Currently, existing technologies typically deploy "face anti-spoofing recognition" and "general image target classification" as two independent models. This approach has the following significant drawbacks: First, it results in severe waste of computational resources. The two models repeatedly extract basic visual features such as edges and textures, leading to a large amount of redundant computation in the backbone network. Under massive concurrent requests, this causes excessive server load and large inference latency. Second, features are isolated and lack collaboration. Traditional models cannot extract global scene semantics while using local face anti-spoofing features to assist in judging the overall compliance and authenticity of images. This results in low overall interception accuracy when the system encounters composite fraudulent or non-compliant images with "fake faces + real backgrounds". In addition, existing technologies lack a face anti-spoofing scheme that cross-fuses spatial domain texture features and frequency domain high-frequency residual features. They also lack a collaborative decision-making architecture that dynamically aligns the weights of local anti-spoofing features and global semantic features through a cross-attention mechanism, making it difficult to efficiently complete multi-dimensional visual perception tasks within a unified framework. Summary of the Invention

[0006] To address the aforementioned technical problems, this technical solution provides a method and system for multidimensional visual semantic perception and target classification in complex scenarios, solving the problems mentioned in the background.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect of the present invention, a multi-dimensional visual semantic perception and target classification method for complex scenes is provided, comprising: Receive the input image to be recognized, perform size normalization preprocessing on the input image, and obtain a preprocessed image of standard size; The preprocessed image is fed into a shared backbone network for forward inference to extract multi-scale shared feature maps. The shared feature map is input into the local face anti-spoofing perception branch and the global multidimensional semantic perception branch for parallel processing. The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features and frequency domain high-frequency residual features, outputting the face identity credibility and forgery risk score. The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map, suppresses background noise interference, and outputs multi-label scene classification results and confidence probabilities of each label. By using a cross-attention bridging mechanism, the forgery risk features output by the local face anti-spoofing perception branch and the scene semantic features output by the global multi-dimensional semantic perception branch are dynamically weighted, aligned, and fused with the Hadamard product to generate a comprehensive judgment vector. Based on the joint distribution of the face forgery risk score and the confidence probability of scene semantic labels in the comprehensive judgment vector, a structured multidimensional visual perception judgment result is output.

[0008] Preferably, the step of performing size normalization preprocessing on the input image to obtain a preprocessed image of standard size specifically includes: Receive an input image of any resolution, calculate the ratio of the original width to the original height of the input image, and determine whether the ratio falls within the preset target aspect ratio range. If the target falls within the target area, the input image is scaled proportionally based on the target's shorter side, so that the shorter side of the scaled image is equal to the preset standard size side length. The longer side is scaled synchronously according to the original aspect ratio, and the area exceeding the standard size is cropped from the center to obtain a standard size image. If the image does not fall within the specified size, the input image is scaled proportionally so that the long side is equal to the preset standard size side length, and the short side is scaled synchronously according to the original aspect ratio. The area that is smaller than the standard size is then filled with mirrored edge pixel values ​​to obtain a standard size image. Subtract the preset image mean and divide by the preset image standard deviation for each pixel channel of the standard-sized image to normalize the pixel values ​​and output the preprocessed image. The formula for normalizing pixel values ​​is: ; in, The original pixel value of any pixel in a certain channel of the standard-sized image. This is the preset image mean for this channel. This is the preset image standard deviation for this channel. These are the normalized pixel values.

[0009] Preferably, the step of feeding the preprocessed image into the shared backbone network for forward inference and extracting multi-scale shared feature maps specifically includes: The preprocessed image is sequentially fed into the first, second, and third convolution stages of the shared backbone network for layer-by-layer forward convolution operations. In the first convolution stage, the preprocessed image is subjected to two convolution and batch normalization operations with a stride of 2, and the first scale feature map is output. The spatial resolution of the first scale feature map is one-quarter of that of the preprocessed image. In the second convolution stage, the first-scale feature map is subjected to two convolution and batch normalization operations with a stride of 2, and the second-scale feature map is output. The spatial resolution of the second-scale feature map is one-sixteenth of that of the preprocessed image. In the third convolution stage, the second-scale feature map is subjected to two convolution and batch normalization operations with a stride of 2 to output the third-scale feature map. The spatial resolution of the third-scale feature map is one-sixtieth of the preprocessed image. The first-scale feature map, the second-scale feature map, and the third-scale feature map are concatenated along the channel dimension to obtain a multi-scale shared feature map.

[0010] Preferably, the local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features, specifically including: The multi-scale shared feature map is input into the object detection operator. The object detection operator generates candidate face regions on the multi-scale shared feature map with preset anchor box scale and aspect ratio. For each candidate region, the face existence probability score and bounding box regression offset are calculated. Candidate regions with probability scores higher than preset thresholds are retained and non-maximum suppression is performed to remove overlapping boxes. The face bounding box coordinates are output. Based on the coordinates of the face bounding box, the corresponding face region feature map is cropped from the multi-scale shared feature map. The face region feature map is then fed into the feature embedding sub-network trained by the triple loss function. The feature embedding sub-network sequentially performs depthwise separable convolution, channel attention weighting, and global average pooling on the face region feature map, and outputs a fixed-dimensional face identity embedding vector. Calculate the Euclidean distance between the face identity embedding vector and each vector in the preset registration identity embedding vector library. The formula for calculating the Euclidean distance is as follows: ; in, Embedding vectors for facial identity, For each registered identity, embed any registered vector from the vector library. The dimension of the embedded vector. and The two vectors are respectively in the th... Values ​​on the dimension, The Euclidean distance between the two; The identity corresponding to the minimum distance is taken as the recognition result, and the reciprocal of the minimum distance is used as the face identity credibility. The formula for calculating face identity credibility is: ; in, This is the minimum Euclidean distance between the face identity embedding vector and all vectors in the registration identity embedding vector library. To prevent small constants from being divided by zero, To assess the credibility of facial recognition.

[0011] Preferably, the local face anti-spoofing perception branch locates the face region in the shared feature map and extracts high-frequency residual features in the frequency domain, outputting a forgery risk score, specifically including: The facial region feature map is converted from a channel dimension to a grayscale single-channel image. A two-dimensional fast Fourier transform is performed on the grayscale single-channel image to map the spatial domain pixel values ​​to a frequency domain complex number representation. In the complex representation of the frequency domain, the center point of the image is taken as the origin. Frequency domain components with radii greater than the preset low-frequency cutoff radius are retained, while frequency domain components with radii less than or equal to the preset low-frequency cutoff radius are set to zero, thus obtaining the high-frequency retained spectrum. Perform a two-dimensional inverse fast Fourier transform on the high-frequency preserved spectrum to restore the frequency domain complex representation to the spatial domain image, thus obtaining the high-frequency reconstructed image; The high-frequency reconstructed image is compared with the grayscale single-channel image pixel by pixel to obtain the high-frequency residual image. The formula for calculating the high-frequency residual image is as follows: ; in, It is a grayscale single-channel image. It is a two-dimensional fast Fourier transform operator. It is a two-dimensional inverse fast Fourier transform operator. For high-frequency retention filters, the radius in the frequency domain is less than or equal to the preset low-frequency cutoff radius. The position value is 0, and the radius is greater than 0. The position value is 1. This is a high-frequency residual plot; The high-frequency residual map is fed into a spoofing discriminant subnetwork trained by a binary classification cross-entropy loss function. The spoofing discriminant subnetwork performs three layers of convolution and global average pooling on the high-frequency residual map and outputs a single spoofing probability value, which is used as a spoofing risk score.

[0012] Preferably, the global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map to suppress background noise interference, specifically including: The multi-scale shared feature map is input into the global average pooling layer, and the mean value in the spatial dimension is calculated for each channel of the multi-scale shared feature map to obtain the channel description vector. The channel description vector is sequentially fed into two fully connected layers and a sigmoid activation function to generate a channel attention weight vector. The formula for calculating the channel attention weight vector is as follows: ; in, For multi-scale shared feature maps, This is a global average pooling operation. and These are the weight matrices for the two fully connected layers. It is the ReLU activation function. It is the Sigmoid activation function. This is the channel attention weight vector; Each element of the channel attention weight vector corresponds to a channel of the multi-scale shared feature map; The channel attention weight vector is multiplied with the multi-scale shared feature map channel by channel to obtain the channel-weighted feature map. Adaptive max pooling and adaptive average pooling are performed on the channel-weighted feature map in the spatial dimension. The two pooling results are concatenated in the channel dimension and then fed into a convolutional layer and a sigmoid activation function to generate a spatial attention weight map. The spatial attention weight map and the channel weighted feature map are multiplied pixel-by-pixel in the spatial dimension to obtain a global semantic feature map after context aggregation.

[0013] Preferably, the output of the multi-label scene classification result and the confidence probability of each label specifically includes: The global semantic feature map is fed into the multi-label classification head. The multi-label classification head performs global average pooling, two fully connected layers and a sigmoid activation function on the global semantic feature map in sequence, and outputs a probability vector of the same length as the preset label set. Each element in the probability vector corresponds to the independent existence probability of a scene label in the preset label set. Each element in the probability vector is compared with a preset independent label threshold. Labels with probability values ​​higher than the corresponding threshold are marked as positive sample labels, and labels with probability values ​​lower than the corresponding threshold are marked as negative sample labels. The multi-label scene classification result is composed of all positive sample labels and their corresponding probability values, and the probability values ​​corresponding to all positive sample labels are used as the confidence probabilities of each label. The formula for calculating the confidence probability of each label is as follows: ; in, This is a global semantic feature map processed by context aggregation. This is a global average pooling operation. For the first The weight vector of the fully connected layer corresponding to each scene label For the first The bias term corresponding to each scene label It is the Sigmoid activation function. Let be the confidence probability of the j-th scene label; During the training phase, the multi-label classification head uses the multi-label classification cross-entropy loss function as the optimization objective.

[0014] Preferably, the step of dynamically aligning the forgery risk features output by the local face anti-spoofing perception branch and fusing the scene semantic features output by the global multi-dimensional semantic perception branch with dynamic weights and Hadamard product through the cross-attention bridging mechanism to generate a comprehensive judgment vector specifically includes: The forgery risk features output by the local face anti-counterfeiting perception branch are used as the query vector, and the scene semantic features output by the global multidimensional semantic perception branch are used as the key vector and value vector. Perform linear projections on the query vector and key vector respectively to obtain the query projection vector and key projection vector. Calculate the dot product between the transposes of the query projection vector and key projection vector, divide the dot product by a preset scaling factor, and then input... The function yields the attention weight distribution, and the formula for calculating the attention weight distribution is: ; in, To fabricate risk characteristics, and These are scene semantic features, , , These are the linear projection weight matrices for the query, key, and value, respectively. Let be the dimension of the key vector. The preset scaling factor, for Normalization function, Aligned global semantic features; The attention weight distribution is summed with the value vector to obtain the aligned global semantic features; The forgery risk features and the aligned global semantic features are then subjected to an element-wise Hadamard product operation along the feature dimension to obtain a fused feature vector. The formula for calculating the Hadamard product is as follows: ; in, This represents the element-wise Hadamard product operation. To fuse feature vectors; The fused feature vector is fed into a fully connected layer and a Tanh activation function to output a comprehensive decision vector.

[0015] Preferably, the step of outputting a structured multidimensional visual perception judgment result based on the joint distribution of the face forgery risk score and the scene semantic label confidence probability in the comprehensive judgment vector specifically includes: The face forgery risk score and the confidence probability of semantic labels for each scene are extracted from the comprehensive judgment vector, and then mapped to... interval; If the face forgery risk score is higher than the preset forgery judgment threshold, the judgment result will be marked as high risk, and the face identity credibility, forgery risk score, and multi-label scene classification results with the highest confidence probability will be ranked first. Each label and its confidence probability are encapsulated into a structured JSON output in the first format; If the face forgery risk score is not higher than the forgery judgment threshold and there are non-compliant labels with a confidence probability higher than the preset compliance threshold in the multi-label scene classification results, the judgment result will be marked as non-compliant, and the face identity credibility, forgery risk score, all scene labels and their confidence probabilities will be encapsulated into a structured JSON output in the second format. If the face forgery risk score is not higher than the forgery judgment threshold and the confidence probability of all scene labels is not higher than the compliance threshold, the judgment result will be marked as compliant and the face identity credibility, forgery risk score, all scene labels and their confidence probabilities will be encapsulated into a third-format structured JSON output.

[0016] Preferably, the local face anti-spoofing perception branch locates the face region in the shared feature map, specifically including: The multi-scale shared feature map is fed into the feature pyramid network of the target detection operator, and the feature pyramid network constructs detection layers of different scales on the first-scale feature map, the second-scale feature map and the third-scale feature map respectively; On each detection layer, anchor boxes with three aspect ratios are generated based on a preset anchor box scale. For each anchor box, the confidence score of the presence of a face, the center offset of the bounding box, and the scaling ratio of the width and height are predicted. For all candidate boxes output by the detection layer, calculate the intersection-union ratio (IU) between them and the ground truth boxes, and retain candidate boxes with IU higher than a preset threshold as positive candidate boxes. Non-maximum suppression is performed on all positive candidate boxes. The overlap threshold is set to a preset IOU threshold. Candidate boxes with an overlap exceeding the IOU threshold with the highest-scoring box are removed in turn, and the final face bounding box coordinates are output.

[0017] In a second aspect of the invention, a multi-dimensional visual semantic perception and target classification system for complex scenes is also provided, comprising: Receiving module: Receives the input image to be recognized, performs size normalization preprocessing on the input image, and obtains a preprocessed image of standard size; Extraction module: Feeds the preprocessed image into the shared backbone network for forward inference and extracts multi-scale shared feature maps; The first output module inputs the shared feature map into the local face anti-spoofing perception branch and the global multidimensional semantic perception branch for parallel processing. The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features and frequency domain high-frequency residual features, outputting the face identity credibility and forgery risk score. The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map, suppresses background noise interference, and outputs multi-label scene classification results and confidence probabilities of each label. Generation module: Through a cross-attention bridging mechanism, the forgery risk features output by the local face anti-spoofing perception branch and the scene semantic features output by the global multi-dimensional semantic perception branch are dynamically weighted, aligned, and fused with the Hadamard product to generate a comprehensive judgment vector. The second output module outputs a structured, multi-dimensional visual perception judgment result based on the joint distribution of the face forgery risk score and the confidence probability of scene semantic labels in the comprehensive judgment vector.

[0018] Compared with existing technologies, this invention provides a multi-dimensional visual semantic perception and target classification method and system for complex scenarios, which has the following beneficial effects: This invention extracts multi-scale features through a shared backbone network, avoiding redundant feature calculations in two independent models: face anti-spoofing and scene classification, significantly improving system processing capabilities in high-concurrency scenarios. In the local face anti-spoofing perception branch, this invention introduces a dual-domain cross-analysis mechanism combining spatial domain texture features and frequency domain high-frequency residual features. This effectively captures high-frequency microscopic forgery traces unique to AI-generated images, significantly enhancing the detection capability for deepfake images compared to traditional schemes relying solely on spatial domain features. Simultaneously, the global multidimensional semantic perception branch effectively suppresses the interference of complex background noise on classification results through contextual aggregation processing of channel attention and spatial attention, improving robustness for scene classification of low-quality images such as children's drawings and everyday photos. Furthermore, this invention achieves dynamic weight alignment and Hadamard product fusion of local anti-spoofing features and global semantic features through a cross-attention bridging mechanism. This enables the system to comprehensively evaluate face authenticity and content compliance when facing complex fraudulent images such as "fake faces spliced ​​with real backgrounds," significantly improving the overall accuracy of multidimensional visual perception. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the method flow for S101-S105 in this invention; Figure 2 This is a schematic diagram of the method flow for S201-S204 in this invention; Figure 3 This is a schematic diagram of the method flow for S301-S305 in this invention; Figure 4This is a schematic diagram of the method flow for S401-S404 in this invention; Figure 5 This is a schematic diagram of the method flow for S501-S505 in this invention. Detailed Implementation

[0020] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0021] Example 1 Please refer to Figure 1 As shown, in a first aspect of the present invention, a multi-dimensional visual semantic perception and target classification method for complex scenes is provided, comprising: S101. Receive the input image to be recognized, perform size normalization preprocessing on the input image, and obtain a preprocessed image of standard size; S102. Feed the preprocessed image into the shared backbone network for forward inference and extract multi-scale shared feature maps. S103. The shared feature map is input into the local face anti-spoofing perception branch and the global multidimensional semantic perception branch for parallel processing. The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features and frequency domain high-frequency residual features, and outputs the face identity credibility and forgery risk score. The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map, suppresses background noise interference, and outputs multi-label scene classification results and confidence probability of each label. S104. Through the cross-attention bridging mechanism, the forgery risk features output by the local face anti-counterfeiting perception branch and the scene semantic features output by the global multi-dimensional semantic perception branch are dynamically weighted, aligned, and fused with the Hadamard product to generate a comprehensive judgment vector. S105. Based on the joint distribution of the face forgery risk score and the scene semantic label confidence probability in the comprehensive judgment vector, output a structured multidimensional visual perception judgment result.

[0022] As will be understood by those skilled in the art, this invention extracts multi-scale features through a shared backbone network, avoiding redundant feature calculations in two independent models: face anti-spoofing and scene classification, thus significantly improving the system's processing capabilities in high-concurrency scenarios. This invention introduces a dual-domain cross-analysis mechanism of spatial domain texture features and frequency domain high-frequency residual features in the local face anti-spoofing perception branch, effectively capturing high-frequency microscopic forgery traces unique to AI-generated images. Compared to traditional schemes relying solely on spatial domain features, this significantly enhances the detection capability for deepfake images. Simultaneously, the global multidimensional semantic perception branch effectively suppresses the interference of complex background noise on classification results through contextual aggregation processing of channel attention and spatial attention, improving the robustness of scene classification for low-quality images such as children's drawings and everyday photos. Furthermore, this invention achieves dynamic weight alignment and Hadamard product fusion of local anti-spoofing features and global semantic features through a cross-attention bridging mechanism, enabling the system to comprehensively evaluate the authenticity of the face and the compliance of the content when facing composite fraudulent images such as "fake faces spliced ​​with real backgrounds," significantly improving the overall accuracy of multidimensional visual perception.

[0023] Please refer to Figure 2 As shown, the input image undergoes size normalization preprocessing to obtain a preprocessed image of standard size, specifically including: S201. Receive an input image of arbitrary resolution, calculate the ratio of the original width to the original height of the input image, and determine whether the ratio falls within the preset target aspect ratio range. S202. If it falls into the target area, the input image is scaled proportionally based on the target short side so that the short side of the scaled image is equal to the preset standard size side length. The long side is scaled synchronously according to the original aspect ratio. The area exceeding the standard size is cropped in the center to obtain the standard size image. S203. If it does not fall within the range, the input image is scaled proportionally so that the long side is equal to the preset standard size side length, and the short side is scaled synchronously according to the original aspect ratio. The area that is not within the standard size is filled by mirroring the edge pixel values ​​to obtain a standard size image. S204. Subtract the preset image mean from each pixel channel of the standard-size image and divide by the preset image standard deviation to complete the pixel value normalization and output the preprocessed image. The formula for normalizing pixel values ​​is: ; in, This represents the original pixel value of any pixel in a standard-sized image on a specific channel. This is the preset image mean for this channel. This is the preset image standard deviation for this channel. These are the normalized pixel values.

[0024] Please refer to Figure 3 As shown, the preprocessed image is fed into a shared backbone network for forward inference to extract multi-scale shared feature maps, specifically including: S301. The preprocessed image is sequentially fed into the first convolution stage, the second convolution stage and the third convolution stage of the shared backbone network for layer-by-layer forward convolution operation. S302. In the first convolution stage, the preprocessed image is subjected to two convolution and batch normalization operations with a stride of 2 to output the first scale feature map. The spatial resolution of the first scale feature map is one-quarter of that of the preprocessed image. S303. In the second convolution stage, the first scale feature map is subjected to two convolution and batch normalization operations with a stride of 2, and the second scale feature map is output. The spatial resolution of the second scale feature map is one-sixteenth of that of the preprocessed image. S304. In the third convolution stage, the second-scale feature map is subjected to two convolution and batch normalization operations with a stride of 2 to output the third-scale feature map. The spatial resolution of the third-scale feature map is one-sixtieth of the preprocessed image. S305. The first-scale feature map, the second-scale feature map, and the third-scale feature map are concatenated along the channel dimension to obtain a multi-scale shared feature map.

[0025] Please refer to Figure 4 As shown, the local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features, specifically including: S401. Input the multi-scale shared feature map into the object detection operator. The object detection operator generates candidate face regions on the multi-scale shared feature map with a preset anchor box scale and aspect ratio. For each candidate region, calculate the face existence probability score and bounding box regression offset. Retain candidate regions with probability scores higher than a preset threshold and perform non-maximum suppression to remove overlapping boxes. Output the face bounding box coordinates. S402. Based on the coordinates of the face bounding box, the corresponding face region feature map is cropped from the multi-scale shared feature map. The face region feature map is then fed into the feature embedding sub-network trained by the triple loss function. The feature embedding sub-network sequentially performs depthwise separable convolution, channel attention weighting, and global average pooling on the face region feature map, and outputs a fixed-dimensional face identity embedding vector. S403. Calculate the Euclidean distance between the face identity embedding vector and each vector in the preset registration identity embedding vector library. The formula for calculating the Euclidean distance is: ; in, Embedding vectors for facial identity, For each registered identity, embed any registered vector from the vector library. The dimension of the embedded vector. and The two vectors are respectively in the th... Values ​​on the dimension, The Euclidean distance between the two; S404. Take the identity corresponding to the minimum distance as the recognition result, and use the reciprocal of the minimum distance as the face identity credibility. The formula for calculating face identity credibility is: ; in, This is the minimum Euclidean distance between the face identity embedding vector and all vectors in the registration identity embedding vector library. To prevent small constants from being divided by zero, To assess the credibility of facial recognition.

[0026] Please refer to Figure 5 As shown, the local face anti-spoofing perception branch locates the face region in the shared feature map and extracts high-frequency residual features in the frequency domain, outputting a forgery risk score, specifically including: S501. Convert the face region feature map from the channel dimension to a grayscale single-channel image, perform a two-dimensional fast Fourier transform on the grayscale single-channel image, and map the spatial domain pixel values ​​to the frequency domain complex number representation. S502. In the complex representation of the frequency domain, with the center point of the image as the origin, retain the frequency domain components with a radius greater than the preset low-frequency cutoff radius, and set the frequency domain components with a radius less than or equal to the preset low-frequency cutoff radius to zero to obtain the high-frequency retained spectrum. S503. Perform a two-dimensional inverse fast Fourier transform on the high-frequency preserved spectrum to restore the frequency domain complex representation to the spatial domain image, and obtain the high-frequency reconstructed image. S504. Perform pixel-by-pixel difference calculation between the high-frequency reconstructed image and the grayscale single-channel image to obtain the high-frequency residual image. The calculation formula for the high-frequency residual image is as follows: ; in, It is a grayscale single-channel image. It is a two-dimensional fast Fourier transform operator. It is a two-dimensional inverse fast Fourier transform operator. For high-frequency retention filters, the radius in the frequency domain is less than or equal to the preset low-frequency cutoff radius. The position value is 0, and the radius is greater than 0. The position value is 1. This is a high-frequency residual plot; S505. Feed the high-frequency residual map into the spoofing discriminant sub-network trained by the binary classification cross-entropy loss function. The spoofing discriminant sub-network performs three layers of convolution and global average pooling on the high-frequency residual map and outputs a single spoofing probability value. The spoofing probability value is used as the spoofing risk score.

[0027] The global multidimensional semantic perception branch performs global context aggregation on the shared feature maps to suppress background noise interference, specifically including: The multi-scale shared feature map is input into the global average pooling layer, and the mean value in the spatial dimension is calculated for each channel of the multi-scale shared feature map to obtain the channel description vector. The channel description vector is sequentially fed into two fully connected layers and a sigmoid activation function to generate a channel attention weight vector. The formula for calculating the channel attention weight vector is as follows: ; in, For multi-scale shared feature maps, This is a global average pooling operation. and These are the weight matrices for the two fully connected layers. It is the ReLU activation function. It is the Sigmoid activation function. This is the channel attention weight vector; Each element of the channel attention weight vector corresponds to a channel of the multi-scale shared feature map; The channel attention weight vector is multiplied with the multi-scale shared feature map channel by channel to obtain the channel-weighted feature map. Adaptive max pooling and adaptive average pooling are performed on the channel-weighted feature map in the spatial dimension. The two pooling results are concatenated in the channel dimension and then fed into a convolutional layer and a sigmoid activation function to generate a spatial attention weight map. The spatial attention weight map and the channel weighted feature map are multiplied pixel-by-pixel in the spatial dimension to obtain a global semantic feature map after context aggregation.

[0028] Output the multi-label scene classification results and the confidence probability of each label, specifically including: The global semantic feature map is fed into the multi-label classification head. The multi-label classification head performs global average pooling, two fully connected layers and a sigmoid activation function on the global semantic feature map in sequence, and outputs a probability vector of the same length as the preset label set. Each element in the probability vector corresponds to the independent existence probability of a scene label in the preset label set. Each element in the probability vector is compared with a preset independent label threshold. Labels with probability values ​​higher than the corresponding threshold are marked as positive sample labels, and labels with probability values ​​lower than the corresponding threshold are marked as negative sample labels. The multi-label scene classification result is composed of all positive sample labels and their corresponding probability values, and the probability values ​​corresponding to all positive sample labels are used as the confidence probabilities of each label. The formula for calculating the confidence probability of each label is as follows: ; in, This is a global semantic feature map processed by context aggregation. This is a global average pooling operation. For the first The weight vector of the fully connected layer corresponding to each scene label For the first The bias term corresponding to each scene label It is the Sigmoid activation function. Let be the confidence probability of the j-th scene label; During the training phase, the multi-label classification head uses the multi-label classification cross-entropy loss function as the optimization objective.

[0029] A cross-attention bridging mechanism is used to dynamically align the forgery risk features output from the local face anti-spoofing perception branch with the scene semantic features output from the global multi-dimensional semantic perception branch, and fuse them with the Hadamard product to generate a comprehensive judgment vector, specifically including: The forgery risk features output by the local face anti-counterfeiting perception branch are used as the query vector, and the scene semantic features output by the global multidimensional semantic perception branch are used as the key vector and value vector. Perform linear projections on the query vector and key vector respectively to obtain the query projection vector and key projection vector. Calculate the dot product between the transposes of the query projection vector and key projection vector, divide the dot product by a preset scaling factor, and then input... The function yields the attention weight distribution, and the formula for calculating the attention weight distribution is: ; in, To fabricate risk characteristics, and These are scene semantic features, , , These are the linear projection weight matrices for the query, key, and value, respectively. Let be the dimension of the key vector. The preset scaling factor, for Normalization function, Aligned global semantic features; The attention weight distribution is summed with the value vector to obtain the aligned global semantic features; The forgery risk features and the aligned global semantic features are then subjected to an element-wise Hadamard product operation along the feature dimension to obtain a fused feature vector. The formula for calculating the Hadamard product is as follows: ; in, This represents the element-wise Hadamard product operation. To fuse feature vectors; The fused feature vector is fed into a fully connected layer and a Tanh activation function to output a comprehensive decision vector.

[0030] Based on the joint distribution of the face forgery risk score and the scene semantic label confidence probability in the comprehensive judgment vector, a structured multidimensional visual perception judgment result is output, specifically including: The face forgery risk score and the confidence probability of semantic labels for each scene are extracted from the comprehensive judgment vector, and then mapped to... interval; If the face forgery risk score is higher than the preset forgery judgment threshold, the judgment result will be marked as high risk, and the face identity credibility, forgery risk score, and multi-label scene classification results with the highest confidence probability will be ranked first. Each label and its confidence probability are encapsulated into a structured JSON output in the first format; If the face forgery risk score is not higher than the forgery judgment threshold and there are non-compliant labels with a confidence probability higher than the preset compliance threshold in the multi-label scene classification results, the judgment result will be marked as non-compliant, and the face identity credibility, forgery risk score, all scene labels and their confidence probabilities will be encapsulated into a structured JSON output in the second format. If the face forgery risk score is not higher than the forgery judgment threshold and the confidence probability of all scene labels is not higher than the compliance threshold, the judgment result will be marked as compliant and the face identity credibility, forgery risk score, all scene labels and their confidence probabilities will be encapsulated into a third-format structured JSON output.

[0031] The local face anti-spoofing perception branch locates the face region in the shared feature map, specifically including: The multi-scale shared feature map is fed into the feature pyramid network of the target detection operator. The feature pyramid network constructs detection layers of different scales on the first-scale feature map, the second-scale feature map, and the third-scale feature map, respectively. On each detection layer, anchor boxes with three aspect ratios are generated based on a preset anchor box scale. For each anchor box, the confidence score of the presence of a face, the center offset of the bounding box, and the scaling ratio of the width and height are predicted. For all candidate boxes output by the detection layer, calculate the intersection-union ratio (IU) between them and the ground truth boxes, and retain candidate boxes with IU higher than a preset threshold as positive candidate boxes. Non-maximum suppression is performed on all positive candidate boxes. The overlap threshold is set to the preset IOU threshold. Candidate boxes with an overlap exceeding the IOU threshold with the highest-scoring box are removed in turn, and the final face bounding box coordinates are output.

[0032] In a second aspect of the invention, a multi-dimensional visual semantic perception and target classification system for complex scenes is also provided, comprising: Receiving module: Receives the input image to be recognized, performs size normalization preprocessing on the input image, and obtains a preprocessed image of standard size; Extraction module: Feeds the preprocessed image into the shared backbone network for forward inference and extracts multi-scale shared feature maps; The first output module inputs the shared feature map into the local face anti-spoofing perception branch and the global multidimensional semantic perception branch for parallel processing. The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features and frequency domain high-frequency residual features, outputting the face identity credibility and forgery risk score. The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map, suppresses background noise interference, and outputs multi-label scene classification results and confidence probabilities of each label. Generation module: Through a cross-attention bridging mechanism, the forgery risk features output by the local face anti-spoofing perception branch and the scene semantic features output by the global multi-dimensional semantic perception branch are dynamically weighted, aligned, and fused with the Hadamard product to generate a comprehensive judgment vector. The second output module outputs a structured, multi-dimensional visual perception judgment result based on the joint distribution of the face forgery risk score and the confidence probability of scene semantic labels in the comprehensive judgment vector.

[0033] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A multi-dimensional visual semantic perception and target classification method for complex scenes, characterized in that, include: Receive the input image to be recognized, perform size normalization preprocessing on the input image, and obtain a preprocessed image of standard size; The preprocessed image is fed into a shared backbone network for forward inference to extract multi-scale shared feature maps. The shared feature map is input into the local face anti-spoofing perception branch and the global multidimensional semantic perception branch for parallel processing. The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features and frequency domain high-frequency residual features, outputting the face identity credibility and forgery risk score. The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map, suppresses background noise interference, and outputs multi-label scene classification results and confidence probabilities of each label. By using a cross-attention bridging mechanism, the forgery risk features output by the local face anti-spoofing perception branch and the scene semantic features output by the global multi-dimensional semantic perception branch are dynamically weighted, aligned, and fused with the Hadamard product to generate a comprehensive judgment vector. Based on the joint distribution of the face forgery risk score and the confidence probability of scene semantic labels in the comprehensive judgment vector, a structured multidimensional visual perception judgment result is output.

2. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 1, characterized in that, The step of performing size normalization preprocessing on the input image to obtain a preprocessed image of standard size specifically includes: Receive an input image of any resolution, calculate the ratio of the original width to the original height of the input image, and determine whether the ratio falls within the preset target aspect ratio range. If the target falls within the target area, the input image is scaled proportionally based on the target's shorter side, so that the shorter side of the scaled image is equal to the preset standard size side length. The longer side is scaled synchronously according to the original aspect ratio, and the area exceeding the standard size is cropped from the center to obtain a standard size image. If the image does not fall within the specified size, the input image is scaled proportionally so that the long side is equal to the preset standard size side length, and the short side is scaled synchronously according to the original aspect ratio. The area that is smaller than the standard size is then filled with mirrored edge pixel values ​​to obtain a standard size image. Subtract the preset image mean and divide by the preset image standard deviation for each pixel channel of the standard-sized image to normalize the pixel values ​​and output the preprocessed image. The formula for normalizing pixel values ​​is: ； in, The original pixel value of any pixel in a certain channel of the standard-sized image. This is the preset image mean for this channel. This is the preset image standard deviation for this channel. These are the normalized pixel values.

3. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 2, characterized in that, The step of feeding the preprocessed image into the shared backbone network for forward inference and extracting multi-scale shared feature maps specifically includes: The preprocessed image is sequentially fed into the first, second, and third convolution stages of the shared backbone network for layer-by-layer forward convolution operations. In the first convolution stage, the preprocessed image is subjected to two convolution and batch normalization operations with a stride of 2, and the first scale feature map is output. The spatial resolution of the first scale feature map is one-quarter of that of the preprocessed image. In the second convolution stage, the first-scale feature map is subjected to two convolution and batch normalization operations with a stride of 2, and the second-scale feature map is output. The spatial resolution of the second-scale feature map is one-sixteenth of that of the preprocessed image. In the third convolution stage, the second-scale feature map is subjected to two convolution and batch normalization operations with a stride of 2 to output the third-scale feature map. The spatial resolution of the third-scale feature map is one-sixtieth of the preprocessed image. The first-scale feature map, the second-scale feature map, and the third-scale feature map are concatenated along the channel dimension to obtain a multi-scale shared feature map.

4. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 3, characterized in that, The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features, specifically including: The multi-scale shared feature map is input into the object detection operator. The object detection operator generates candidate face regions on the multi-scale shared feature map with preset anchor box scale and aspect ratio. For each candidate region, the face existence probability score and bounding box regression offset are calculated. Candidate regions with probability scores higher than preset thresholds are retained and non-maximum suppression is performed to remove overlapping boxes. The face bounding box coordinates are output. Based on the coordinates of the face bounding box, the corresponding face region feature map is cropped from the multi-scale shared feature map. The face region feature map is then fed into the feature embedding sub-network trained by the triple loss function. The feature embedding sub-network sequentially performs depthwise separable convolution, channel attention weighting, and global average pooling on the face region feature map, and outputs a fixed-dimensional face identity embedding vector. Calculate the Euclidean distance between the face identity embedding vector and each vector in the preset registration identity embedding vector library. The formula for calculating the Euclidean distance is as follows: ； in, Embedding vectors for facial identity, For each registered identity, embed any registered vector from the vector library. The dimension of the embedded vector. and The two vectors are respectively in the th... Values ​​on the dimension, The Euclidean distance between the two; The identity corresponding to the minimum distance is taken as the recognition result, and the reciprocal of the minimum distance is used as the face identity credibility. The formula for calculating face identity credibility is: ； in, This is the minimum Euclidean distance between the face identity embedding vector and all vectors in the registration identity embedding vector library. To prevent small constants from being divided by zero, To assess the credibility of facial recognition.

5. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 4, characterized in that, The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts high-frequency residual features in the frequency domain, outputting a forgery risk score, specifically including: The facial region feature map is converted from a channel dimension to a grayscale single-channel image. A two-dimensional fast Fourier transform is performed on the grayscale single-channel image to map the spatial domain pixel values ​​to a frequency domain complex number representation. In the complex representation of the frequency domain, the center point of the image is taken as the origin. Frequency domain components with radii greater than the preset low-frequency cutoff radius are retained, while frequency domain components with radii less than or equal to the preset low-frequency cutoff radius are set to zero, thus obtaining the high-frequency retained spectrum. Perform a two-dimensional inverse fast Fourier transform on the high-frequency preserved spectrum to restore the frequency domain complex representation to the spatial domain image, thus obtaining the high-frequency reconstructed image; The high-frequency reconstructed image is compared with the grayscale single-channel image pixel by pixel to obtain the high-frequency residual image. The formula for calculating the high-frequency residual image is as follows: ； in, It is a grayscale single-channel image. It is a two-dimensional fast Fourier transform operator. It is a two-dimensional inverse fast Fourier transform operator. For high-frequency retention filters, the radius in the frequency domain is less than or equal to the preset low-frequency cutoff radius. The position value is 0, and the radius is greater than 0. The position value is 1. This is a high-frequency residual plot; The high-frequency residual map is fed into a spoofing discriminant subnetwork trained by a binary classification cross-entropy loss function. The spoofing discriminant subnetwork performs three layers of convolution and global average pooling on the high-frequency residual map and outputs a single spoofing probability value, which is used as a spoofing risk score.

6. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 5, characterized in that, The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map to suppress background noise interference, specifically including: The multi-scale shared feature map is input into the global average pooling layer, and the mean value in the spatial dimension is calculated for each channel of the multi-scale shared feature map to obtain the channel description vector. The channel description vector is sequentially fed into two fully connected layers and a sigmoid activation function to generate a channel attention weight vector. The formula for calculating the channel attention weight vector is as follows: ； in, For multi-scale shared feature maps, This is a global average pooling operation. and These are the weight matrices for the two fully connected layers. It is the ReLU activation function. It is the Sigmoid activation function. This is the channel attention weight vector; Each element of the channel attention weight vector corresponds to a channel of the multi-scale shared feature map; The channel attention weight vector is multiplied with the multi-scale shared feature map channel by channel to obtain the channel-weighted feature map. Adaptive max pooling and adaptive average pooling are performed on the channel-weighted feature map in the spatial dimension. The two pooling results are concatenated in the channel dimension and then fed into a convolutional layer and a sigmoid activation function to generate a spatial attention weight map. The spatial attention weight map and the channel weighted feature map are multiplied pixel-by-pixel in the spatial dimension to obtain a global semantic feature map after context aggregation.

7. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 6, characterized in that, The output of the multi-label scene classification results and the confidence probability of each label specifically includes: The global semantic feature map is fed into the multi-label classification head. The multi-label classification head performs global average pooling, two fully connected layers and a sigmoid activation function on the global semantic feature map in sequence, and outputs a probability vector of the same length as the preset label set. Each element in the probability vector corresponds to the independent existence probability of a scene label in the preset label set. Each element in the probability vector is compared with a preset independent label threshold. Labels with probability values ​​higher than the corresponding threshold are marked as positive sample labels, and labels with probability values ​​lower than the corresponding threshold are marked as negative sample labels. The multi-label scene classification result is composed of all positive sample labels and their corresponding probability values, and the probability values ​​corresponding to all positive sample labels are used as the confidence probabilities of each label. The formula for calculating the confidence probability of each label is as follows: ； in, This is a global semantic feature map processed by context aggregation. This is a global average pooling operation. For the first The weight vector of the fully connected layer corresponding to each scene label For the first The bias term corresponding to each scene label It is the Sigmoid activation function. Let be the confidence probability of the j-th scene label; During the training phase, the multi-label classification head uses the multi-label classification cross-entropy loss function as the optimization objective.

8. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 7, characterized in that, The method involves dynamically aligning the forgery risk features output from the local face anti-spoofing perception branch and fusing them with the scene semantic features output from the global multi-dimensional semantic perception branch using a cross-attention bridging mechanism, thereby generating a comprehensive judgment vector. Specifically, this includes: The forgery risk features output by the local face anti-counterfeiting perception branch are used as the query vector, and the scene semantic features output by the global multidimensional semantic perception branch are used as the key vector and value vector. Perform linear projections on the query vector and key vector respectively to obtain the query projection vector and key projection vector. Calculate the dot product between the transposes of the query projection vector and key projection vector, divide the dot product by a preset scaling factor, and then input... The function yields the attention weight distribution, and the formula for calculating the attention weight distribution is: ； in, To fabricate risk characteristics, and These are scene semantic features, , , These are the linear projection weight matrices for the query, key, and value, respectively. Let be the dimension of the key vector. The preset scaling factor, for Normalization function, Aligned global semantic features; The attention weight distribution is summed with the value vector to obtain the aligned global semantic features; The forgery risk features and the aligned global semantic features are then subjected to an element-wise Hadamard product operation along the feature dimension to obtain a fused feature vector. The formula for calculating the Hadamard product is as follows: ； in, This represents the element-wise Hadamard product operation. To fuse feature vectors; The fused feature vector is fed into a fully connected layer and a Tanh activation function to output a comprehensive decision vector.

9. The multi-dimensional visual semantic perception and target classification method for complex scenes according to claim 8, characterized in that, The method outputs a structured, multi-dimensional visual perception judgment result based on the joint distribution of the face forgery risk score and the scene semantic label confidence probability in the comprehensive judgment vector, specifically including: The face forgery risk score and the confidence probability of semantic labels for each scene are extracted from the comprehensive judgment vector, and then mapped to... interval; If the face forgery risk score is higher than the preset forgery judgment threshold, the judgment result will be marked as high risk, and the face identity credibility, forgery risk score, and multi-label scene classification results with the highest confidence probability will be ranked first. Each label and its confidence probability are encapsulated into a structured JSON output in the first format; If the face forgery risk score is not higher than the forgery judgment threshold and there are non-compliant labels with a confidence probability higher than the preset compliance threshold in the multi-label scene classification results, the judgment result will be marked as non-compliant, and the face identity credibility, forgery risk score, all scene labels and their confidence probabilities will be encapsulated into a structured JSON output in the second format. If the face forgery risk score is not higher than the forgery judgment threshold and the confidence probability of all scene labels is not higher than the compliance threshold, the judgment result will be marked as compliant and the face identity credibility, forgery risk score, all scene labels and their confidence probabilities will be encapsulated into a third-format structured JSON output.

10. A multi-dimensional visual semantic perception and target classification system for complex scenes, used to implement the multi-dimensional visual semantic perception and target classification method for complex scenes as described in any one of claims 1-9, characterized in that, include: Receiving module: Receives the input image to be recognized, performs size normalization preprocessing on the input image, and obtains a preprocessed image of standard size; Extraction module: Feeds the preprocessed image into the shared backbone network for forward inference and extracts multi-scale shared feature maps; The first output module inputs the shared feature map into the local face anti-spoofing perception branch and the global multidimensional semantic perception branch for parallel processing. The local face anti-spoofing perception branch locates the face region in the shared feature map and extracts spatial domain texture features and frequency domain high-frequency residual features, outputting the face identity credibility and forgery risk score. The global multidimensional semantic perception branch performs global context aggregation processing on the shared feature map, suppresses background noise interference, and outputs multi-label scene classification results and confidence probabilities of each label. Generation module: Through a cross-attention bridging mechanism, the forgery risk features output by the local face anti-spoofing perception branch and the scene semantic features output by the global multi-dimensional semantic perception branch are dynamically weighted, aligned, and fused with the Hadamard product to generate a comprehensive judgment vector. The second output module outputs a structured, multi-dimensional visual perception judgment result based on the joint distribution of the face forgery risk score and the confidence probability of scene semantic labels in the comprehensive judgment vector.

Images