Virtual image screening method and device, equipment, medium and product

Abstract

The invention discloses a virtual image screening method, apparatus and device, a medium and a product. The method comprises the steps of obtaining a plurality of real images and a plurality of initial virtual images; performing feature extraction on each real image and each initial virtual image to obtain a first image feature corresponding to each first pixel classification in each real image and a second image feature corresponding to each second pixel classification in each initial virtual image; determining a screening threshold value corresponding to the target pixel classification according to the first image feature corresponding to the target pixel classification; according to the target pixel classification, screening the plurality of second image features to obtain an initial virtual feature set; screening the second image features in the initial virtual feature set according to a screening threshold corresponding to the target pixel classification to obtain a target virtual feature set; and according to the initial virtual image corresponding to the second image feature in the target virtual feature set, determining the target virtual image, and screening out the virtual image capable of improving the model performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a virtual image screening method and device, equipment, medium and product. BACKGROUND

[0002] In the field of artificial intelligence, especially in deep learning models, the scale, quality and distribution of data are the core elements that determine the upper limit of model performance. Although large-scale, high-quality data sets have driven many technological breakthroughs, when deploying models from laboratory environments to complex and variable real-world scenarios, their generalization ability often faces severe challenges. A fundamental reason is that the distribution of the training data set is usually difficult to completely cover all possible situations in real-world applications, resulting in a significant decline in model performance when facing out-of-distribution data.

[0003] The virtual data generation process can generate a large number of samples with highly similar semantic features. Adding these redundant samples to the training set not only does not help improve model diversity, but also makes the model overfit to the specific patterns of these virtual samples, exacerbating overfitting and damaging the model's generalization ability. SUMMARY

[0004] The embodiments of the present application provide a virtual image screening method, device, equipment, medium and product, which can screen out virtual images that improve the generalization performance of the model.

[0005] In a first aspect, the embodiments of the present application provide a virtual image screening method, which comprises: obtaining a plurality of real images and a plurality of initial virtual images; performing feature extraction on each of the real images and each of the initial virtual images respectively, to obtain first image features corresponding to each first pixel classification in each of the real images and second image features corresponding to each second pixel classification in each of the initial virtual images; for a target pixel classification, determining a screening threshold corresponding to the target pixel classification according to the first image features corresponding to the target pixel classification, wherein the target pixel classification is any pixel classification included in the plurality of real images; screening a plurality of the second image features according to the target pixel classification to obtain an initial virtual feature set, wherein the second image features in the initial virtual feature set correspond to the same second pixel classification as the target pixel classification; screening the second image features in the initial virtual feature set according to the screening threshold corresponding to the target pixel classification to obtain a target virtual feature set; determining a target virtual image according to the initial virtual image corresponding to the second image features in the target virtual feature set.

[0006] Secondly, embodiments of this application provide a virtual image filtering device, the device comprising: The acquisition module is used to acquire multiple real images and multiple initial virtual images; The feature extraction module is used to extract features from each of the real images and each of the initial virtual images respectively, to obtain the first image features corresponding to each first pixel category in each of the real images and the second image features corresponding to each second pixel category in each of the initial virtual images; The first determining module is used to classify target pixels and determine a filtering threshold corresponding to the target pixel classification based on the first image features corresponding to the target pixel classification, wherein the target pixel classification is any pixel classification included in the plurality of real images; The first filtering module is used to filter multiple second image features according to the target pixel classification to obtain an initial virtual feature set, wherein the second pixel classification corresponding to the second image features in the initial virtual feature set is the same as the target pixel classification. The second filtering module is used to filter the second image features in the initial virtual feature set according to the filtering threshold corresponding to the target pixel classification, so as to obtain the target virtual feature set; The second determining module is used to determine the target virtual image based on the initial virtual image corresponding to the second image feature in the target virtual feature set.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the virtual image filtering method as described in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the virtual image filtering method as described in the first aspect.

[0009] Fifthly, embodiments of this application provide a computer program product, wherein instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the virtual image filtering method as described in the first aspect.

[0010] In this embodiment, by using the above method, since the screening threshold corresponding to the target pixel classification is not preset but determined based on multiple real images, the screening threshold corresponding to the target pixel classification can be used to characterize the "safe distance" of the real image in the feature space. When screening the second image features corresponding to the target pixel classification, screening by the screening threshold corresponding to the target pixel classification can accurately identify those initial virtual images that fall within the "safe distance" of the real image. In this way, it is possible to avoid introducing initial virtual images that include repetitive and redundant semantic information, and to avoid the risk of overfitting in subsequent model training caused by the introduction of low-quality, highly similar initial virtual images. This ensures the basic quality and high diversity of the selected initial virtual images, thereby improving the generalization ability of the model. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the virtual image filtering method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the virtual image filtering device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0014] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0015] In all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. Additionally, when embodiments of this application require access to sensitive personal information, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments obtained.

[0016] To alleviate the problems of data scarcity and imbalanced distribution, generating virtual data using technologies such as computer graphics and generative artificial intelligence (e.g., GANs, Diffusion Models) has become an important means of expanding training sets and improving model robustness. However, the use of virtual data is not without its costs. A common misconception in current technical practices is the blind pursuit of quantity in generating virtual data while neglecting the quality of its integration with real datasets. This "extensive" data augmentation strategy introduces two new key problems: (1) Redundancy risk: The virtual data generation process may produce a large number of samples with highly similar semantic features. Adding these redundant samples to the training set will not only fail to improve the diversity of the model, but will also cause the model to overfit to the specific patterns of these virtual samples, exacerbating overfitting and impairing generalization ability.

[0017] (2) Risk of distribution mismatch: The distribution characteristics of virtual data may not be able to effectively complement the distribution of real data. If the generated virtual data is mainly concentrated in the feature regions where the real data is already dense, then this expansion only further strengthens the "bias" of the model, and has little effect on improving the model's ability to identify sparse regions in the feature space.

[0018] To address the problems of the prior art, embodiments of this application provide a virtual image filtering method, apparatus, device, medium, and product. The virtual image filtering method provided in this application embodiment will be described first below.

[0019] Figure 1 A flowchart illustrating a virtual image filtering method according to an embodiment of this application is shown. Figure 1 As shown, the virtual image filtering method provided in this application embodiment includes the following steps 101-106, wherein: Step 101: Obtain multiple real images and multiple initial virtual images.

[0020] Real-world images are original images captured from real-world scenes that have not been tampered with or synthesized. They contain feature distributions and label information from the real world. Real-world images are the foundation for models to learn real-world patterns and directly affect the model's generalization ability and prediction accuracy.

[0021] Real images can be obtained based on actual needs. For example, if the selected virtual images are used for model training in the field of autonomous driving, then the real images can be images captured by cameras installed on the vehicle during driving, without limitation. Similarly, if the selected virtual images are used for model training in the field of automatic monitoring, then the real images can be images captured by cameras set up in the monitoring area, without limitation.

[0022] The initial virtual image is not an image captured from the actual scene; it can be a synthetic image generated by an algorithm, or an image obtained by processing the synthetic image.

[0023] Step 102: Feature extraction is performed on each of the real images and each of the initial virtual images to obtain first image features corresponding to each first pixel category in each real image and second image features corresponding to each second pixel category in each initial virtual image. The first image features and the second image features are features in the same feature space.

[0024] For ease of description, the category to which a pixel in the real image belongs is called the first pixel category. The first pixel category is determined based on the semantic mask of the real image. The semantic mask is a matrix of the same size as the real image, and each pixel value in the real image corresponds to a semantic category in the semantic mask (e.g., "person," "vehicle," "sky"). Pixels in the real image belonging to the same semantic category belong to the same pixel category. Similarly, the category to which a pixel in the initial virtual image belongs is called the second pixel category. The second pixel category is determined based on the semantic mask of the initial virtual image. Pixels in the initial virtual image belonging to the same semantic category belong to the same pixel category.

[0025] In one embodiment of this application, feature extraction is performed on each of the real images and each of the initial virtual images to obtain first image features corresponding to each first pixel category in each of the real images and second image features corresponding to each second pixel category in each of the initial virtual images, including: For each of the real images, feature extraction is performed on the real image based on its semantic mask to obtain the features of each pixel in the real image; and, For each of the first pixel categories in the real image, the features of the pixels included in the first pixel category are subjected to max pooling to obtain the first image features of the first pixel category. For each of the initial virtual images, feature extraction is performed on the initial virtual image based on its semantic mask to obtain the features of each pixel in the initial virtual image; and, For each second pixel category of the initial virtual image, the features of the pixels included in the second pixel category are subjected to max pooling to obtain the second image features of the second pixel category.

[0026] Through the above processing, features can be extracted by classifying pixels in the real image and the initial virtual image, which facilitates subsequent feature analysis by pixel classification and thus filters the initial virtual image.

[0027] Specifically, a target model can be used to extract features from each of the real images and each of the initial virtual images to obtain image features for each real image and image features for each initial virtual image. The image features of the real images include the features of pixels located in the masked region of the real image, which is determined based on the semantic category of the semantic mask marker of the real image. Similarly, the image features of the initial virtual images include the features of pixels located in the masked region of the initial virtual image, which is determined based on the semantic category of the semantic mask marker of the initial virtual image.

[0028] The target model is obtained through the following process: The CLIP model is pre-trained using multiple samples, including sample images and descriptive text for the sample images; The image encoder in the trained CLIP model is used as the target model. The image encoder has the ability to process the input image and output the corresponding image features.

[0029] For example, the CLIP (Contrastive Language-Image Pre-training) model can be used to extract features from real images and initial virtual images. The CLIP model consists of an image encoder and a text encoder. The CLIP model can be pre-trained using multiple samples; for example, it can be jointly trained with 4 million image-text pairs. This data contains complex scenes, allowing the model to learn rich visual and linguistic features. The CLIP model mainly consists of an image encoder and a text encoder. Through contrastive learning, it embeds image and text features into a common feature space, thereby capturing the semantic correspondence between cross-modal samples.

[0030] The CLIP model undergoes structural optimization by removing its text encoder, the linear layers and softmax layers on the output side of the image encoder, and retaining only the remaining image encoder (the layer that outputs image features). The trained CLIP model parameters are then loaded, specifically the parameters of the retained structural components, which are then fixed and encapsulated. By removing the text encoder and other related structures, the CLIP model is transformed into a dedicated feature extractor (i.e., the target model).

[0031] The CLIP model is trained using multiple samples, and the image encoder in the trained CLIP model is used as the target model. This target model is then used to extract image features from both real and initial virtual images. This maps real and initial virtual images to a unified feature space, mitigating data bias caused by domain differences (i.e., domain bias caused by inherent physical differences between the real world and the virtual simulation space). This facilitates subsequent similarity comparisons between the initial virtual and real images in the same feature space, thereby selecting virtual images more suitable for specific downstream tasks.

[0032] Step 103: For the target pixel classification, determine the filtering threshold corresponding to the target pixel classification based on the first image feature corresponding to the target pixel classification, wherein the target pixel classification is any pixel classification included in the plurality of real images. Each pixel classification included in the plurality of real images can be processed using the method for processing target pixel classification in this embodiment to obtain a target virtual sample.

[0033] The first image features are filtered to obtain a true feature set. This true feature set includes all first image features corresponding to the target pixel classification in multiple true images. Thus, each first image feature in the true feature set corresponds to a target pixel classification. Each first image feature in the true feature set is analyzed; for example, the distance between every two first image features is calculated, such as cosine distance, maximum mean discrepancy (MMD) distance, etc. The average of the obtained distances is then calculated, or the minimum distance is taken as the filtering threshold corresponding to the target pixel classification; no specific limitation is made here.

[0034] Step 104: Based on the target pixel classification, filter multiple second image features to obtain an initial virtual feature set, wherein the second pixel classification corresponding to the second image features in the initial virtual feature set is the same as the target pixel classification.

[0035] Step 105: Based on the filtering threshold corresponding to the target pixel classification, filter the second image features in the initial virtual feature set to obtain the target virtual feature set.

[0036] For example, calculate the distance between every two second image features in the initial virtual feature set, and determine whether to add the two second image features to the target virtual feature set based on the relationship between the distance and the screening threshold corresponding to the target pixel classification.

[0037] For example, if the distance between two second image features in the initial virtual feature set is greater than or equal to the screening threshold corresponding to the target pixel classification, then the two second image features are added to the target virtual feature set. Examples include cosine distance, maximum mean discrepancy (MMD) distance, etc. In this way, initial virtual images that fall into the "semantic safe zone" of the real image can be accurately identified and eliminated. For instance, if the distance between two second image features is less than the screening threshold, the initial virtual images corresponding to these two second image features are considered to fall into the "semantic safe zone" of the real image. This avoids introducing initial virtual images containing repetitive and redundant semantic information, preventing overfitting risks caused by the introduction of low-quality, highly similar initial virtual images from the source, and ensuring the basic quality and high diversity of the selected initial virtual images.

[0038] Step 106: Determine the target virtual image based on the initial virtual image corresponding to the second image feature in the target virtual feature set.

[0039] It should be noted that the above-described method for classifying target pixels can be used for each first pixel classification to obtain the corresponding target virtual image. These target virtual images are then added to the sample set containing the real images to expand the sample set.

[0040] For example, the initial virtual image corresponding to the second image feature in the target virtual feature set can be used as the target virtual image. Alternatively, the initial virtual image corresponding to the second image feature in the target virtual feature set can be further filtered to obtain the target virtual image. No limitation is made here.

[0041] In the above embodiments, a screening threshold corresponding to the target pixel category is determined based on the first image feature corresponding to the target pixel category, and the second image feature in the initial virtual feature set is screened based on the screening threshold to obtain a target virtual feature set; the target virtual image is determined based on the initial virtual image corresponding to the second image feature in the target virtual feature set. Since the screening threshold corresponding to the target pixel category is not preset but determined based on multiple real images, the screening threshold corresponding to the target pixel category can be used to characterize the "safe distance" of real images in the feature space. When screening the second image feature corresponding to the target pixel category, screening by the screening threshold corresponding to the target pixel category can accurately identify and remove those initial virtual images that fall within the "safe distance" of real images. In this way, it is possible to avoid introducing initial virtual images that include repetitive and redundant semantic information, avoid the risk of overfitting in subsequent model training caused by the introduction of low-quality, highly similar initial virtual images, ensure the basic quality and high diversity of the selected initial virtual images, thereby improving the generalization ability of the model.

[0042] In one embodiment of this application, determining the filtering threshold corresponding to the target pixel category based on the first image feature corresponding to the target pixel category includes: Based on the target pixel classification, a true feature set is obtained, and the true feature set includes the first image features corresponding to the target pixel classification; For each of the first image features in the real feature set, calculate the first distance between the first image feature and each of the other first image features in the real feature set, and determine the neighborhood radius corresponding to the first image feature based on the obtained multiple first distances; The filtering threshold corresponding to the target pixel classification is determined based on the neighborhood radius corresponding to each of the first image features.

[0043] In the above, determining the neighborhood radius corresponding to the first image feature based on the obtained multiple first distances can be achieved by using the minimum value among the obtained multiple first distances as the neighborhood radius corresponding to the first image feature. The mean of all neighborhood radii corresponding to the first image features is calculated, and this mean is used as the filtering threshold corresponding to the target pixel classification. Thus, this filtering threshold can accurately quantify the minimum distinguishable semantic distance between multiple real images. The filtering threshold obtained by averaging these neighborhood radii represents the average size of the "minimum safe region" (also known as the "semantic safe region") surrounding each sample point (i.e., the first image feature) in the feature space. Using the filtering threshold as a filtering criterion ensures that the added initial virtual image does not fall within the "minimum safe region" of the real image (or other virtual images). If the distance between an initial virtual image and a real image is less than the filtering threshold, it indicates that it intrudes into the "personal space" of the real image, providing repetitive and redundant semantic information, and therefore should be discarded.

[0044] For example, if the true feature set includes 10 first image features, then for each first image feature, the first image feature is compared with the other 9 first image features in the true feature set to obtain 9 first distances (the first distance can refer to cosine distance). The first distance with the smallest value among these 9 first distances is taken as the neighborhood radius corresponding to the first image feature. In this way, the neighborhood radii corresponding to the 10 first image features can be calculated. The average of the neighborhood radii corresponding to these 10 first image features is then used as the screening threshold corresponding to the target pixel classification.

[0045] Using the above method, a filtering threshold can be dynamically generated based on the first image features included in the real feature set. If the first image features included in the real feature set are relatively abundant, the filtering threshold will be smaller. When using the filtering threshold to filter the initial virtual images, the filtered initial virtual images can be made semantically closer. If the first image features included in the real feature set are relatively sparse, the filtering threshold will be larger. When using the filtering threshold to filter the initial virtual images, the filtered initial virtual images can maintain greater diversity.

[0046] In another embodiment of this application, step 106, determining the target virtual image based on the initial virtual image corresponding to the second image features in the target virtual feature set, includes: Step 1061: Based on the target pixel classification, obtain the true feature set, which includes the first image features corresponding to the target pixel classification.

[0047] Step 1062: Perform clustering processing on the first image features in the real feature set to obtain the clustering features of the target pixel classification.

[0048] In the above, the average value of the first image features in the real feature set can be calculated, and the result of the average value calculation can be used as the clustering feature for target pixel classification. Alternatively, other clustering processing methods can be used to obtain the clustering feature for target pixel classification. No limitation is made here.

[0049] Step 1063: Determine multiple distance intervals based on the distance between each of the first image features in the real feature set and the clustering features of the target pixel classification.

[0050] Specifically, based on the second distance between each of the first image features in the true feature set and the clustering feature of the target pixel classification, the maximum distribution radius corresponding to the true feature set is determined; the maximum distribution radius is divided to obtain multiple distance intervals.

[0051] For example, the second distance with the largest value can be used as the maximum distribution radius corresponding to the true feature set, or the sum of the second distance with the largest value and a positive number can be used as the maximum distribution radius corresponding to the true feature set, without any limitation.

[0052] The maximum distribution radius is divided into multiple distance intervals. The number of distance intervals can be pre-configured and is not limited here. Preferably, each distance interval has the same width.

[0053] In the above description, based on the second distance between each of the first image features in the real feature set and the clustering features of the target pixel classification, the distribution of the first image features in the real feature set in the feature space can be characterized, thereby determining the maximum distribution radius. The maximum distribution radius is then divided into multiple distance intervals. Using this process, the first image features in the real feature set can be quantized into multiple discrete distance intervals. This facilitates determining the number of initial virtual images to be added based on the number of real images in each distance interval, ensuring that the filling of initial virtual images is no longer random but has a clear objective. This is beneficial for adding more initial virtual images to fill sparsely distributed real image areas and adding fewer initial virtual images to fill densely distributed real image areas. This compensation strategy can effectively optimize the feature space distribution of the entire training set from "imbalanced" to "balanced," thereby improving the model's generalization ability.

[0054] Step 1064: For each distance interval, filter the second image features in the target virtual feature set according to the distance interval to obtain the candidate virtual feature set corresponding to the distance interval.

[0055] For example, for each of the second image features in the target virtual feature set, a third distance (which may be a cosine distance) between the second image feature and the clustering feature of the target pixel classification is calculated; if the third distance is located in a first distance interval among the plurality of distance intervals, the second image feature is added to the candidate virtual feature set corresponding to the first distance interval.

[0056] For ease of distinction, the distance interval containing the third distance is referred to as the first distance interval, which is one of multiple distance intervals.

[0057] For each of the second image features in the target virtual feature set, the above method is applied, and each second image feature can be summarized into a candidate virtual feature set corresponding to a distance interval. When subsequently filtering the initial virtual images, the filtering can be based on the candidate virtual feature set corresponding to each distance interval, making the feature space distribution of the training set (including real images and the filtered initial virtual images) more balanced.

[0058] Step 1065: Determine the target virtual image based on the candidate virtual feature set corresponding to the distance interval.

[0059] In this embodiment, multiple distance intervals are determined based on the distance between each first image feature in the real feature set and the clustering feature of the target pixel classification. For each distance interval, the second image features in the target virtual feature set are filtered according to the distance interval to obtain a candidate virtual feature set corresponding to the distance interval. The target virtual image is determined based on the candidate virtual feature set corresponding to the distance interval. Through the above process, filtering can be performed based on the candidate virtual feature set corresponding to each distance interval, making the feature space distribution of the training set (including real images and the filtered initial virtual images) more balanced.

[0060] In another embodiment of this application, determining the target virtual image based on the candidate virtual feature set corresponding to the distance interval includes: For each distance interval, obtain the number of initial virtual images corresponding to the distance interval, and, N second image features are selected from the candidate virtual feature set corresponding to the distance interval, and the initial virtual images corresponding to the N second image features are used as the target virtual image, where N is the number of initial virtual images corresponding to the distance interval.

[0061] Specifically, for each distance interval, the number of initial virtual images corresponding to each distance interval is obtained. For example, the number of real images in each distance interval can be determined first, and then the number of initial virtual images corresponding to each distance interval can be determined. For example, for each distance interval, the number of first image features whose second distance lies within the distance interval is determined, and the number of initial virtual images corresponding to the distance interval is determined based on the number of first image features corresponding to the distance interval, wherein the second distance refers to the distance between the first image features in the real feature set and the clustering features of the target pixel classification.

[0062] In the above process, the distance interval corresponding to each first image feature is determined based on the distance between the first image features in the real feature set and the clustering features of the target pixel classification. The number of first image features corresponding to each distance interval is then counted, which represents the number of real images. Next, based on the number of first image features corresponding to each distance interval, the number of initial virtual images corresponding to that distance interval is determined. For example, if the number of first image features corresponding to a certain distance interval is small, a larger number of initial virtual images can be configured for that distance interval; conversely, if the number of first image features corresponding to a certain distance interval is large, a smaller number of initial virtual images can be configured for that distance interval. This ensures a more balanced number of samples (including real images and selected initial virtual images) in each distance interval, thereby making the feature space distribution of the training set (including real images and selected initial virtual images) more balanced.

[0063] In this embodiment, for each distance interval, the number of initial virtual images corresponding to the distance interval is obtained, and N second image features are selected from the candidate virtual feature set corresponding to the distance interval, and the initial virtual images corresponding to the N second image features are used as the target virtual images. Through this method, initial virtual images corresponding to each distance interval can be filtered based on the number of initial virtual images corresponding to that distance interval, thereby effectively adjusting the number of images whose image features fall within each distance interval, making the feature space distribution of the training set (including real images and the filtered initial virtual images) more balanced.

[0064] In another embodiment of this application, determining the number of initial virtual images corresponding to the distance interval based on the number of first image features corresponding to the distance interval includes: The ratio of the number of first image features corresponding to the distance interval to the number of first image features in the real feature set is taken as the first ratio. Based on the first ratio, a first weight of the distance interval is determined, wherein the first ratio is negatively correlated with the first weight; The ratio of the first weight to the total weight is used as the second ratio, where the total weight is the sum of the first weights for each distance interval; The product of the total number and the second ratio is taken as the number of initial virtual images corresponding to the distance interval, where the total number is the total number of initial virtual images for sample augmentation for the target pixel classification.

[0065] Specifically, the first ratio is negatively correlated with the first weight, meaning that the larger the first ratio, the smaller the first weight; and vice versa. The first weight can be the reciprocal of the first ratio. For certain distance intervals, no first image feature may fall within that interval, therefore the number of initial virtual images corresponding to that distance interval is 0, and correspondingly, the calculated first ratio is also 0. When the first ratio is 0, the denominator in the reciprocal of the first ratio is 0. To avoid division by zero errors, after obtaining the first ratio, a very small positive number (e.g., ...) is added to the first ratio. The reciprocal of this sum is used as the first weight. The first weight can also be determined in other ways, as long as the first ratio is negatively correlated with the first weight; this is not limited here.

[0066] For each distance interval, the corresponding first weight can be determined using the above method, and the sum of the first weights for each interval is taken as the total weight. The ratio of the first weight to the total weight is taken as the second ratio, and the product of the total number and the second ratio is taken as the number of initial virtual images corresponding to the distance interval (that is, the distance interval for which the number of initial virtual images is calculated).

[0067] By determining the number of initial virtual images for each distance interval using the above method, it is ensured that the selection of initial virtual images is no longer random but has a clear objective. For example, more initial virtual images can be added to fill areas where the real images are sparsely distributed, while fewer initial virtual images can be added to fill areas where the real images are densely distributed. This compensation strategy enables a balanced distribution of the feature space of the entire training set, thereby improving the model's generalization ability.

[0068] The virtual image filtering method provided in this application is illustrated below with examples. The virtual image filtering method provided in this application includes the following process: Step 1. This invention uses a pre-trained CLIP model as a feature extractor. Specifically, the CLIP model includes an image encoder and a text encoder. The CLIP model can be pre-trained using multiple samples, for example, jointly trained with 4 million image-text pairs. This data contains complex scenes, allowing the model to learn rich visual and linguistic features. The CLIP model mainly consists of an image encoder and a text encoder. Through contrastive learning, the features of the image and text are embedded in a common feature space, thereby capturing the semantic correspondence between cross-modal samples. The CLIP model is structurally optimized by removing the linear layer and softmax layer on the output side of the text encoder and image encoder, retaining only the remaining image encoder (the layer that outputs image features). The pre-trained CLIP model parameters are loaded; here, only the parameters of the retained structural parts are loaded and fixed. By removing the text encoder and other related structures, the CLIP model is transformed into a dedicated feature extractor. This feature extractor will serve as the image feature extraction module. The CLIP model is used to extract features from the real image and the initial virtual image.

[0069] Step 2. Image Feature Extraction and Aggregation: Input a set of real images (in For real images, (Its corresponding semantic mask) to the feature extraction module This obtains the features of each mask region (that is, the features of each pixel in the mask region). For the first... The first real image Each category (i.e., pixel classification) is aggregated using max pooling (which refers to the features of each pixel in the mask region, obtained by the encoder) to obtain the feature vector (i.e., the first image feature) representing the category of the sample, as shown in equation (1) below: (1) Then, each category is constructed. Real feature set ,in For included categories The number of real images. For ease of description, the feature vectors in the real feature set are also called samples or elements.

[0070] Step 3. Calculate the adaptive minimum similarity threshold To avoid introducing redundant virtual images that could lead to model overfitting, embodiments of this application provide a method for automatically "learning" and dynamically generating thresholds from the feature distribution of real images. The mechanism of the (i.e., the screening threshold). This threshold acts as an adaptive, category-specific "semantic density ruler" to precisely quantify the minimum distinguishable semantic distance between samples. Threshold Can be categorized Real feature set The result is obtained by calculation using the following formula (2): (2) It dynamically captures categories. The "natural density" of internal real-image clusters. For feature-rich categories, its The value will be smaller, allowing virtual images to be semantically closer; for categories with sparse features, its The values ​​will be relatively large, requiring virtual images to maintain greater diversity. In essence... It is a category The mean of the "nearest neighbor" radius of each sample point in the true feature set, which represents the average size of the "minimum safe zone" around each sample point in the feature space. As a redundancy criterion, its technical essence is to ensure that newly added virtual images do not fall within the "minimum safe area" of existing real samples (or other virtual images). If a virtual image is related to a category... The distance between the true samples in the true feature set is less than This indicates that it intrudes into the "personal space" of the real sample, providing repetitive and redundant semantic information, and therefore should be removed.

[0071] Step 4. Establish a feature space distribution model: To characterize the distribution of real images in the feature space, first calculate the distribution of each category. The characteristic prototype (centroid, also known as cluster feature). Next, calculate all features. To its category prototype The cosine distance is ,category Characteristic distribution radius Defined as maximum distance . The interval Average score Distance intervals (e.g.) (hereinafter referred to as intervals), then the width of each interval is , No. each interval It is defined as shown in the following equation (3): (3) True features fall within the interval sample proportion (i.e., the first ratio) is calculated according to the following formula (4): (4) in, This indicates that the true features are concentrated in the interval. The number of samples.

[0072] Step 5. Determine the compensation quota for virtual images: Let the plan be categorized as follows. The total number of expanded virtual images (i.e., the total quantity) is To reverse the imbalance of the original distribution, each interval... Virtual image compensation quota Calculated using the inverse proportional distribution formula shown in equation (5): (5) Among them, weight Defined as (i.e., the first weight) It is a very small positive number (e.g.) To prevent division by zero errors, the distance interval can be calculated according to the following formula (6). The corresponding number of initial virtual images : (6) The above formula ensures that sparsely distributed regions are represented in real images. (Small) will obtain more virtual image compensation, thereby achieving a more balanced distribution of the feature space.

[0073] Step 6. Two-way filtering mechanism for virtual images: (1) Virtual image set After feature extraction, the category is obtained. The virtual feature set (i.e., the initial virtual feature set) is shown in equation (7) below: (7) (2) First step (redundancy removal based on density threshold): Calculate the virtual features in the initial virtual feature set. (i.e., the second image feature) and its in Cosine distance of nearest neighbor The filtering criteria are: Samples that meet this condition constitute the target virtual feature set. ; (3) Second step (balanced sampling based on distribution quotas): For Each feature in Calculate its value to the real prototype Distance: And determine the interval to which it belongs. (i.e., satisfy) )of Ultimately, from each interval Randomly selected from the corresponding candidate virtual feature set These samples form the final target virtual image set used for training. .

[0074] (4) Output integrated training set: The final selected virtual feature indexes are mapped back to their corresponding initial virtual images, and together with the real images, they are used as the optimized training set and output to the downstream tasks for model training.

[0075] In this embodiment, the screening threshold is not preset but calculated based on the true feature distribution (i.e., the true feature set) of each category. It represents the "natural minimum safe distance" for that type of sample in the feature space. When screening virtual data, it is mandatory that the cosine distance between each virtual sample and its nearest neighbor sample must be greater than or equal to this threshold. This mechanism acts like a smart filter, accurately identifying and eliminating redundant virtual samples that intrude into the "semantic safe zone" of existing samples. This avoids the overfitting risk introduced by low-quality, highly similar data from the outset, ensuring the basic quality and high diversity of the selected virtual samples.

[0076] On the other hand, a balanced compensation mechanism based on an inverse proportional distribution model is provided. This mechanism employs a calculation method of "feature space distribution interval division" and "inverse proportional quota." First, by calculating the category prototype and the maximum distribution radius, the feature space is quantified into multiple discrete intervals, and the distribution ratio of real samples is statistically analyzed. Then, based on the principle of "more filling in sparse areas, less filling in dense areas," the number of virtual samples to be filled in each interval is calculated. This mechanism makes the filling of virtual data no longer random but directional. It actively guides virtual data to "fill" the sparsest regions of the real data distribution. This "peak-shaving and valley-filling" compensation strategy can effectively optimize the feature space distribution of the entire training set from "imbalanced" to "balanced," directly expanding the feature recognition boundary of the model and thus improving its generalization ability for rare scenarios and marginal cases.

[0077] The two aspects above complement each other. The first aspect ensures the "quality" of the data, while the second aspect (balanced compensation) further optimizes the "structure" of the data, together ensuring that the final integrated training set reaches its optimal state in both quality and structure.

[0078] Figure 2 A structural diagram of the virtual image filtering device provided in an embodiment of this application is shown. Figure 2 As shown, the virtual image filtering device 200 includes: The acquisition module 201 is used to acquire multiple real images and multiple initial virtual images; The feature extraction module 202 is used to extract features from each of the real images and each of the initial virtual images respectively, to obtain the first image features corresponding to each first pixel category in each of the real images and the second image features corresponding to each second pixel category in each of the initial virtual images; The first determining module 203 is used to determine a filtering threshold corresponding to the target pixel classification based on the first image features corresponding to the target pixel classification, wherein the target pixel classification is any pixel classification included in the multiple real images; The first filtering module 204 is used to filter multiple second image features according to the target pixel classification to obtain an initial virtual feature set, wherein the second pixel classification corresponding to the second image features in the initial virtual feature set is the same as the target pixel classification. The second filtering module 205 is used to filter the second image features in the initial virtual feature set according to the filtering threshold corresponding to the target pixel classification, so as to obtain the target virtual feature set; The second determining module 206 is used to determine the target virtual image based on the initial virtual image corresponding to the second image feature in the target virtual feature set.

[0079] In one embodiment of this application, the first determining module 203 includes: The first acquisition submodule is used to obtain a real feature set based on the target pixel classification, wherein the real feature set includes the first image features corresponding to the target pixel classification; The first determining submodule is used to calculate, for each first image feature in the real feature set, a first distance between the first image feature and other first image features in the real feature set, and to determine the neighborhood radius corresponding to the first image feature based on the obtained multiple first distances; The second determining submodule is used to determine the filtering threshold corresponding to the target pixel classification based on the neighborhood radius corresponding to each of the first image features.

[0080] In one embodiment of this application, the second screening module 205 is specifically used for: If there are two second image features in the initial virtual feature set whose distance is greater than or equal to the filtering threshold corresponding to the target pixel classification, then the two second image features are added to the target virtual feature set.

[0081] In one embodiment of this application, the second determining module 206 includes: The first acquisition submodule is used to obtain a real feature set based on the target pixel classification, wherein the real feature set includes the first image features corresponding to the target pixel classification; The clustering processing submodule is used to perform clustering processing on the first image features in the real feature set to obtain the clustering features of the target pixel classification; The third determining submodule is used to determine multiple distance intervals based on the distance between each of the first image features in the real feature set and the clustering features of the target pixel classification; The fourth determining submodule is used to filter the second image features in the target virtual feature set according to the distance interval for each distance interval, so as to obtain the candidate virtual feature set corresponding to the distance interval; The fifth determining submodule is used to determine the target virtual image for each distance interval based on the candidate virtual feature set corresponding to the distance interval.

[0082] In one embodiment of this application, the third determining submodule includes: The first determining unit is configured to determine the maximum distribution radius corresponding to the real feature set based on the second distance between each first image feature in the real feature set and the clustering feature of the target pixel classification; A dividing unit is used to divide the maximum distribution radius to obtain multiple distance intervals.

[0083] In one embodiment of this application, the fourth determining submodule includes: The calculation unit is used to calculate, for each of the second image features in the target virtual feature set, a third distance between the second image feature and the clustering feature of the target pixel classification; The second determining unit is configured to add the second image feature to the candidate virtual feature set corresponding to the first distance interval if the third distance is located in the first distance interval among the plurality of distance intervals.

[0084] In one embodiment of this application, the fifth determining submodule includes: The acquisition unit is used to acquire the number of initial virtual images corresponding to each distance interval; The filtering unit is used to select N second image features from the candidate virtual feature set corresponding to the distance interval, and use the initial virtual images corresponding to the N second image features as the target virtual image, where N is the number of initial virtual images corresponding to the distance interval.

[0085] In one embodiment of this application, the acquisition unit includes: The first acquisition subunit is configured to determine, for each distance interval, the number of first image features whose second distance lies within the distance interval; The second acquisition subunit is used to determine the number of initial virtual images corresponding to the distance interval based on the number of first image features corresponding to the distance interval, wherein the second distance refers to the distance between the first image features in the real feature set and the clustering features of the target pixel classification.

[0086] In one embodiment of this application, the second acquisition subunit is specifically used for: The ratio of the number of first image features corresponding to the distance interval to the number of first image features in the true feature set is taken as the first ratio. Based on the first ratio, a first weight of the distance interval is determined, wherein the first ratio is negatively correlated with the first weight; The ratio of the first weight to the total weight is used as the second ratio, where the total weight is the sum of the first weights for each distance interval; The product of the total number and the second ratio is taken as the number of initial virtual images corresponding to the distance interval, where the total number is the total number of initial virtual images for sample augmentation for the target pixel classification.

[0087] In one embodiment of this application, the feature extraction module 202 includes: The first feature extraction submodule is used to extract features from each real image based on the semantic mask of the real image to obtain the features of each pixel in the real image. The second acquisition submodule is used to perform max pooling on the features of the pixels included in the first pixel classification for each first pixel classification of the real image, so as to obtain the first image features of the first pixel classification. The second feature extraction submodule is used to extract features from each initial virtual image based on the semantic mask of the initial virtual image to obtain the features of each pixel in the initial virtual image. The third acquisition submodule is used to perform max pooling on the features of the pixels included in the second pixel category for each second pixel category of the initial virtual image, so as to obtain the second image features of the second pixel category.

[0088] The virtual image filtering device 200 provided in this application embodiment can implement the various processes implemented in the aforementioned virtual image filtering method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0089] Figure 3 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0090] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0091] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0092] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0093] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to the first or second aspect of this disclosure.

[0094] The processor 601 implements any of the information auditing methods described in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0095] In one example, the electronic device may also include a communication interface 603 and a bus 610. For example, Figure 3 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 610 and complete communication with each other.

[0096] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0097] Bus 610 includes hardware, software, or both, that couples components of an information auditing method or verification device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0098] Furthermore, in conjunction with the virtual image filtering methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the virtual image filtering methods in the above embodiments.

[0099] This application provides a computer program product in which the instructions are executed by the processor of an electronic device, causing the electronic device to perform any of the virtual image filtering methods described in the above embodiments.

[0100] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described as examples. However, the method process of this application is not limited to the specific steps described. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0101] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0102] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0103] The foregoing flowcharts and / or block diagrams of methods, apparatus (systems) according to embodiments of the present disclosure have described various aspects of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0104] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A virtual image filtering method, characterized in that, The method includes: Acquire multiple real images and multiple initial virtual images; Feature extraction is performed on each of the real images and each of the initial virtual images to obtain the first image features corresponding to each first pixel category in each real image and the second image features corresponding to each second pixel category in each initial virtual image; For target pixel classification, a filtering threshold corresponding to the target pixel classification is determined based on the first image feature corresponding to the target pixel classification, wherein the target pixel classification is any pixel classification included in the multiple real images; Based on the target pixel classification, multiple second image features are filtered to obtain an initial virtual feature set, wherein the second pixel classification corresponding to the second image features in the initial virtual feature set is the same as the target pixel classification; Based on the filtering threshold corresponding to the target pixel classification, the second image features in the initial virtual feature set are filtered to obtain the target virtual feature set; The target virtual image is determined based on the initial virtual image corresponding to the second image feature in the target virtual feature set.

2. The virtual image filtering method according to claim 1, characterized in that, Determining the filtering threshold corresponding to the target pixel category based on the first image feature corresponding to the target pixel category includes: Based on the target pixel classification, a true feature set is obtained, wherein the true feature set includes the first image features corresponding to the target pixel classification; For each of the first image features in the real feature set, calculate the first distance between the first image feature and each of the other first image features in the real feature set, and determine the neighborhood radius corresponding to the first image feature based on the obtained multiple first distances; The filtering threshold corresponding to the target pixel classification is determined based on the neighborhood radius corresponding to each of the first image features.

3. The virtual image filtering method according to claim 1, characterized in that, Based on the filtering threshold corresponding to the target pixel classification, the second image features in the initial virtual feature set are filtered to obtain the target virtual feature set, including: If there are two second image features in the initial virtual feature set whose distance is greater than or equal to the filtering threshold corresponding to the target pixel classification, then the two second image features are added to the target virtual feature set.

4. The virtual image filtering method according to claim 1, characterized in that, Determine the target virtual image based on the initial virtual image corresponding to the second image features in the target virtual feature set, including: Based on the target pixel classification, a true feature set is obtained, and the true feature set includes the first image features corresponding to the target pixel classification; Clustering is performed on the first image features in the real feature set to obtain the clustering features for the target pixel classification; Based on the distance between each of the first image features in the true feature set and the clustering features of the target pixel classification, multiple distance intervals are determined; For each distance interval, the second image features in the target virtual feature set are filtered according to the distance interval to obtain the candidate virtual feature set corresponding to the distance interval; For each distance interval, the target virtual image is determined based on the candidate virtual feature set corresponding to the distance interval.

5. The virtual image filtering method according to claim 4, characterized in that, Based on the distance between each of the first image features in the true feature set and the clustering features of the target pixel classification, multiple distance intervals are determined, including: The maximum distribution radius corresponding to the real feature set is determined based on the second distance between each first image feature in the real feature set and the clustering feature of the target pixel classification. The maximum distribution radius is divided to obtain multiple distance intervals.

6. The virtual image filtering method according to claim 4, characterized in that, Based on the distance interval, the second image features in the target virtual feature set are filtered to obtain the candidate virtual feature set corresponding to the distance interval, including: For each of the second image features in the target virtual feature set, a third distance is calculated between the second image feature and the clustering features of the target pixel classification; and, If the third distance is located in the first distance interval among the multiple distance intervals, then the second image feature is added to the candidate virtual feature set corresponding to the first distance interval.

7. The virtual image filtering method according to claim 4, characterized in that, For each distance interval, the target virtual image is determined based on the candidate virtual feature set corresponding to the distance interval, including: For each distance interval, obtain the number of initial virtual images corresponding to the distance interval, and, N second image features are selected from the candidate virtual feature set corresponding to the distance interval, and the initial virtual images corresponding to the N second image features are used as the target virtual image, where N is the number of initial virtual images corresponding to the distance interval.

8. The virtual image filtering method according to claim 7, characterized in that, For each distance interval, the number of initial virtual images corresponding to the distance interval is obtained, including: For each distance interval, determine the number of first image features whose second distance lies within the distance interval, and, The number of initial virtual images corresponding to the distance interval is determined based on the number of first image features corresponding to the distance interval, wherein the second distance refers to the distance between the first image features in the real feature set and the clustering features of the target pixel classification.

9. The virtual image filtering method according to claim 8, characterized in that, Determining the number of initial virtual images corresponding to the distance interval based on the number of first image features corresponding to the distance interval includes: The ratio of the number of first image features corresponding to the distance interval to the number of first image features in the true feature set is taken as the first ratio. Based on the first ratio, a first weight of the distance interval is determined, wherein the first ratio is negatively correlated with the first weight; The ratio of the first weight to the total weight is used as the second ratio, where the total weight is the sum of the first weights for each distance interval; The product of the total number and the second ratio is taken as the number of initial virtual images corresponding to the distance interval, where the total number is the total number of initial virtual images for sample augmentation for the target pixel classification.

10. The virtual image filtering method according to claim 1, characterized in that, Feature extraction is performed on each of the real images and each of the initial virtual images to obtain first image features corresponding to each first pixel category in each real image and second image features corresponding to each second pixel category in each initial virtual image, including: For each of the real images, feature extraction is performed on the real image based on its semantic mask to obtain the features of each pixel in the real image; and, For each of the first pixel categories in the real image, the features of the pixels included in the first pixel category are subjected to max pooling to obtain the first image features of the first pixel category. For each of the initial virtual images, feature extraction is performed on the initial virtual image based on its semantic mask to obtain the features of each pixel in the initial virtual image; and, For each second pixel category of the initial virtual image, the features of the pixels included in the second pixel category are subjected to max pooling to obtain the second image features of the second pixel category.

11. A virtual image filtering device, characterized in that, The device includes: The acquisition module is used to acquire multiple real images and multiple initial virtual images; The feature extraction module is used to extract features from each of the real images and each of the initial virtual images respectively, to obtain the first image features corresponding to each first pixel category in each of the real images and the second image features corresponding to each second pixel category in each of the initial virtual images; The first determining module is used to classify target pixels and determine a filtering threshold corresponding to the target pixel classification based on the first image features corresponding to the target pixel classification, wherein the target pixel classification is any pixel classification included in the plurality of real images; The first filtering module is used to filter multiple second image features according to the target pixel classification to obtain an initial virtual feature set, wherein the second pixel classification corresponding to the second image features in the initial virtual feature set is the same as the target pixel classification. The second filtering module is used to filter the second image features in the initial virtual feature set according to the filtering threshold corresponding to the target pixel classification, so as to obtain the target virtual feature set; The second determining module is used to determine the target virtual image based on the initial virtual image corresponding to the second image feature in the target virtual feature set.

12. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the virtual image filtering method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the virtual image filtering method as described in any one of claims 1-10.

14. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the virtual image filtering method as described in any one of claims 1-10.

Images