A method and system for scenario adaptation for a target model

Abstract

The present disclosure provides a method and system for scene adaptation of a target model, relating to the technical field of data processing. The method and system provided by the present disclosure obtain N image samples under a target scene, determine an important region prediction graph and an imaging prediction result in the N image samples through a target model, determine a comprehensive confidence of a processing result of the target model on the N image samples under the target scene based on the important region prediction graph and the imaging prediction result, and further adapt the target model to the target scene based on the comprehensive confidence, thereby solving the trade-off problem of adaptation performance and adaptation efficiency of visual imaging technology in cross-scene adaptation, reducing the adaptation complexity to the greatest extent on the basis of ensuring the adaptation performance, and improving the adaptation efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method and system for scene adaptation of a target model. Background Technology

[0002] User profiling technology refers to the technology of predicting various attribute tags of users. Because profile tags can characterize users from multiple dimensions, profiling technology has become a foundational technology for search recommendation and various marketing techniques. Among them, profile prediction technology based on visual information (such as images and videos) (also known as user profiling technology) is an important part of profiling technology.

[0003] User profiling technology is generally based on deep learning models for label prediction, and therefore often relies on a large amount of training data. However, when it comes to adapting to new scenario data that differs greatly from the training data, existing methods cannot simultaneously balance adaptation performance and adaptation efficiency. Summary of the Invention

[0004] The main purpose of this disclosure is to provide a method and system for scene adaptation of target models, which can solve the trade-off between adaptation performance and adaptation efficiency in cross-scene adaptation of visual profiling technology, and minimize adaptation complexity and improve adaptation efficiency while ensuring adaptation performance.

[0005] This disclosure provides a method for scene adaptation of a target model, the target model being used for image processing, comprising:

[0006] Obtain N image samples from the target scene, where N is a natural number greater than 1;

[0007] Using the target model, the prediction maps and portrait prediction results of important regions in the N image samples are determined, wherein the prediction maps of important regions include maps corresponding to local features in the N image samples;

[0008] Based on the predicted map of the important region and the predicted image, determine the comprehensive confidence level of the target model's processing results for the N image samples in the target scene; and

[0009] Based on the comprehensive confidence level, the target model is adapted to the target scenario.

[0010] In some embodiments, determining the important region prediction map and portrait prediction results in the N image samples includes: for each of the N image samples:

[0011] Based on the target model, feature extraction is performed on the image samples to obtain the feature map of the image samples; and

[0012] Based on the feature map, the important region prediction map and the portrait prediction result of the image sample are determined.

[0013] In some embodiments, determining the important region prediction map and the portrait prediction result of the image sample based on the feature map includes:

[0014] Based on the target model, the feature map is predicted to obtain the predicted map of the important region; and

[0015] Based on the important region prediction map and the feature map, the portrait prediction result of the image sample is determined.

[0016] In some embodiments, determining the portrait prediction result of the image sample includes:

[0017] Obtain the product of the predicted map of the important region and the feature map; and

[0018] The image prediction result of the image sample is determined based on the target model and the product result.

[0019] In some embodiments, determining the comprehensive confidence level of the target scene based on the important region prediction map and the portrait prediction result includes:

[0020] For each of the N image samples: based on the important region prediction map and the portrait prediction result of the image sample, determine the single-sample confidence level of the image sample; and

[0021] The average of the N single-sample confidence scores is used to calculate the overall confidence score of the target scenario.

[0022] In some embodiments, determining the single-sample confidence level of the image sample based on the important region prediction map and the portrait prediction result of the image sample includes:

[0023] The image samples are perturbed to determine a first perturbation deviation in the predicted map of the important region before and after the perturbation, and a second perturbation deviation in the image prediction result; and

[0024] The single-sample confidence level is determined based on the first perturbation bias and the second perturbation bias.

[0025] In some embodiments, determining the first perturbation deviation of the predicted map of the important region before and after the data perturbation and the second perturbation deviation of the portrait prediction result includes:

[0026] The image samples are subjected to M data perturbations, where M is a natural number greater than 1. For each of the M data perturbations: based on the target model, a first deviation of the predicted map of the important region before and after the data perturbation is determined; the average of the M first deviations is calculated to obtain the first perturbation deviation; and...

[0027] Based on the target model, a second deviation of the portrait prediction result before and after the data perturbation is determined, and the average of the M second deviations is calculated to obtain the second perturbation deviation.

[0028] In some embodiments, the image sample includes an initial image sample and a perturbed image sample, wherein the image sample before the data perturbation is the initial image sample, and the image sample after the data perturbation is the perturbed image sample; and

[0029] The step of determining the first deviation of the important region prediction map before and after the data perturbation based on the target model includes: determining the initial important region prediction map of the initial image sample based on the target model.

[0030] Based on the target model, the predicted map of the important perturbation region of the perturbation image sample is determined, and

[0031] Based on the initial important region prediction map and the perturbation important region prediction map, the first deviation of the image sample is determined, wherein the important region prediction map includes the initial important region prediction map and the perturbation important region prediction map.

[0032] In some embodiments, determining a second deviation of the profile prediction result before and after the data perturbation based on the target model includes:

[0033] The initial portrait prediction result of the initial image sample is determined based on the target model;

[0034] Based on the target model, the perturbed image prediction result of the perturbed image sample is determined; and

[0035] Based on the initial image prediction result and the perturbed image prediction result, the second deviation of the image sample is determined, wherein the image prediction result includes the initial image prediction result and the perturbed image prediction result.

[0036] In some embodiments, the data perturbation may be performed in a manner including at least one of Gaussian noise, salt-and-pepper noise, or random noise.

[0037] In some embodiments, adapting the target model to the target scenario based on the comprehensive confidence level includes:

[0038] The overall confidence level is divided into confidence levels to obtain the confidence level of the target scene; and

[0039] The target model is adapted to the target scenario based on the confidence level.

[0040] In some embodiments, the confidence levels include a first level of confidence, a second level of confidence, and a third level of confidence, wherein the confidence level of the first level of confidence is greater than the confidence level of the second level of confidence, and the confidence level of the second level of confidence is greater than the confidence level of the third level of confidence.

[0041] In some embodiments, adapting the target model to the target scenario based on the comprehensive confidence score includes:

[0042] The confidence level is determined to be the first level of confidence, without changing the parameters of the target model;

[0043] Determine the confidence level as the second level of confidence, and adjust the decision threshold of the target model; or

[0044] The confidence level is determined to be the third level of confidence, and the target model is retrained.

[0045] In some embodiments, the training process of the target model includes:

[0046] Obtain the N image samples in the target scene; and

[0047] The target model is trained based on the N image samples to obtain the trained target model.

[0048] In some embodiments, training the target model based on the N image samples includes:

[0049] For each of the N image samples: feature extraction is performed on the image sample based on the target model to obtain the feature map of the image sample;

[0050] Based on the feature map and the target model, the important region prediction map and the portrait prediction result of the image sample are determined; and

[0051] Based on the predicted map of the important region and the predicted image, the target model is converged to obtain the trained target model.

[0052] In some embodiments, the step of converging the target model based on the important region prediction map and the portrait prediction results to obtain the trained target model includes:

[0053] Obtain the important region map of the image sample, compare the important region map with the predicted important region map to obtain important region loss information;

[0054] Obtain the original image classification label of the image sample, compare the original image classification label with the image prediction result, and obtain image classification loss information;

[0055] The loss information of the important regions and the loss information of the portrait classification are fused to obtain the target model loss information; and

[0056] The target model is converged based on the target model loss information to obtain the trained target model.

[0057] In some embodiments, obtaining the important region map of the image sample includes:

[0058] Based on a preset image labeling model, feature extraction is performed on the image samples to obtain a sensitive region map of the image samples.

[0059] The saliency of the image samples is predicted based on a pre-defined saliency detection model, resulting in a saliency map of the image samples; and

[0060] The important region map of the image sample is determined based on the sensitive region map and the saliency map.

[0061] This disclosure also provides a system for scene adaptation of a target model, comprising: at least one storage medium including at least one instruction set for implementing analysis of a method for scene adaptation of the target model; and at least one processor communicatively connected to the at least one storage medium, wherein, when the system is running, the at least one processor reads the at least one instruction set and executes the above-described method for scene adaptation of the target model according to the instructions of the at least one instruction set.

[0062] As can be seen from the above technical solutions, the present disclosure provides a method for scene adaptation of a target model and a system for executing this method. The method and system first use the target model to predict N image samples contained in the target scene. While predicting the portrait prediction results of the N image samples, they also predict the important regions (important region prediction maps) of the N image samples. Based on the predicted portrait label results and the important region prediction maps, the comprehensive confidence level of the target scene is estimated. Then, based on the comprehensive confidence level, confidence levels are divided. Finally, the corresponding adaptation strategy is determined according to the confidence level. This solves the trade-off between adaptation performance and adaptation efficiency in cross-scene adaptation of visual portrait technology, minimizing adaptation complexity and improving adaptation efficiency while ensuring adaptation performance.

[0063] Other functionalities of the methods and systems for scene adaptation of target models provided in this disclosure will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the methods and systems for scene adaptation of target models provided in this disclosure can be fully explained through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 A schematic diagram of a system 100 for scene adaptation of a target model according to some embodiments of the present disclosure is shown;

[0066] Figure 2 A schematic diagram of the structure of a computing device provided according to some embodiments of the present disclosure is shown;

[0067] Figure 3 A schematic diagram of the basic structure of a target model provided according to some embodiments of the present disclosure is shown;

[0068] Figure 4 A flowchart illustrating a method for scene adaptation of a target model according to some embodiments of the present disclosure is shown.

[0069] Figure 5 A basic flowchart of data input and output for a target model provided according to some embodiments of the present disclosure is shown;

[0070] Figure 6A schematic diagram illustrating the calculation of a comprehensive confidence level according to some embodiments of the present disclosure is shown;

[0071] Figure 7 This diagram illustrates a confidence level classification scheme and a corresponding adaptation method according to some embodiments of the present disclosure.

[0072] Figure 8 A flowchart illustrating a method for training a target model according to some embodiments of the present disclosure is shown; and

[0073] Figure 9 A method for calculating an important region map is shown according to some embodiments of the present disclosure. Detailed Implementation

[0074] The following description provides specific application scenarios and requirements for this disclosure, intended to enable those skilled in the art to make and use the content of this disclosure. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0075] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” as used herein may also include the plural forms. When used in this disclosure, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not preclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0076] In view of the following description, these and other features of this disclosure, as well as the operation and function of the related elements of the structure, and the economy of assembly and manufacture of the components, can be significantly improved. All of these form part of this disclosure with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this disclosure. It should also be understood that the drawings are not drawn to scale.

[0077] The flowcharts used in this disclosure illustrate operations implemented according to some embodiments of this disclosure. It should be clearly understood that the operations in the flowcharts may not be implemented sequentially. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0078] For ease of description, the terms that will appear in the following descriptions are explained as follows.

[0079] Heatmaps: These are images typically used to categorize data. Similar to infrared imaging, hotter areas appear red, while cooler areas appear blue. Likewise, heatmaps can represent, in weighted terms, which parts of an image receive the highest activation values ​​from a neural network.

[0080] Gradient-weighted Class Activation Mapping (Grad-CAM): The full-class activation heatmap of the last convolutional layer is weighted by the gradient of the target class on the last convolutional layer of the model, so as to obtain the distribution of the sensitive regions of the model inferring the target class on the training / prediction image, and is displayed in the form of a "heatmap".

[0081] Saliency Map: A saliency map addresses the impact of pixels in an image on the image classification result. Methods for generating saliency maps include calculating the derivative of the output class with respect to the input class. It shows how the model's output class changes when the input pixels undergo small changes. If a region in the input image has a large positive reciprocal of the output class, then even a slight change in this region will have a significant impact on the output class. This region can be considered a region in the input image that significantly affects the model's classification result.

[0082] Feature extraction involves transforming raw features into a set of features with clear physical, statistical, or kernel significance. A key characteristic of feature extraction is "reproducibility," meaning that the features extracted from different images of the same scene should be identical. Features can include image features, audio features, etc.

[0083] Image feature extraction involves extracting accurate, complete, and non-redundant information describing the target object from initial digital image data. In image label prediction, if the extracted features are not precise enough, the judgments made based on those features will inevitably be inaccurate. Furthermore, if the extracted features are not refined enough or the feature space has excessively large dimensions, it can lead to extremely high complexity in subsequent discrimination algorithms, resulting in the "curse of dimensionality." Commonly used image features include geometric features, shape features, color features, texture features, and grayscale features.

[0084] Feature map: The result of a neural network convolution on an input image (such as initial digital image data), representing a feature in the neural space, whose resolution depends on the stride of the previous convolution kernel.

[0085] Local features reflect the unique characteristics of a local area of ​​an image. They are abundant, have low correlation between features, and are not easily affected by noise such as partial occlusion or lighting conditions.

[0086] Important regions: In this disclosure, important regions refer to areas that significantly influence the image label prediction results (such as face regions, clothing regions, etc.). Different images have different important regions. Feature extraction based on important regions of image samples can yield local features of the image samples. For example, the important region can be a saliency map corresponding to the image sample, or a region with a large activation value in the heatmap corresponding to the image sample (e.g., an activation value greater than a preset value, which can be set according to actual conditions), or a region combining both.

[0087] Attention mechanism: In neural networks, attention can be considered a resource allocation mechanism. It can be understood as redistributing resources that were originally evenly distributed according to the importance of the attention object. Important units get more resources, and unimportant or bad units get less resources. In the structural design of deep neural networks, the resources that attention needs to allocate are basically weights.

[0088] Confidence level: Also known as reliability, confidence level, or confidence coefficient, it refers to the uncertainty of the conclusion when estimating population parameters through sampling due to the randomness of the sample. Therefore, a probabilistic statement method is used, namely the interval estimation method in mathematical statistics. The probability that the estimated value is within a certain allowable error range from the population parameter is called the confidence level. For a target scene, assuming that the target scene and the target model are well-fitted, the prediction results before and after data perturbation of the image samples contained in the target scene using the target model should be as consistent as possible (the deviation between the image prediction result after perturbation and the image prediction result before perturbation should be as small as possible). This indicates a high confidence level; conversely, a low confidence level indicates a low confidence level.

[0089] Figure 1 A schematic diagram of a system 100 for scene adaptation of a target model according to some embodiments of the present disclosure is shown. System 100 may include a client 110, a network 120, a server 130, and a database 140.

[0090] Client 110 can be deployed in different target scenarios to collect visual information covered by the target scenarios. This visual information includes at least one of image information or video information. Video consists of still images, which are called frames. In some embodiments, the video information can be broken down into several image samples, each containing several image information. In some embodiments, client 110 may include devices with image acquisition or video recording capabilities, such as computer 110A, mobile phone 110B, camera 110C, tablet 110D, or multi-functional device 110E. The visual information covered by different target scenarios can be collected in real-time by the client or pre-stored in the client 110 or database 140 of system 100 for direct use when scene matching is required.

[0091] Network 120 can facilitate the exchange of information and / or data. For example... Figure 1 As shown, client 110, server 130, and database 140 can connect to network 120 and transmit information and / or data to each other through network 120. In some embodiments, network 120 can be any type of wired or wireless network, or a combination thereof. For example, network 120 may include a cable network, wired network, fiber optic network, telecommunications network, intranet, Internet, local area network (LAN), wide area network (WAN), wireless local area network (WLAN), metropolitan area network (MAN), public switched telephone network (PSTN), Bluetooth network, ZigBee network, near field communication (NFC) network, or similar network. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include one or more wired or wireless network access points through which one or more components of client 110, server 130, and database 140 can connect to network 120 to exchange data and / or information.

[0092] Server 130 may be a computing device in system 100 specifically designed to process methods for scene adaptation of a target model. Server 130 may store data or instructions for performing the methods for scene adaptation of the target model described herein, and may execute or be used to execute said data and / or instructions. Server 130 may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices. Alternatively, server 130 may be solely a hardware device with data processing capabilities, or simply a program running on the hardware device. In some embodiments, server 130 may also be a plug-in deployed on client 110.

[0093] Database 140 may store data and / or instructions. In some embodiments, database 140 may store data and / or instructions executed by server 130 or used to execute methods for scene adaptation of a target model in the program described in this disclosure. Client 110 and server 130 may have access to database 140, and client 110 and server 130 may access data or instructions stored in database 140 via a network. In some embodiments, database 140 may be directly connected to client 110 and server 130. In some embodiments, database 110 may be part of server 130. In some embodiments, database 140 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), or similar content, or any combination thereof. Exemplary mass storage may include non-transitory storage media such as disks, optical discs, and solid-state drives. Exemplary removable storage may include flash drives, floppy disks, optical discs, memory cards, zip disks, magnetic tapes, etc. Typical volatile read-write memory may include random access memory (RAM). Example RAMs may include dynamic RAM (DRAM), dual date rate synchronous dynamic RAM (DDR SDRAM), static RAM (SRAM), thyristor RAM (T-RAM), and zero-capacitance RAM (Z-RAM), etc. Exemplary ROMs may include mask ROM (MROM), programmable ROM (PROM), virtual programmable ROM (PEROM), electronically programmable ROM (EEPROM), optical disc (CD-ROM), and digital multifunction disk ROM, etc.

[0094] It should be understood that Figure 1 The number of clients 110 and servers 130 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 110 and servers 130.

[0095] It should be noted that the method for scene adaptation of the target model can be executed entirely on the client 110, entirely on the server 130, or partially on the client 110 and partially on the server 130.

[0096] For ease of description, the following descriptions of this disclosure will use the method of scene adaptation for the target model executed on server 130 as an example to describe the technical solutions involved in this disclosure.

[0097] Figure 2A schematic diagram of a computing device 200 according to some embodiments of the present disclosure is shown. The computing device 200 can be a general-purpose computer or a special-purpose computer. For example, the computing device 200 can be a server, a personal computer, a portable computer (e.g., a laptop computer, a tablet computer, etc.), or an electronic device with other computing capabilities. Of course, the computing device can be... Figure 1 The server 130 can also be a terminal device on the client 110 used for program development over the network.

[0098] like Figure 2 As shown, the computing device 200 may include a COM port 250, which can be connected to or from a network to facilitate data communication. The computing device 200 may also include a processor 220, such as a central processing unit (CPU), in the form of one or more processors for executing program instructions. The computing device 200 may also include an internal communication bus 210 and various forms of program storage media and data storage media, such as a disk 270 (non-transitory memory) and read-only memory (ROM) 230 or random access memory (RAM) 240, etc., for storing various data files to be processed and / or transmitted. The storage media may be local to the computing device 200 or shared by the computing device 200 (e.g., Figure 1 The computing device 200 may also include program instructions stored in ROM 230, RAM 240, and / or other types of non-transitory storage media to be executed by processor 220. The computing device 200 may also include I / O components 260 to support data communication with other computing devices in the distributed computing system 100. The computing device 200 may also receive programming and data via network communication.

[0099] For illustrative purposes only, only one processor 220 is described in the computing device 200. However, those skilled in the art will understand that the computing device 200 in this application may also include multiple processors. Therefore, the methods / steps / operations performed by one processor as described in this application may also be performed jointly or separately by multiple processors. For example, in this application, the processor of the computing device 200 may simultaneously execute step A and step B. It should be understood that step A and step B may also be performed jointly by two different processors. For example, a first processor executes step A, a second processor executes step B, or a first processor and a second processor jointly execute steps A and B.

[0100] Figure 3A schematic diagram of the basic structure of a target model 300 according to some embodiments of the present disclosure is shown. In this disclosure, the target model 300 is used for image processing, which can be used for portrait prediction or other purposes. Figure 3 As shown, the model structure of the target model 300 can include three parts: a basic feature extraction module, an important region prediction module, and a profile prediction module.

[0101] Figure 4 A method flowchart 400 is shown, illustrating a method for scene adaptation of a target model 300 according to some embodiments of the present disclosure;

[0102] The following will combine Figure 4 This disclosure describes the technical solution. The subject implementing the technical solution may be... Figure 1 The system comprises at least one of client 110, network 120, and server 130. Specifically, the client 110, network 120, and / or server 130 may have the following characteristics: Figure 2 The aforementioned structure, namely the client 110, network 120, and / or server 130, can be a device for scene adaptation of a target model, comprising: at least one storage medium and at least one processor. The at least one storage medium includes at least one instruction set for a method in a program to perform scene adaptation of the target model. The at least one processor is communicatively connected to the at least one storage medium. When the system is running, the at least one processor can read the at least one instruction set and execute instructions according to the at least one instruction set. Figure 4 The method 400. For illustrative purposes only, this application will describe the method 400 as being performed by server 130. The method 400 may include:

[0103] S410, Obtain N image samples from the target scene, where N is a natural number greater than 1;

[0104] In this disclosure, client devices (including computers 110A, mobile phones 110B, cameras 110C, tablets 110D, multi-functional devices 110E, and other devices with image acquisition or video recording functions) can be deployed in a target scene to collect visual information covered by the target scene. The visual information may include at least one of still image information (image samples) or dynamic video information. The video information is composed of frames of images; therefore, for ease of description, this disclosure uses N (N is a natural number greater than 1) image samples to represent the visual information covered by the target scene.

[0105] S420, using the target model 300, determine the important region prediction map and portrait prediction results in the N image samples, wherein the important region prediction map includes the map corresponding to the local features in the N image samples;

[0106] The target model 300 provided in this disclosure can determine the prediction map of important regions in image samples while performing portrait prediction on image samples. In this disclosure, the target model 300 can be a pre-trained neural network model that can be used for portrait prediction. The server 130 can use the target model 300 to perform portrait prediction on N image samples in a target scene, thereby obtaining the prediction map of important regions and the portrait prediction result for the N image samples. The prediction map of important regions can be the map corresponding to local features in the N image samples obtained by the server 130 when performing portrait prediction on the N image samples in the target scene using the target model 300. These local features can be features corresponding to important regions in the image samples. As mentioned above, important regions can include regions that have a significant impact on the portrait label prediction result, such as face regions, clothing regions, etc. The portrait prediction result can be various attribute labels generated by the server 130 after performing user portrait prediction on the N image samples in the target scene using the target model 300.

[0107] In this disclosure, server 130 performs perception prediction of important regions on image samples through target model 300 to obtain important region prediction maps. Compared with traditional portrait prediction technology, which performs image processing based on the global region of image samples, it can reduce the probability of background interference in image samples and improve data processing efficiency.

[0108] In this disclosure, when server 130 performs portrait prediction on N image samples in a target scene using target model 300, each of the N image samples can be used as the input image of target model 300. Furthermore, server 130 can use target model 300 to perform portrait prediction on each image sample, thereby obtaining a portrait prediction result corresponding to each image sample (which can be called a single-sample portrait prediction result). The portrait prediction result of N image samples can be a comprehensive result obtained by superimposing the portrait prediction results (single-sample portrait prediction results) of each image sample included in the N image samples. Therefore, in some embodiments, step S420 may include: for each of the N image samples, server 130 performs the following steps:

[0109] S421, Based on the target model 300, feature extraction is performed on the image sample to obtain the feature map of the image sample;

[0110] Figure 5A basic flowchart of data input and output for a target model 300 provided according to some embodiments of the present disclosure is shown. In step S421, as... Figure 5 As shown, the input of the feature extraction module of the target model 300 is an image sample. The server 130 uses the feature extraction module of the target model 300 to extract image features from the image sample and outputs the feature map (i.e., feature map) of the image sample.

[0111] The feature map can include geometric features, shape features, color features, texture features, and grayscale features of the image sample.

[0112] Step S420 may further include S422, determining the important region prediction map and the portrait prediction result of the image sample based on the feature map. In step S422, as... Figure 5 As shown, server 130 continues to use the feature map as input to the important region prediction module, and outputs the important region prediction map through the important region prediction module in sequence, and then outputs the image prediction result through the image prediction module.

[0113] In some embodiments, step S422 may include:

[0114] S422-a, based on the target model 300, the feature map is predicted to obtain the important region prediction map. In this step, as follows: Figure 5 As shown, server 130 uses the feature map as input to the important region prediction module of target model 300. Server 130 uses the important region prediction module to predict the feature map and outputs the map corresponding to the region that has a greater impact on the image label prediction result (such as face region, clothing region, etc.), i.e. important region prediction map.

[0115] And S422-b, based on the important region prediction map and the feature map, determine the portrait prediction result of the image sample. In this step, such as... Figure 5 As shown, server 130 combines the important region map and feature map as input to the portrait prediction module of target model 300, and uses the portrait prediction module to predict portrait labels and output the final portrait prediction result.

[0116] Specifically, determining the portrait prediction result of the image sample may include: obtaining the product result of the important region prediction map and the feature map; and determining the portrait prediction result of the image sample based on the target model 300 and the product result.

[0117] In this disclosure, server 130 uses the product of the important region prediction map and the feature map as input to the portrait prediction module of target model 300, and uses the portrait prediction module to predict portrait labels, thereby outputting the final portrait prediction result. Using the product of the important region prediction map and the feature map as input to the portrait prediction module of target model 300 can be understood as server 130 allocating resources to important regions of image samples through an attention mechanism. By introducing an attention mechanism, focusing on information more critical to the current task (i.e., the important region prediction map of this disclosure) among numerous input information, reducing attention to other information, and even filtering out irrelevant information, can solve the problem of information overload and improve the efficiency and accuracy of task processing.

[0118] S430, Based on the prediction map of the important region and the prediction result of the portrait, determine the comprehensive confidence of the target model 300 on the processing results of the N image samples in the target scene;

[0119] Figure 6 The illustration shows a schematic diagram of a comprehensive confidence score calculation according to some embodiments of the present disclosure. In this disclosure, the comprehensive confidence score can be determined based on the prediction map of important regions and the image prediction results. Furthermore, the comprehensive confidence score reflects the degree of fit between the target model 300 and the target scene; a higher comprehensive confidence score indicates a higher degree of fit between the target model 300 and the target scene, while a lower comprehensive confidence score indicates a lower degree of fit between the target model 300 and the target scene.

[0120] Since the target scene includes N image samples, for each image sample, after prediction by the target model 300, the degree of fit between the target model 300 and each image sample can be obtained. In this disclosure, the degree of fit between the target model 300 and a single image sample in the target scene can be called the single-sample confidence level. Therefore, in some embodiments, such as... Figure 6 As shown, step S430 may include: for each of the N image samples:

[0121] S431, based on the important region prediction map of the image sample and the portrait prediction result, determine the single-sample confidence level of the image sample; and

[0122] S432, the average of the N single-sample confidence scores is calculated to obtain the comprehensive confidence score of the target scene.

[0123] In some embodiments, the method for determining the single-sample confidence level in step S431 may include:

[0124] S431-a, perform data perturbation on the image sample, and determine the first perturbation deviation of the prediction map of the important region and the second perturbation deviation of the portrait prediction result before and after the data perturbation;

[0125] In this disclosure, the first perturbation bias reflects the reliability of the important region prediction map output by the server 130 when predicting a single sample of the target scene using the target model 300; while the second perturbation bias reflects the reliability of the image prediction result output by the server 130 when predicting a single sample of the target scene using the target model 300. By subjecting the image samples to random signal interference, the reliability and stability of the target model 300 in understanding and analyzing the image samples can be tested.

[0126] In this disclosure, the aforementioned data perturbation methods may include at least one of Gaussian noise, salt-and-pepper noise, or random noise. Gaussian noise refers to noise whose probability density function follows a Gaussian distribution; salt-and-pepper noise is noise caused by the intensity of the signal pulse; random noise, also known as background noise, is caused by the accumulation of a large number of temporally random fluctuations and disturbances, and its value cannot be predicted at a given instant.

[0127] In this disclosure, the aforementioned perturbation methods can be used individually. For example, server 130 can use only Gaussian noise, only salt-and-pepper noise, or only random noise to perturb the image samples. Furthermore, the aforementioned perturbation methods can be used in combination. For example, server 130 can combine Gaussian noise and salt-and-pepper noise to perturb the image samples; or, for another example, server 130 can combine Gaussian noise, salt-and-pepper noise, and random noise to perturb the image samples. Server 130 can also use other perturbation combinations to perturb the image samples, which will not be listed here.

[0128] It should be understood that the above-mentioned data perturbation methods are merely examples of data perturbation methods that may be used in this disclosure. The data perturbation methods that may be used in this disclosure are not limited to the data perturbation methods listed above, but may also be other types of image data perturbation methods.

[0129] In some embodiments, S431-a may include:

[0130] The image samples are subjected to M data perturbations, where M is a natural number greater than 1. For each of the M data perturbations, the following processing method is included:

[0131] Based on the target model 300, the first deviation of the predicted map of the important region before and after the data perturbation is determined, and the average of the M first deviations is calculated to obtain the first perturbation deviation; and

[0132] Based on the target model 300, a second deviation of the portrait prediction result before and after the data perturbation is determined, and the average of the M second deviations is calculated to obtain the second perturbation deviation.

[0133] In this disclosure, such as Figure 6 As shown, when server 130 calculates the average of M first deviations or M second deviations, for ease of description, the image samples can be initial image samples and perturbed image samples, respectively. The image sample before the data perturbation is the initial image sample, and the image sample after the data perturbation is the perturbed image sample. Generally, when server 130 calculates the confidence level of a single sample, for a single image sample, the number of initial image samples is 1, while the number of perturbed image samples is determined according to the number of times server 130 perturbs the image sample (M times). If server 130 performs M data perturbations on the image sample, then M perturbed image samples are obtained. The M data perturbations can be 20, 30, 50, 100, or any other number. Correspondingly, the perturbed image samples corresponding to a single image sample can be 20, 30, 50, 100, or any other number. Accordingly, the M perturbed image samples correspond to M first deviations and M second deviations.

[0134] In some embodiments, determining the first deviation of the predicted map of the important region before and after the data perturbation based on the target model includes:

[0135] Based on the target model 300, the initial important region prediction map of the initial image sample is determined. In this step, the server 130 inputs the initial image sample into the target model 300 to obtain the initial important region prediction map.

[0136] Based on the target model 300, a predicted map of the important disturbance regions of the disturbed image samples is determined. In this step, the server 130 inputs the disturbed image samples into the target model 300 to obtain the predicted map of the important disturbance regions; and

[0137] Based on the initial important region prediction map and the perturbation important region prediction map, the first deviation of the image sample is determined. In this disclosure, the important region prediction map includes the initial important region prediction map and the perturbation important region prediction map. In this step, both the initial important region prediction map and the perturbation important region prediction map are image data after feature extraction of the image sample. Both the initial important region prediction map and the perturbation important region prediction map can be represented by matrix data, and the matrix corresponding to the initial important region prediction map is the same size as the matrix corresponding to the perturbation important region prediction map. At this time, the server 130 can obtain the difference between two elements by performing matrix operations on two elements in the same position in the two matrices, and take the absolute value of each element in the calculated matrix. Finally, the absolute values ​​of the differences between the elements at each position are mathematically calculated to obtain a comprehensive difference, which can be used as the first deviation.

[0138] To facilitate understanding, the calculation method for the first deviation by server 130 will be illustrated below. Assume an initial important region prediction map can be represented by matrix A, and a perturbed important region prediction map can be represented by matrix B. Matrix A is... Matrix B is Server 130 can perform matrix subtraction on matrices A and B to obtain matrix C. Continuing with matrix C, i.e. Take the absolute value of each element to obtain the matrix. Finally, the elements of matrix D are summed, resulting in 1+1+1+2+2+5+3+1+2=18. Here, 18 can represent the first deviation.

[0139] It should be understood that the specific calculation process of the first deviation by server 130 described above is only an example for ease of understanding. Matrix A and matrix B are merely examples and do not represent the actual matrix data corresponding to the initial important region prediction map or the perturbation important region map. Matrix A and matrix B do not have a limiting effect on the actual matrix data corresponding to the initial important region prediction map or the perturbation important region map.

[0140] In some embodiments, determining a second deviation of the profile prediction result before and after the data perturbation based on the target model 300 includes:

[0141] Based on the target model 300, the initial portrait prediction result of the initial image sample is determined. In this step, such as... Figure 6 As shown, server 130 can input initial image samples into target model 300 to obtain initial portrait prediction results;

[0142] Based on the target model 300, the perturbed image prediction result of the perturbed image sample is determined. In this step, such as... Figure 6 As shown, server 130 can input perturbed image samples into target model 300 to obtain perturbed image prediction results; and

[0143] Based on the initial portrait prediction result and the perturbed portrait prediction result, the second deviation of the image sample is determined. In this disclosure, the portrait prediction result includes the initial portrait prediction result and the perturbed portrait prediction result. In this step, both the initial portrait prediction result and the perturbed portrait prediction result are portrait prediction results obtained after the image sample is predicted for portrait labels by the target model 300. The portrait prediction result can be expressed as scalar data such as prediction probabilities corresponding to various attribute labels or matching values.

[0144] Taking the prediction probability as an example, for the same image sample, if the initial image prediction result obtained by server 130 using target model 300 shows that the probability of a certain attribute in the initial image sample is 88%, while the prediction result of the perturbed image shows that the probability of the same attribute in the perturbed image sample is 95%, then the second deviation is the absolute value of the difference between these two prediction probabilities, which is equal to 7%.

[0145] It should be understood that the specific calculation process of the second deviation by server 130 described above is only an example for ease of understanding. The prediction probability is merely an example and does not represent the actual initial image prediction result or the perturbation image prediction result. The prediction probability does not limit the representation of the actual initial image prediction result or the perturbation image prediction result.

[0146] S431-b, Based on the first perturbation bias and the second perturbation bias, determine the single-sample confidence level.

[0147] In this disclosure, the server 130 determines the confidence level of a single sample by calculating the following formula: Single sample confidence level = 100 - (first perturbation bias + second perturbation bias) / 2.

[0148] In this disclosure, the method 400 further includes:

[0149] S440, based on the comprehensive confidence score, the target model 300 is adapted to the target scene. As mentioned above, the comprehensive confidence score is the result of averaging the confidence scores of N individual samples. The comprehensive confidence score reflects the overall confidence of the target model 300 in predicting the N image samples contained in the target scene. Therefore, based on the comprehensive confidence score, the server 130 can evaluate the accuracy of the target model 300 in predicting the image of the target scene, that is, the adaptation of the target model 300 to the target scene, in order to determine what adjustments to make to the target model 300 to adapt it to the target scene.

[0150] In some embodiments, step S440 may include:

[0151] S441, the comprehensive confidence level is divided into confidence levels to obtain the confidence level of the target scene. In this disclosure, the server 130 divides the comprehensive confidence level into levels. The obtained confidence level can reflect the accuracy of the target model 300 in predicting the target scene's profile, or it can be understood as the fit between the target model 300 and the target scene. If the confidence level is higher, it indicates that the server 130 has higher accuracy in predicting the target scene's profile using the target model 300, and correspondingly, the fit between the target model 300 and the target scene is better. Conversely, if the confidence level is lower, it indicates that the server 130 has lower accuracy in predicting the target scene's profile using the target model 300, and correspondingly, the fit between the target model 300 and the target scene is worse.

[0152] In this disclosure, server 130 can divide the confidence level into several different levels. For example, server 130 can divide the confidence level into 2 levels, 3 levels, 4 levels, 5 levels, or other numbers of levels.

[0153] In this disclosure, the confidence level can be determined by setting thresholds. For example, server 130 can set two thresholds T1 and T2 (T1 > T2) to divide the overall confidence level into three levels. Specifically, server 130 can define an overall confidence level greater than T1 as a high confidence level, an overall confidence level between T1 and T2 as a medium confidence level, and an overall confidence level less than T2 as a low confidence level.

[0154] Generally, the more confidence levels there are, the more refined the distinction of the overall confidence level, and correspondingly, the more types of adaptation schemes the server 130 can subsequently develop for the target model 300 in the target scenario.

[0155] It should be understood that the number of confidence levels obtained by server 130 in dividing the overall confidence level and the method of dividing the overall confidence level are merely examples for ease of understanding and do not limit the actual number of confidence levels or the method of dividing the level. In this disclosure, the overall confidence level can be divided into more confidence levels and other methods can be used for level division.

[0156] S442, adapt the target model 300 to the target scenario based on the confidence level.

[0157] In some embodiments, such as Figure 7 As shown, Figure 7 This illustration shows a confidence level classification diagram and a corresponding adaptation method according to some embodiments of the present disclosure. The confidence levels may include a first level of confidence, a second level of confidence, and a third level of confidence, wherein the confidence level of the first level of confidence is greater than the confidence level of the second level of confidence, and the confidence level of the second level of confidence is greater than the confidence level of the third level of confidence. In this disclosure, the first level of confidence may also be referred to as a high confidence level, the second level of confidence may also be referred to as a medium confidence level, and the third level of confidence may also be referred to as a low confidence level.

[0158] In this disclosure, when server 130 divides the confidence level into three levels (first-level confidence, second-level confidence, and third-level confidence), step S442 may include the following three cases:

[0159] In the first scenario, server 130 determines the confidence level to be the first level of confidence (high confidence level) and does not change the parameters of the target model 300. In this disclosure, when the confidence level is the first level of confidence, it indicates that the target model 300 has a higher accuracy in predicting user profiles for the target scene, and correspondingly, the target model 300 is better adapted to the target scene. In this case, server 130 can directly deploy the target model 300 in the target scene without changing its parameters and perform user profile prediction for the target scene.

[0160] In the second scenario: Server 130 determines that the confidence level is the second level (medium confidence level) and adjusts the decision threshold of the target model 300. In this disclosure, when the confidence level is the second level, it indicates that the accuracy of the target model 300 in predicting user profiles for the target scene is generally low, and correspondingly, the fit between the target model 300 and the target scene is also generally low. At this time, server 130 can adjust the decision threshold in the target model 300; server 130 can either increase or decrease the decision threshold. Furthermore, for target models that typically contain multiple decision thresholds, server 130 can increase some decision thresholds and decrease the rest. The adjustment of the decision threshold in the target model 300 by server 130 does not change the network structure or number of network layers of the target model. Afterwards, server 130 deploys the target model 300 with the adjusted decision threshold in the target scene and performs user profile prediction for the target scene.

[0161] The third scenario: Server 130 determines that the confidence level is the third level (low confidence level) and retrains the target model 300. In this disclosure, when the confidence level is the third level, it indicates that the target model 300 has low accuracy in predicting the image of the target scene, and correspondingly, the fit between the target model 300 and the target scene is low. At this time, server 130 can fine-tune the target model 300, that is, use image samples contained in the target scene as input to retrain the target model 300 until the fit between the retrained target model 300 and the target scene reaches the expected level (first level confidence level).

[0162] In this disclosure, server 130 first uses target model 300 to predict N image samples contained in the target scene. While predicting the portrait prediction results of the N image samples, it also predicts the important regions (important region prediction map) of the N image samples. Based on the predicted portrait label results and important region prediction map, it estimates the comprehensive confidence of the target scene. Then, based on the comprehensive confidence, it divides the confidence level. Finally, it determines the corresponding adaptation strategy according to the confidence level. This solves the trade-off between adaptation performance and adaptation efficiency in the cross-scene adaptation problem of visual portrait technology. While ensuring adaptation performance, it minimizes adaptation complexity and improves adaptation efficiency.

[0163] Figure 8 A flowchart 500 illustrating a method for training a target model according to some embodiments of the present disclosure is shown. Figure 8 As shown, the training method 500 for the target model may include:

[0164] S510, acquire the N image samples in the target scene; and

[0165] S520, the target model 300 is trained based on the N image samples to obtain the trained target model 300. As mentioned above, the server 130 can use the image samples contained in the target scene as input to retrain the target model 300. In this disclosure, if the target scene contains N image samples, then the N image samples are used as input to train the target model 300.

[0166] Step S520 may include:

[0167] S521, for each of the N image samples: feature extraction is performed on the image sample based on the target model 300 to obtain the feature map of the image sample. In this disclosure, step S521 is basically the same as step S421. The server 130 uses the feature extraction module of the target model 300 to extract image features from the image sample and outputs the feature map (i.e., feature map) of the image sample.

[0168] S522, based on the feature map and the target model 300, the important region prediction map and the portrait prediction result of the image sample are determined. In this disclosure, step S522 is basically the same as step S422. The server 130 continues to use the feature map as the input of the important region prediction module, and outputs the important region prediction map through the important region prediction module in sequence, and then outputs the portrait prediction result through the portrait prediction module.

[0169] S523, based on the important region prediction map and the portrait prediction result, the target model 300 is converged to obtain the trained target model 300. In this disclosure, during the training process of the target model 300, the server 130 can combine the information from both the important region prediction map and the portrait prediction result to converge the target model 300.

[0170] In some embodiments, step S523 may include:

[0171] S523-a, Obtain the important region map of the image sample, and compare the important region map with the predicted important region map to obtain important region loss information. In this disclosure, the important region map can be understood as the map corresponding to the region (such as the face region, clothing region, etc.) in the image sample that has a significant impact on the image label prediction result. In this disclosure, the important region loss information can be understood as the loss information formed by the difference between the important region map and the predicted important region map. This loss information can be used to constrain the predicted important region map obtained by the target model 300 to be consistent with the important region map.

[0172] In this disclosure, the server 130 can acquire the important region map of the image sample in various ways. For example, such as Figure 9 As shown, Figure 9 This illustration shows a method for calculating an important region map according to some embodiments of the present disclosure. The method by which the server 130 obtains the important region map of the image sample may include:

[0173] Based on a preset portrait labeling model, feature extraction is performed on the image sample to obtain a sensitive region map of the image sample. In this step, the sensitive region map can be understood as a map of sensitive regions in the image sample that can affect the inference of the portrait prediction result. In this disclosure, the server 130 can obtain the sensitive region map of the image sample in various ways. For example, after acquiring the image sample, the server 130 can use a preset portrait labeling model to extract features from the image sample, thereby obtaining the sensitive region map of the image sample. In some embodiments, such as... Figure 9 After obtaining the image sample, the server 130 can also combine the preset image label model with GradCAM and extract features from the image sample to obtain the sensitive area map of the image sample.

[0174] The image samples are saliency predicted based on a preset saliency detection model to obtain a saliency map of the image samples. In this step, the saliency map of the image samples can be understood as the map corresponding to the regions (e.g., face regions) in the image samples that have a significant impact on the portrait prediction results. After obtaining the image samples, the server 130 can use the preset saliency detection model to predict the saliency of the image samples, thereby outputting the saliency map (saliency map) of the image samples.

[0175] The important region map of the image sample is determined based on the sensitive region map and the saliency map. After obtaining the sensitive region map and the saliency map, the server 130 can calculate the mean of the sensitive region map and the saliency map to obtain the important region map of the image sample. The corresponding calculation method can be expressed as: Important region map = (Sensitive region map + Saliency map) / 2. Both the sensitive region map and the saliency map mentioned above can be represented using matrix vectors. Therefore, the server 130 can use matrix algorithms to calculate the important region map.

[0176] After obtaining the map of the important region, server 130 can compare the map of the important region with the predicted map of the important region to determine the differences between the two maps, thereby obtaining the loss information of the important region.

[0177] S523-b, Obtain the original image classification label of the image sample, compare the original image classification label with the image prediction result, and obtain image classification loss information. In this disclosure, the original image classification label can be understood as a pre-labeled classification label for the image sample. In this disclosure, there are multiple ways to obtain the original image classification label of the image sample. For example, server 130 can directly obtain the original image classification label corresponding to the image sample; or, server 130 can also send the image sample to a classification labeling server (the classification labeling server can label the image sample to obtain the original image classification label), and receive the original image classification label corresponding to the image sample returned by the classification labeling server.

[0178] In this disclosure, the image classification loss information can be understood as the loss information formed by the difference between the pre-labeled classification labels and the image prediction results. This loss information can be used to constrain the image prediction results obtained by the target model 300 to be consistent with the pre-labeled classification labels. After obtaining the original image classification labels of the image samples, the server 130 can compare the original image classification labels with the image prediction results (there can be multiple comparison methods; for example, the server 130 can use a classification loss function to determine the difference between the pre-labeled classification labels and the image prediction results) to obtain the image classification loss information.

[0179] In this disclosure, the classification loss function can be of various types. For example, the classification loss function can be the mean squared error loss function, the cross-entropy loss function, or other loss functions that can be used to determine classification loss information.

[0180] S523-c, the loss information of the important region and the loss information of the portrait classification are fused to obtain the loss information of the target model.

[0181] In this disclosure, the target model loss information (Loss) involved in the training process of the target model 300 is discussed. model It can include two parts. The first part is the loss information of important areas. region The second part is the image label prediction loss (Loss). cls Loss information for important areas. region ) and profile classification loss information (Loss) cls There are several ways to integrate data. For example, server 130 can integrate loss information from important areas (Loss). region ) and profile classification loss information (Loss) cls The loss information of the target model is obtained by adding the two loss information together. model ), that is, Loss model =Loss region+Loss cls .

[0182] In some embodiments, server 130 can also obtain loss information for important areas. region ) and portrait classification loss information (Loss cls Each region has its own loss weight, and based on these weights, the loss information for important regions is analyzed. region ) and portrait classification loss information (Loss cls The weighted values ​​are then used to calculate the loss information for the important regions. region ) and portrait classification loss information (Loss cls The loss information of the target model is obtained by adding the two loss information together. model ).

[0183] After obtaining the target model loss information, the server 130 can execute step S523-d to converge the target model 300 based on the target model loss information, thereby obtaining the trained target model 300.

[0184] In this disclosure, server 130 can perform performance training on target model 300 based on the aforementioned model structure and target model loss information until target model 300 converges, thereby obtaining trained target model 300. Server 130 can use various methods to converge target model 300 based on target model loss information. For example, server 130 can use a gradient descent algorithm to update the network parameters of target model 300 based on target model loss information until target model 300 converges, thereby obtaining trained target model 300. Furthermore, server 130 can also use other parameter update algorithms to update the network parameters of target model 300 based on target model loss information until target model converges, thereby obtaining trained target model 300.

[0185] It should be understood that the above-mentioned convergence method is merely an example of the convergence method of the target model 300 based on the target model loss information in this disclosure, and it does not limit the actual convergence method of the target model 300 based on the target model loss information in this disclosure.

[0186] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this disclosure is intended to encompass various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this disclosure and are within the spirit and scope of the exemplary embodiments of this disclosure.

[0187] Furthermore, certain terms used in this disclosure have been used to describe embodiments of this disclosure. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this disclosure. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this disclosure do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this disclosure.

[0188] It should be understood that in the foregoing description of the embodiments of this disclosure, various features are sometimes combined in a single embodiment, drawing, or description for the purpose of simplifying the disclosure and to aid in understanding a feature. Alternatively, various features may be distributed across multiple embodiments of this disclosure. However, this does not mean that the combination of these features is necessary, and those skilled in the art may extract some features as separate embodiments when reading this disclosure. That is, the embodiments in this disclosure can also be understood as an integration of multiple sub-embodiments. It is also possible for each sub-embodiment to contain fewer features than all of the features of a single foregoing disclosed embodiment.

Claims

1. A method for scene adaptation of a target model, the target model being used for image processing, comprising: Obtain N image samples from the target scene, where N is a natural number greater than 1; Using the target model, the prediction maps and portrait prediction results of important regions in the N image samples are determined, wherein the prediction maps of important regions include maps corresponding to local features in the N image samples; For each of the N image samples: the image sample is perturbed, the first perturbation deviation of the prediction map of the important region and the second perturbation deviation of the portrait prediction result are determined before and after the data perturbation, and the single sample confidence of the image sample is determined based on the first perturbation deviation and the second perturbation deviation; Based on the individual sample confidence scores corresponding to the N image samples, the overall confidence score of the target model's processing results for the N image samples in the target scene is determined; and Based on the comprehensive confidence level, the adaptation scheme of the target model for the target scenario is determined.

2. The method of claim 1, wherein, The determination of the prediction atlas and portrait prediction results of important regions in the N image samples includes: For each of the N image samples: Based on the target model, feature extraction is performed on the image samples to obtain the feature map of the image samples; and Based on the feature map, the important region prediction map and the portrait prediction result of the image sample are determined.

3. The method of claim 2, wherein, The process of determining the important region prediction map and the portrait prediction result of the image sample based on the feature map includes: Based on the target model, the feature map is predicted to obtain the predicted map of the important region; and Based on the important region prediction map and the feature map, the portrait prediction result of the image sample is determined.

4. The method of claim 3, wherein, The process of determining the portrait prediction result of the image sample includes: Obtain the product of the predicted map of the important region and the feature map; and The image prediction result of the image sample is determined based on the target model and the product result.

5. The method of claim 1, wherein, The determination of the overall confidence level of the target scene based on the individual sample confidence levels corresponding to the N image samples includes: The average of the N single-sample confidence scores is used to calculate the overall confidence score of the target scenario.

6. The method of claim 1, wherein, The determination of the first perturbation bias of the predicted map of the important region before and after the data perturbation and the second perturbation bias of the portrait prediction result includes: The image samples are subjected to M data perturbations, where M is a natural number greater than 1. For each of the M data perturbations: based on the target model, a first deviation of the predicted map of the important region before and after the data perturbation is determined; the average of the M first deviations is calculated to obtain the first perturbation deviation; and... Based on the target model, a second deviation of the portrait prediction result before and after the data perturbation is determined, and the average of the M second deviations is calculated to obtain the second perturbation deviation.

7. The method of claim 6, wherein, The image samples include initial image samples and perturbed image samples. The image samples before the data perturbation are the initial image samples, and the image samples after the data perturbation are the perturbed image samples. as well as The determination of the first deviation of the predicted map of the important region before and after the data perturbation based on the target model includes: Based on the target model, the initial important region prediction map of the initial image sample is determined. Based on the target model, the predicted map of the important perturbation region of the perturbation image sample is determined, and Based on the initial important region prediction map and the perturbation important region prediction map, the first deviation of the image sample is determined, wherein the important region prediction map includes the initial important region prediction map and the perturbation important region prediction map.

8. The method of claim 7, wherein, The second deviation of the profile prediction results before and after the data perturbation, determined based on the target model, includes: The initial portrait prediction result of the initial image sample is determined based on the target model; Based on the target model, the perturbed image prediction result of the perturbed image sample is determined; and Based on the initial image prediction result and the perturbed image prediction result, the second deviation of the image sample is determined, wherein the image prediction result includes the initial image prediction result and the perturbed image prediction result.

9. The method of claim 1, wherein, The data perturbation method includes at least one of Gaussian noise, salt-and-pepper noise, or random noise.

10. The method of claim 1, wherein, The step of determining the adaptation scheme of the target model for the target scenario based on the comprehensive confidence level includes: The overall confidence level is divided into confidence levels to obtain the confidence level of the target scene; and Based on the confidence level, the adaptation scheme of the target model for the target scenario is determined.

11. The method of claim 10, wherein, The confidence levels include a first level of confidence, a second level of confidence, and a third level of confidence. The confidence level of the first level of confidence is greater than that of the second level of confidence, and the confidence level of the second level of confidence is greater than that of the third level of confidence.

12. The method of claim 11, wherein, The step of determining the adaptation scheme of the target model for the target scenario based on the comprehensive confidence score includes: The confidence level is determined to be the first level of confidence, without changing the parameters of the target model; Determine the confidence level as the second level of confidence, and adjust the decision threshold of the target model; or The confidence level is determined to be the third level of confidence, and the target model is retrained.

13. The method of claim 1, wherein, The training process of the target model includes: Obtain the N image samples in the target scene; and The target model is trained based on the N image samples to obtain the trained target model.

14. The method of claim 13, wherein, The training of the target model based on the N image samples includes: For each of the N image samples: feature extraction is performed on the image sample based on the target model to obtain the feature map of the image sample; Based on the feature map and the target model, the important region prediction map and the portrait prediction result of the image sample are determined; and Based on the predicted map of the important region and the predicted image, the target model is converged to obtain the trained target model.

15. The method of claim 14, wherein, The step of converging the target model based on the predicted map of the important region and the predicted image to obtain the trained target model includes: Obtain the important region map of the image sample, compare the important region map with the predicted important region map to obtain important region loss information; Obtain the original image classification label of the image sample, compare the original image classification label with the image prediction result, and obtain image classification loss information; The loss information of the important regions and the loss information of the portrait classification are fused to obtain the target model loss information; and The target model is converged based on the target model loss information to obtain the trained target model.

16. The method of claim 15, wherein, The process of obtaining the important region map of the image sample includes: Based on a preset image labeling model, feature extraction is performed on the image samples to obtain a sensitive region map of the image samples. The saliency of the image samples is predicted based on a pre-defined saliency detection model, resulting in a saliency map of the image samples; and The important region map of the image sample is determined based on the sensitive region map and the saliency map.

17. A system for scene adaptation of a target model, comprising: At least one storage medium, including at least one instruction set, for implementation analysis of a method for scene adaptation of a target model; as well as At least one processor is communicatively connected to the at least one storage medium. When the system is running, the at least one processor reads the at least one instruction set and executes the method of any one of claims 1-16 according to the instructions of the at least one instruction set.

Images