A method for constructing adaptive supervisory signals applied to model sustained testing

Abstract

This invention relates to the field of artificial intelligence technology and discloses a method for constructing adaptive supervision signals for continuous model testing. The method includes: constructing a synthetic knowledge base; obtaining a set of test images at the current time step to construct a joint image set; performing style bridging on the synthetic images in the joint image set based on the test images to obtain stylized synthetic images; extracting features from the stylized synthetic images and performing statistical alignment based on the test images to obtain a target shallow feature map; extracting deep features from the target shallow feature map and the test images; and calculating a multi-dimensional loss based on the deep features as a supervision signal. This invention adapts the synthetic images to the current target domain through style injection and statistical alignment, and constructs accurate, comprehensive, and unbiased supervision signals by calculating multi-dimensional losses. This solves the problem of scarce and unreliable supervision signals in adaptive scenarios during continuous testing, and helps support forward facilitation for continuous model adaptation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to a method for constructing adaptive supervision signals for continuous model testing. Background Technology

[0002] In practical applications such as autonomous driving and online perception systems, after visual perception models are trained offline and deployed to real-world testing environments, they continuously adjust their parameters online and in real-time to adapt to the dynamic drift of data distribution, maintaining stable inference / recognition performance in response to constantly changing unlabeled test data, without accessing the original training data or retraining the model. This is known as Continuous Test-Time Adaptation (CTTA). However, factors such as lighting, weather, and imaging equipment in real-world testing environments are constantly changing, causing continuous and unpredictable drift between the actual collected test data and training data. The model must adjust its parameters to adapt to the testing environment, which necessitates the use of supervisory signals.

[0003] Existing technologies employ two methods to construct supervisory signals: one is the backward alignment method, which uses the model training data as a static anchor point to construct a supervisory agent, aligns the test data with this static anchor point, and thus guides the model adjustment; the other is the generative diffusion model method, which uses generative techniques to construct static synthetic domain knowledge as an anchor point, projects the test data back to the static synthetic domain, and constructs a supervisory signal through denoising generation to guide the model adaptation.

[0004] However, both of the aforementioned methods essentially constrain dynamically changing test environments with static anchors. But the knowledge of static anchors and the dynamic test environment are not perfectly matched, meaning the supervision signals based on them inherently contain errors—essentially using flawed references to guide model adjustments. Furthermore, in CTTA, the errors in noisy supervision signals accumulate continuously, and because the model is not perfectly aligned with static anchors, it gradually forgets the effective knowledge adapted to the previous stage's environment, ultimately leading to error accumulation and catastrophic forgetting. In addition, the generative diffusion model method suffers from additional generation bias; the projection of test data into the synthetic domain further reduces the reliability of the supervision signals. In summary, existing technologies cannot solve the problem of scarce and unreliable supervision signals in CTTA scenarios, causing model performance to continuously decline with environmental changes. Summary of the Invention

[0005] This invention provides an adaptive supervision signal construction method for continuous model testing, which solves the problem of scarce and unreliable supervision signals in CTTA scenarios, leading to a continuous decline in model performance as the environment changes.

[0006] In a first aspect, the present invention provides a method for constructing adaptive supervision signals for continuous model testing, the method comprising: Multiple synthetic images are generated for each category label, and each synthetic image is bound to its corresponding category label to build a synthetic knowledge base; Obtain the test image set at the current time step, and construct a joint image set based on the test image set and the synthetic knowledge base. Each synthetic image in the joint image set corresponds one-to-one with a test image. For each synthesized image in the joint image set, style bridging is performed on the synthesized image based on the test image corresponding to the synthesized image to obtain a stylized synthesized image; Feature extraction is performed on the stylized synthetic image corresponding to the synthetic image, and statistical alignment is performed based on the test image to obtain the target shallow feature map of the synthetic image; Based on the target shallow feature map of each synthesized image, extract the corresponding deep features, and extract the deep features of each test image. Based on each test image and its deep features in the joint image set, and each synthetic image and its corresponding stylized synthetic image and deep features, a multidimensional loss is calculated as a supervision signal for the target model at the current time step.

[0007] This invention generates multiple synthetic images for each category label to construct a stable and semantically pure synthetic knowledge base, which serves as a reliable source of supervision throughout the adaptation process. A set of test images for the current time step is obtained, and a joint image set is constructed based on the test image set and the synthetic knowledge base. For synthetic images in the joint image set, the visual style of the target domain contained in the corresponding test image is injected into the synthetic image. This ensures that the stylized synthetic image retains both the pure semantic information of the synthetic image itself and the true visual style features of the current target domain, achieving alignment between the synthetic data and the target domain test data at the style level. Then, the synthetic image and the test image are statistically aligned at the feature map level to obtain a shallow feature map adapted to the current target domain, reducing model inference errors caused by cross-domain feature drift. Further feature extraction is performed on the shallow feature map and the test image to obtain deep features. Based on the image and deep features of all samples, a multi-dimensional loss is calculated as the supervision signal for the current time step. This ensures that the supervision signal has both semantic reliability and stability for model adaptation, thus constructing an accurate, comprehensive, and unbiased supervision signal for the target model. This fundamentally solves the problem of scarce and unreliable supervision signals in adaptive scenarios during continuous testing, and helps support forward facilitation for continuous model adaptation, preventing the model performance from continuously declining with environmental changes.

[0008] In one alternative implementation, a joint image set is constructed based on the test image set and the synthetic knowledge base, including: Select a set of synthetic images from the synthetic knowledge base that is the same size as the test image set; The test image set and the synthetic image set are aligned in index order so that each test image in the aligned test image set corresponds one-to-one with a synthetic image in the synthetic image set, thus obtaining a joint image set.

[0009] This embodiment filters the synthetic image set and aligns it with the test image set according to the index order to form a joint image set. It establishes an association mapping between the synthetic images and the test images, which helps to perform subsequent cross-domain style alignment.

[0010] In one optional implementation, for each synthesized image in the joint image set, style bridging is performed on the synthesized image based on the test image corresponding to the synthesized image to obtain a stylized synthesized image, including: For each synthesized image in the joint image set, the phase spectrum of the synthesized image is extracted using Fourier transform; Fourier transform is used to extract the amplitude spectrum of the test image corresponding to the synthesized image; The phase spectrum and amplitude spectrum are integrated and then subjected to inverse Fourier transform to obtain a stylized composite image.

[0011] This embodiment reconstructs a stylized synthetic image by replacing the amplitude spectrum of the synthetic image with the amplitude spectrum of the test image, thereby achieving efficient injection of the visual style of the current target domain test image into the synthetic image at the input layer.

[0012] In one optional implementation, feature extraction is performed on the stylized synthetic image corresponding to the synthetic image, and statistical alignment is performed based on the test image to obtain the target shallow feature map of the synthetic image, including: Using the target model, shallow feature extraction is performed on the stylized synthetic image corresponding to the synthetic image to obtain the first shallow feature map. The first mean and the first standard deviation of the first shallow feature map are then calculated. Using the target model, shallow features are extracted from the test image corresponding to the synthesized image to obtain the second shallow feature map. The second mean and the second standard deviation of the second shallow feature map are then calculated. The first shallow feature map is normalized based on the first mean and the first standard deviation; Based on the second mean and the second standard deviation, the normalized first shallow feature map is rescaled to obtain the target shallow feature map of the synthesized image.

[0013] This embodiment eliminates inherent distributional differences by normalizing the shallow features of the synthesized image and then rescaling it based on the statistics of the test image, so that the statistics of the shallow features of the stylized synthesized image perfectly match the test image in the target domain. This achieves deep fusion of synthesized data and real target domain data at the statistical layer, making up for the shortcomings of style injection only at the input layer. When performing subsequent deep feature extraction, it can effectively reduce the model inference error caused by cross-domain feature distribution drift, and help the model achieve stable and efficient adaptive learning in the dynamically changing target domain.

[0014] In an alternative implementation, based on each test image and its deep features in the joint image set and each synthesized image and its corresponding stylized synthesized image and deep features, a multidimensional loss is calculated as a supervision signal for the target model at the current time step. The method further includes: Generate pseudo-labels for each test image in the joint image set; For each synthesized image in the joint image set, other synthesized images in the joint image set with the same category label as the synthesized image, as well as test images with the same pseudo-label as the category label of the synthesized image, are used as positive samples of the synthesized image.

[0015] This embodiment generates pseudo-labels for test images, thereby selecting consistent positive samples based on category labels, which helps the model learn domain-invariant core semantic features.

[0016] In one optional implementation, a multi-dimensional loss is calculated based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, as a supervision signal for the target model at the current time step, including: Based on the deep features of all images in the joint image set, the first loss is calculated using each synthesized image as an anchor point, the positive samples of the synthesized images as positive examples, and the other images in the joint image set as negative examples. For each synthesized image in the joint image set, the target model is used to predict the stylized generated image of the synthesized image to obtain the first prediction distribution; For each test image in the joint image set, the target model and the teacher model of the target model are used to predict the test image respectively, and the second prediction distribution and the third prediction distribution are obtained. The second loss is calculated based on the category label and the first prediction distribution corresponding to each synthesized image in the joint image set; The third loss is calculated based on the second and third prediction distributions corresponding to each test image in the joint image set. The sum of the first loss, the second loss, and the third loss is calculated as the multi-dimensional loss, which is then used as the supervision signal for the target model at the current time step.

[0017] This embodiment achieves multi-dimensional collaborative optimization of deep semantic alignment, category prediction accuracy, and model learning stability through hierarchical and multi-dimensional loss design and fusion, which helps the target model achieve efficient and stable online adaptation in dynamically changing target domains.

[0018] In one alternative implementation, the method further includes: Update the target model based on the supervision signal at the current time step; Based on the updated target model, the teacher model is updated using an exponential moving average to complete the current time step and proceed to the next time step.

[0019] This embodiment uses the multi-dimensional loss supervision signal obtained at the current time step as the optimization basis to achieve adaptive parameter optimization of the target model in the current target domain, thereby improving the robustness and adaptability of the target model in dynamic environments. Based on the target model with updated parameters, the network parameters of the teacher model are updated synchronously using an exponential moving average method, so that the parameters of the teacher model always maintain smooth iteration. The adaptive process ends when the continuous testing at the current time step ends, and the online adaptive optimization of the model continues in the next time step.

[0020] Secondly, the present invention provides an adaptive supervision signal construction device for continuous model testing, the device comprising: The first construction module is used to generate multiple synthetic images for each category label, bind each synthetic image to its corresponding category label, and build a synthetic knowledge base. The second construction module is used to obtain the test image set at the current time step, and to construct a joint image set based on the test image set and the synthesis knowledge base. Each synthesized image in the joint image set corresponds one-to-one with the test image. The first bridging module is used to perform style bridging on each synthesized image in the joint image set based on the test image corresponding to the synthesized image, so as to obtain a stylized synthesized image. The second bridging module is used to extract features from the stylized synthetic image corresponding to the synthetic image and perform statistical alignment based on the test image to obtain the target shallow feature map of the synthetic image. The extraction module is used to extract the corresponding deep features based on the target shallow feature map of each synthesized image, and to extract the deep features of each test image. The determination module is used to calculate a multi-dimensional loss based on each test image and its deep features in the joint image set, and each synthetic image and its corresponding stylized synthetic image and deep features, as a supervision signal for the target model at the current time step.

[0021] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the adaptive supervision signal construction method for continuous model testing described in the first aspect or any corresponding embodiment.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the adaptive supervision signal construction method applied to continuous model testing as described in the first aspect or any corresponding embodiment. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a flowchart of an adaptive supervision signal construction method for continuous model testing according to an embodiment of the present invention; Figure 3 This is a flowchart of another adaptive supervision signal construction method applied to continuous model testing according to an embodiment of the present invention; Figure 4 This is a structural block diagram of an adaptive supervision signal construction device for continuous model testing according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0027] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] As an optional application scenario of this invention, such as Figure 1 As shown, the adaptive supervision signal construction system applied during continuous model testing may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0029] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0030] Existing technologies employ two methods to construct supervision signals: backward alignment and generative diffusion models. Both methods essentially constrain dynamically changing test environments with static anchor points, failing to address the scarcity and unreliability of supervision signals in CTTA scenarios, leading to a continuous decline in model performance as the environment changes. To address this, this invention abandons strict alignment with static anchor points. Instead, it actively adapts the synthesized image to the current target domain through style injection and statistical alignment. By calculating multi-dimensional loss, it precisely provides the reliable supervision signals needed for the model's adaptation process, fundamentally solving the problem of scarce and unreliable supervision signals in adaptive scenarios during continuous testing. This helps support forward facilitation for continuous model adaptation and prevents a continuous decline in model performance due to environmental changes.

[0031] According to an embodiment of the present invention, an embodiment of an adaptive supervision signal construction method for continuous model testing is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides an adaptive supervision signal construction method for continuous model testing. Figure 2 This is a flowchart of an adaptive supervision signal construction method applied to continuous model testing according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Generate multiple synthetic images for each category label, bind each synthetic image to its corresponding category label, and construct a synthetic knowledge base.

[0033] Specifically, leveraging the text-to-image generation capabilities of pre-trained diffusion models (such as Stable Diffusion), a compact set of representative images for each category label is synthesized. The core feature of this image set is that each image contains only a single object corresponding to the category label. Specifically, assuming the category label is "dog," by designing descriptive text prompts, such as "A realistic and clear photo of [dog] against a clean background," M prototype examples, i.e., M synthesized images, can be generated for this category label. Each synthesized image is bound to its category label; this offline construction forms a compact synthetic knowledge base, denoted as: .in, Indicates the index of the composite image; Represents the composition domain; Indicates the first from the composition domain Zhang composite image; Indicates the first from the composition domain Category labels for the composite image; This indicates the number of category labels.

[0034] The synthetic knowledge base possesses two core advantages. First, semantic purity: each synthetic image serves as a prototype sample, eliminating real-world interference factors such as cluttered or occluded backgrounds, thus becoming an ideal semantic anchor point in the CTTA process. Second, computational efficiency: by constructing the knowledge base offline, the huge computational overhead caused by repeatedly calling large diffusion models during online model adaptation is directly eliminated, effectively ensuring deployment feasibility in real-world scenarios. At the same time, this lightweight design also lays the foundation for subsequent dynamic style bridging operations.

[0035] Step S202: Obtain the test image set at the current time step, and construct a joint image set based on the test image set and the synthetic knowledge base. Each synthetic image in the joint image set corresponds one-to-one with a test image.

[0036] Specifically, a set of test images for the current time step in a real test environment is obtained. These images are unlabeled and belong to the original data of the target domain that the model needs to adapt to. A joint image set is constructed based on the test image set and the synthetic knowledge base. The synthetic images in this set correspond one-to-one with the test images, establishing a mapping between synthetic data and real target domain data. This facilitates subsequent style alignment and helps to construct a supervision signal that is tightly coupled with the instantaneous data of the target domain.

[0037] Step S203: For each synthesized image in the joint image set, style bridging is performed on the synthesized image based on the test image corresponding to the synthesized image to obtain a stylized synthesized image.

[0038] Specifically, in the input layer of the target model, for each synthesized image in the joint image set, the visual style of the target domain contained in its corresponding test image is injected into the synthesized image. This ensures that the resulting stylized synthesized image retains both the pure semantic information of the synthesized image itself and the true visual style features of the current target domain, achieving alignment between the synthesized data and the target domain test data at the style level. Here, the target model is the visual perception model.

[0039] Step S204: Extract features from the stylized synthetic image corresponding to the synthetic image, and perform statistical alignment based on the test image to obtain the target shallow feature map of the synthetic image.

[0040] Specifically, in the statistical layer of the target model, features are extracted from the stylized synthetic image of the synthesized image, and statistical alignment is performed using the test image. The statistical features of the target domain are integrated into the extracted feature map, thereby achieving statistical alignment between the synthetic image and the test image at the feature map level, and finally obtaining a shallow feature map of the target that is adapted to the current target domain.

[0041] Step S205: Extract the corresponding deep features based on the target shallow feature map of each synthesized image, and extract the deep features of each test image.

[0042] Specifically, a target model is used to perform a deep feature extraction process on the shallow feature map of each synthesized image that has been adapted to the target domain. Semantic information is mined layer by layer to obtain the corresponding deep features. Simultaneously, the same target model is used to perform a complete feature extraction operation on each test image to obtain the deep features corresponding to the test image. The feature extraction process based on the target model described above is existing technology and will not be elaborated upon here.

[0043] Step S206: Based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, calculate the multi-dimensional loss as a supervision signal for the target model at the current time step.

[0044] Specifically, based on the one-to-one correspondence between the test image and its deep features in the joint image set, the synthetic image and its corresponding stylized synthetic image and deep features, the multi-dimensional total loss is calculated. This multi-dimensional total loss is used as the supervision signal of the target model at the current time step, providing a direct and unbiased optimization direction for the discriminative learning and parameter optimization of the model. This directly solves the problem of scarce and unreliable supervision signals in the CTTA process and effectively supports the stable adaptation of the model in the dynamic target domain.

[0045] This invention generates multiple synthetic images for each category label to construct a stable and semantically pure synthetic knowledge base, which serves as a reliable source of supervision throughout the adaptation process. A set of test images for the current time step is obtained, and a joint image set is constructed based on the test image set and the synthetic knowledge base. For synthetic images in the joint image set, the visual style of the target domain contained in the corresponding test image is injected into the synthetic image. This ensures that the stylized synthetic image retains both the pure semantic information of the synthetic image itself and the true visual style features of the current target domain, achieving alignment between the synthetic data and the target domain test data at the style level. Then, the synthetic image and the test image are statistically aligned at the feature map level to obtain a shallow feature map adapted to the current target domain, reducing model inference errors caused by cross-domain feature drift. Further feature extraction is performed on the shallow feature map and the test image to obtain deep features. Based on the image and deep features of all samples, a multi-dimensional loss is calculated as the supervision signal for the current time step. This ensures that the supervision signal has both semantic reliability and stability for model adaptation, thus constructing an accurate, comprehensive, and unbiased supervision signal for the target model. This fundamentally solves the problem of scarce and unreliable supervision signals in adaptive scenarios during continuous testing, and helps support forward facilitation for continuous model adaptation, preventing the model performance from continuously declining with environmental changes.

[0046] This embodiment provides a method for constructing adaptive supervision signals for continuous model testing. The method specifically includes the following steps: Step S301: Generate multiple composite images for each category label, and bind each composite image to its corresponding category label to construct a composite knowledge base. For details, please refer to [link to details]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0047] Step S302: Obtain the test image set at the current time step, and construct a joint image set based on the test image set and the synthetic knowledge base. Each synthetic image in the joint image set corresponds one-to-one with a test image.

[0048] Specifically, step S302 above constructs a joint image set based on the test image set and the synthetic knowledge base, including: Step S3021: Select a set of synthetic images from the synthetic knowledge base that is the same size as the test image set.

[0049] Specifically, the number of samples in the test image set is used as the selection criterion. If the test image set contains 50 images, then an equal number of 50 synthetic images are randomly selected from the synthetic knowledge base to form a synthetic image set with the same size as the test image set.

[0050] Step S3022: Align the test image set and the composite image set according to the index order, so that each test image in the aligned test image set corresponds one-to-one with the composite image in the composite image set, thus obtaining the joint image set.

[0051] Specifically, the test image set is aligned with a synthetic image set of equal size according to index order, meaning that test images and synthetic images at the same index position in both sets are paired and correspond one-to-one. This structured alignment method integrates them into a joint image set, establishing an association mapping between synthetic and test images, which facilitates subsequent cross-domain style alignment.

[0052] Step S303: For each synthesized image in the joint image set, style bridging is performed on the synthesized image based on the test image corresponding to the synthesized image to obtain a stylized synthesized image.

[0053] Specifically, step S303 includes: Step S3031: For each synthesized image in the joint image set, Fourier transform is used to extract the phase spectrum of the synthesized image.

[0054] Specifically, for each synthesized image in the joint image set, the phase spectrum of the synthesized image is extracted by utilizing the ability of Fourier transform to decouple image style and content, so as to preserve the semantic content of the synthesized image.

[0055] Step S3032: Fourier transform is used to extract the amplitude spectrum of the test image corresponding to the synthesized image.

[0056] Specifically, Fourier transform is used to extract the amplitude spectrum of the test image corresponding to the synthesized image, in order to represent the visual style features of the target domain to which the test image belongs.

[0057] Step S3033: After integrating the phase spectrum and amplitude spectrum, perform an inverse Fourier transform to obtain a stylized composite image.

[0058] Specifically, as shown in equation (1), the phase spectrum and amplitude spectrum are integrated and then subjected to inverse Fourier transform. In essence, the amplitude spectrum of the test image is replaced with the amplitude spectrum of the synthetic image, and the stylized synthetic image is reconstructed to achieve efficient injection of the visual style of the test image in the current target domain into the synthetic image.

[0059] (1) In the formula, Indicates the index of the test image; Indicates a time step; Indicates the first from the composition domain Stylized composite images of Zhang's composite images; express The first time step of data collection Zhang test image.

[0060] Step S304: Extract features from the stylized synthetic image corresponding to the synthetic image, and perform statistical alignment based on the test image to obtain the target shallow feature map of the synthetic image.

[0061] Specifically, step S304 includes: Step S3041: Using the target model, shallow feature extraction is performed on the stylized synthetic image corresponding to the synthetic image to obtain the first shallow feature map, and the first mean and first standard deviation of the first shallow feature map are calculated.

[0062] Specifically, the target model is used to perform shallow feature extraction on the stylized synthetic image of the synthesized image to obtain the first shallow feature map, and the instance-level statistics calculated by channel in the spatial dimension, namely the first mean and the first standard deviation, are extracted from the feature map to represent the shallow feature distribution characteristics of the stylized synthetic image.

[0063] Step S3042: Using the target model, shallow feature extraction is performed on the test image corresponding to the synthesized image to obtain a second shallow feature map, and the second mean and second standard deviation of the second shallow feature map are calculated.

[0064] Specifically, the target model is used to extract shallow features from the test image corresponding to the synthesized image to obtain a second shallow feature map. Then, according to the same calculation rules, the instance-level statistics calculated by channel in the spatial dimension, namely the second mean and the second standard deviation, are extracted from the feature map to represent the shallow feature distribution characteristics of the test image in the target domain.

[0065] Step S3043: Normalize the first shallow feature map based on the first mean and the first standard deviation.

[0066] Specifically, the first shallow feature map is normalized using the first mean and the first standard deviation to achieve the standardization of the feature map distribution.

[0067] Step S3044: Based on the second mean and the second standard deviation, the normalized first shallow feature map is rescaled to obtain the target shallow feature map of the synthesized image.

[0068] Specifically, the normalized first shallow feature map is rescaled using the second mean and second standard deviation of the test image, so that the shallow feature statistics of the stylized synthetic image are consistent with the feature statistics of the current target domain test image, and finally a target shallow feature map adapted to the current target domain is obtained.

[0069] The above steps S3043 and S3044 can be expressed by the following formula (2): (2) In the formula, This represents the shallow feature map of the target. Indicates the second standard deviation; This represents the first mean; Indicates the first standard deviation; This represents the second mean.

[0070] By normalizing the shallow features of the synthesized image to eliminate its inherent distributional differences, and then rescaling based on the statistics of the test image, the statistics of the shallow features of the stylized synthesized image are made to perfectly match the test image in the target domain. This achieves deep fusion of synthesized data and real target domain data at the feature layer, making up for the shortcomings of style injection only at the input layer. When performing subsequent deep feature extraction, it can effectively reduce the model inference error caused by cross-domain feature distribution drift, and help the model achieve stable and efficient adaptive learning in the dynamically changing target domain.

[0071] Step S305: Extract the corresponding deep features based on the shallow feature map of the target in each synthesized image, and extract the deep features of each test image. For details, please refer to [link to details]. Figure 2 Step S205 of the illustrated embodiment will not be described again here.

[0072] Step S306: Generate pseudo-labels for each test image in the joint image set.

[0073] Specifically, the target model is used to perform category inference on each test image in the joint image set and generate a corresponding pseudo-label to represent the category information to which the test image belongs.

[0074] Step S307: For each synthesized image in the joint image set, other synthesized images in the joint image set that have the same category label as the synthesized image, and test images whose pseudo-labels have the same category label as the synthesized image, are taken as positive samples of the synthesized image.

[0075] Specifically, each synthesized image in the joint image set is bound to a category label. Test images with pseudo-labels that match the category label are selected from the set, and other synthesized images with the same category label are also selected. These are then integrated into a positive sample of the synthesized image.

[0076] Step S308: Based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, calculate the multi-dimensional loss as a supervision signal for the target model at the current time step.

[0077] Specifically, step S308 includes: Step S3081: Based on the deep features of all images in the joint image set, take each synthesized image as an anchor point, take the positive samples of the synthesized images as positive examples, and take the other images in the joint image set as negative examples to calculate the first loss.

[0078] Specifically, the supervised comparison loss, also known as the first loss, is calculated using the following formula (3): (3) In the formula, This indicates the first loss; Indicates the number of synthesized images in the combined image set; Index indicating a positive sample; Indicates the first Positive samples of the composite image; Represents an exponential function; Represents the cosine similarity function; Indicates the first Deep features of the composite image; Indicates the first Deep features of a positive sample; Indicates the image index in the joint image set; Indicates the first Deep features of the image.

[0079] By maximizing the deep semantic similarity between the anchor point and positive examples and minimizing the deep semantic similarity between the anchor point and negative examples, the target model is guided to cluster similar samples across domains and separate dissimilar samples in the deep semantic representation layer. This achieves deep semantic alignment between synthetic data and target domain test data, allowing the model to learn domain-invariant core semantic features.

[0080] Step S3082: For each synthesized image in the joint image set, the target model is used to predict the stylized generated image of the synthesized image to obtain the first prediction distribution.

[0081] Specifically, for each synthesized image in the joint image set, the target model performs a category prediction operation on its corresponding stylized synthesized image, outputting the probability distribution of each category to which the image belongs, thus obtaining the first prediction distribution. Optionally, the first prediction distribution can be represented as: ,in, Indicates the first The first predicted distribution of the composite image; Represents the target model.

[0082] Step S3083: For each test image in the joint image set, the target model and the teacher model of the target model are used to predict the test image respectively, so as to obtain the second prediction distribution and the third prediction distribution.

[0083] Specifically, for each test image in the joint image set, the target model performs a category prediction operation on it to obtain a second prediction distribution; simultaneously, a teacher model that is paired with the target model performs the same category prediction operation on the test image to obtain a third prediction distribution. Optionally, the target model is actually a student model, which, along with the teacher model, is initialized and constructed based on the same source model, forming a teacher-student model training architecture. This source model is trained on labeled source domain data.

[0084] Step S3084: Calculate the second loss based on the category label corresponding to each synthesized image in the joint image set and the first prediction distribution.

[0085] Specifically, relying on the agent-based cross-entropy supervision mechanism, the real category label of the synthesized image is used as the supervision benchmark to supervise and constrain the prediction results of the target model on the stylized synthesized image. The second loss is calculated by the following formula (4) to quantify the prediction deviation between the first prediction distribution output by the target model and the real category label.

[0086] (4) In the formula, Indicates the second loss; Indicates the index of the category label; In the The composite image belongs to the first... The value is 1 when there is a category label, and 0 otherwise. Indicates the first The composite image belongs to the first... The predicted probabilities of each category label can be obtained from the first prediction distribution.

[0087] Since the stylized synthetic images are adapted to the visual style of the current target domain, the second loss calculated based on them can provide a direct and unbiased supervisory signal for the transfer of semantic knowledge to the target domain, thereby constructing an accurate semantic supervision system for the adaptive learning of the target model. Furthermore, the second loss comes from the real labels rather than the model's own noisy predictions, thus avoiding error accumulation and catastrophic forgetting. Even if the model's predictions are inaccurate at some time steps, they can still be corrected in the next time step using the real labels of the stylized synthetic images.

[0088] Step S3085: Calculate the third loss based on the second and third prediction distributions corresponding to each test image in the joint image set.

[0089] Specifically, the third loss is calculated using the following formula (5) to quantify the difference in the predicted distribution of the teacher and student models for the same test image: (5) In the formula, Indicates the third loss; This indicates that the teacher model is effective on the test image. Belongs to the The predicted probability of each category label can be obtained from the third prediction distribution; Indicates the target model's response to the test image. Belongs to the The predicted probabilities of each category label can be obtained from the second prediction distribution.

[0090] By aligning the predicted distributions of the target model and the teacher model bidirectionally, the stability of the online adaptation process is ensured, preventing the target model from oscillating or forgetting historical knowledge when rapidly adapting to a new distribution. This enables smooth knowledge transfer and helps the target model achieve smooth knowledge transfer and accumulation under dynamic distributions.

[0091] Step S3086: Calculate the sum of the first loss, the second loss, and the third loss as the multi-dimensional loss, and use the multi-dimensional loss as the supervision signal of the target model at the current time step.

[0092] Specifically, the sum of the three losses mentioned above is used as the supervision signal of the target model at the current time step. This supervision signal is used to guide the parameter adjustment of the target model, thereby achieving multi-dimensional collaborative optimization of deep semantic alignment, category prediction accuracy and model learning stability. This helps the target model achieve efficient and stable online adaptation in the dynamically changing target domain.

[0093] In some alternative implementations, Figure 3 This is a flowchart of another adaptive supervision signal construction method applied to continuous model testing according to an embodiment of the present invention, such as... Figure 3 As shown, based on text prompts and category labels, a pre-trained diffusion model is used to generate multiple synthetic images for each category label. Each synthetic image is then bound to its corresponding category label to construct a synthetic knowledge base. In the input layer of the target model, the phase spectrum of the synthetic image and the amplitude spectrum of the test image are extracted using Fourier transform. These are then combined and subjected to inverse Fourier transform to obtain a stylized synthetic image. In the statistical layer of the target model, shallow feature maps of the stylized synthetic image and the test image are extracted, and their mean and standard deviation are extracted from them respectively. The mean and standard deviation of the test image are then injected into the shallow feature map of the stylized synthetic image to obtain the target shallow feature map. Further feature extraction is performed on the shallow feature maps of the test image and the stylized synthetic image to obtain deep features. Based on the deep features, the first loss, second loss, and third loss are calculated. The first loss represents clustering of similar samples and separation of dissimilar samples; the second and third losses are calculated using predicted distributions. The sum of the three losses is determined as the multi-dimensional loss, which serves as the supervision signal for the target model at the current time step.

[0094] Step S309: Update the target model based on the supervision signal at the current time step.

[0095] Specifically, using the multi-dimensional loss supervision signal obtained at the current time step as the optimization basis, the backpropagation update operation is performed on the network parameters of the target model to achieve adaptive parameter optimization of the target model in the current target domain, thereby improving the robustness and adaptability of the target model in dynamic environments.

[0096] Step S310: Based on the updated target model, update the teacher model using an exponential moving average, complete the current time step, and proceed to the next time step.

[0097] Specifically, the target model is the student model. Based on the updated student model, the network parameters of the teacher model are synchronously updated using an exponential moving average method, ensuring that the teacher model's parameters always maintain smooth iteration. After completing the above parameter update operation, the continuous testing and adaptive process at the current time step ends, and the online adaptive optimization of the model continues at the next time step.

[0098] This invention generates multiple synthetic images for each category label to construct a stable and semantically pure synthetic knowledge base, which serves as a reliable source of supervision throughout the adaptation process. A set of test images for the current time step is obtained, and a joint image set is constructed based on the test image set and the synthetic knowledge base. For synthetic images in the joint image set, the visual style of the target domain contained in the corresponding test image is injected into the synthetic image. This ensures that the stylized synthetic image retains both the pure semantic information of the synthetic image itself and the true visual style features of the current target domain, achieving alignment between the synthetic data and the target domain test data at the style level. Then, the synthetic image and the test image are statistically aligned at the feature map level to obtain a shallow feature map adapted to the current target domain, reducing model inference errors caused by cross-domain feature drift. Further feature extraction is performed on the shallow feature map and the test image to obtain deep features. Based on the image and deep features of all samples, a multi-dimensional loss is calculated as the supervision signal for the current time step. This ensures that the supervision signal has both semantic reliability and stability for model adaptation, thus constructing an accurate, comprehensive, and unbiased supervision signal for the target model. This fundamentally solves the problem of scarce and unreliable supervision signals in adaptive scenarios during continuous testing, and helps support forward facilitation for continuous model adaptation, preventing the model performance from continuously declining with environmental changes.

[0099] This embodiment also provides an adaptive supervision signal construction device for continuous model testing. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] This embodiment provides an adaptive supervision signal construction device for continuous model testing, such as... Figure 4 As shown, it includes: The first construction module 401 is used to generate multiple synthetic images for each category label, bind each synthetic image to its corresponding category label, and build a synthetic knowledge base.

[0101] The second construction module 402 is used to obtain the test image set at the current time step, and to construct a joint image set based on the test image set and the synthesis knowledge base. Each synthesized image in the joint image set corresponds one-to-one with the test image.

[0102] The first bridging module 403 is used to perform style bridging on each synthesized image in the joint image set based on the test image corresponding to the synthesized image, so as to obtain a stylized synthesized image.

[0103] The second bridging module 404 is used to extract features from the stylized synthetic image corresponding to the synthetic image and perform statistical alignment based on the test image to obtain the target shallow feature map of the synthetic image.

[0104] The extraction module 405 is used to extract the corresponding deep features based on the target shallow feature map of each synthesized image, and to extract the deep features of each test image.

[0105] The determination module 406 is used to calculate a multi-dimensional loss based on each test image and its deep features in the joint image set and each synthetic image and its corresponding stylized synthetic image and deep features, as a supervision signal for the target model at the current time step.

[0106] In some alternative implementations, the second building module 402 includes: The filtering unit is used to filter a set of synthetic images from the synthetic knowledge base that is the same size as the test image set.

[0107] Alignment units are used to align the test image set and the composite image set according to the index order, so that each test image in the aligned test image set corresponds one-to-one with the composite image in the composite image set, resulting in a joint image set.

[0108] In some alternative implementations, the first bridging module 403 includes: The first extraction unit is used to extract the phase spectrum of each synthesized image in the joint image set by using Fourier transform.

[0109] The second extraction unit is used to extract the amplitude spectrum of the test image corresponding to the synthesized image using Fourier transform.

[0110] The transform unit is used to integrate the phase spectrum and amplitude spectrum and then perform an inverse Fourier transform to obtain a stylized composite image.

[0111] In some alternative implementations, the second bridging module 404 includes: The third extraction unit is used to perform shallow feature extraction on the stylized synthetic image corresponding to the synthetic image using the target model, to obtain the first shallow feature map, and to calculate the first mean and the first standard deviation of the first shallow feature map.

[0112] The fourth extraction unit is used to perform shallow feature extraction on the test image corresponding to the synthesized image using the target model to obtain a second shallow feature map, and to calculate the second mean and the second standard deviation of the second shallow feature map.

[0113] A normalization unit is used to normalize the first shallow feature map based on a first mean and a first standard deviation.

[0114] The rescaling unit is used to rescale the normalized first shallow feature map based on the second mean and the second standard deviation to obtain the target shallow feature map of the synthesized image.

[0115] In some alternative embodiments, prior to determining module 406, the device further includes: The generation module is used to generate pseudo-labels for each test image in the joint image set.

[0116] The third construction module is used to, for each synthesized image in the joint image set, take other synthesized images in the joint image set that have the same category label as the synthesized image, and test images whose pseudo-labels have the same category label as the synthesized image, as positive samples of the synthesized image.

[0117] In some alternative implementations, the determining module 406 includes: The first computational unit is used to calculate the first loss based on the deep features of all images in the joint image set, using each synthesized image as an anchor point, positive samples of the synthesized images as positive examples, and other images in the joint image set as negative examples.

[0118] The first prediction unit is used to predict the stylized image generated from the synthetic image using the target model for each synthetic image in the joint image set, thereby obtaining the first prediction distribution.

[0119] The second computational unit is used to predict the test image for each test image in the joint image set using the target model and the teacher model of the target model, respectively, to obtain the second prediction distribution and the third prediction distribution.

[0120] The second computational unit is used to calculate the second loss based on the category label corresponding to each synthesized image in the joint image set and the first prediction distribution.

[0121] The third computational unit is used to calculate the third loss based on the second and third prediction distributions corresponding to each test image in the joint image set.

[0122] The unit is determined to calculate the sum of the first loss, the second loss, and the third loss as a multi-dimensional loss, which is then used as a supervision signal for the target model at the current time step.

[0123] In some alternative embodiments, the device further includes: The first update module is used to update the target model based on the supervision signal at the current time step.

[0124] The second update module is used to update the teacher model based on the updated target model using an exponential moving average, complete the current time step, and move on to the next time step.

[0125] The adaptive supervision signal construction device for continuous model testing provided in this embodiment of the invention can execute the adaptive supervision signal construction method for continuous model testing provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0126] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0127] The following is a detailed reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0128] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0129] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the adaptive supervision signal construction method for continuous model testing according to embodiments of the present invention.

[0130] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0131] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the adaptive supervision signal construction method for continuous model testing shown in the above embodiments is implemented.

[0132] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for constructing adaptive supervision signals for continuous model testing, characterized in that, The method includes: Multiple synthetic images are generated for each category label, and each synthetic image is bound to its corresponding category label to build a synthetic knowledge base; Obtain the test image set at the current time step, and construct a joint image set based on the test image set and the synthetic knowledge base, wherein each synthetic image in the joint image set corresponds one-to-one with the test image; For each synthesized image in the joint image set, style bridging is performed on the synthesized image based on the test image corresponding to the synthesized image to obtain a stylized synthesized image; Feature extraction is performed on the stylized composite image corresponding to the composite image, and statistical alignment is performed based on the test image to obtain the target shallow feature map of the composite image; Based on the target shallow feature map of each synthesized image, extract the corresponding deep features, and extract the deep features of each test image. Based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, a multi-dimensional loss is calculated as a supervision signal for the target model at the current time step.

2. The method according to claim 1, characterized in that, The construction of a joint image set based on the test image set and the synthetic knowledge base includes: Select a set of synthetic images of the same size as the test image set from the synthetic knowledge base; The test image set and the composite image set are aligned in index order so that each test image in the aligned test image set corresponds one-to-one with a composite image in the composite image set, thus obtaining the joint image set.

3. The method according to claim 1, characterized in that, For each synthesized image in the joint image set, based on the test image corresponding to the synthesized image, style bridging is performed on the synthesized image to obtain a stylized synthesized image, including: For each synthesized image in the joint image set, the phase spectrum of the synthesized image is extracted using Fourier transform; The amplitude spectrum of the test image corresponding to the synthesized image is extracted using Fourier transform; The phase spectrum and the amplitude spectrum are integrated and then subjected to inverse Fourier transform to obtain the stylized composite image.

4. The method according to claim 1, characterized in that, The step of extracting features from the stylized composite image corresponding to the composite image and performing statistical alignment based on the test image to obtain the target shallow feature map of the composite image includes: Using the target model, shallow feature extraction is performed on the stylized synthetic image corresponding to the synthetic image to obtain a first shallow feature map, and the first mean and first standard deviation of the first shallow feature map are calculated. Using the target model, shallow feature extraction is performed on the test image corresponding to the synthesized image to obtain a second shallow feature map, and the second mean and second standard deviation of the second shallow feature map are calculated. The first shallow feature map is normalized based on the first mean and the first standard deviation; Based on the second mean and the second standard deviation, the normalized first shallow feature map is rescaled to obtain the target shallow feature map of the synthesized image.

5. The method according to claim 1, characterized in that, The method further includes calculating a multi-dimensional loss based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, as a supervision signal for the target model at the current time step. Generate pseudo-labels for each test image in the joint image set; For each synthesized image in the joint image set, other synthesized images in the joint image set that have the same category label as the synthesized image, and test images whose pseudo-labels have the same category label as the synthesized image, are used as positive samples of the synthesized image.

6. The method according to claim 5, characterized in that, The calculation of a multi-dimensional loss based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, serves as a supervision signal for the target model at the current time step, including: Based on the deep features of all images in the joint image set, the first loss is calculated using each synthesized image as an anchor point, the positive samples of the synthesized images as positive examples, and the other images in the joint image set as negative examples. For each synthesized image in the joint image set, the target model is used to predict the stylized generated image of the synthesized image to obtain a first prediction distribution; For each test image in the joint image set, the target model and the teacher model of the target model are used to predict the test image respectively, to obtain a second prediction distribution and a third prediction distribution; Based on the category label and first prediction distribution corresponding to each synthesized image in the joint image set, the second loss is calculated; The third loss is calculated based on the second and third prediction distributions corresponding to each test image in the joint image set. The sum of the first loss, the second loss, and the third loss is calculated as the multi-dimensional loss, which is then used as the supervision signal for the target model at the current time step.

7. The method according to claim 6, characterized in that, The method further includes: The target model is updated based on the supervision signal at the current time step; Based on the updated target model, the teacher model is updated using an exponential moving average to complete the current time step and proceed to the next time step.

8. An adaptive supervision signal construction device for continuous model testing, characterized in that, The device includes: The first construction module is used to generate multiple synthetic images for each category label, bind each synthetic image to its corresponding category label, and build a synthetic knowledge base. The second construction module is used to obtain the test image set at the current time step, and to construct a joint image set based on the test image set and the synthetic knowledge base, wherein each synthetic image in the joint image set corresponds one-to-one with the test image; The first bridging module is used to perform style bridging on each synthesized image in the joint image set based on the test image corresponding to the synthesized image to obtain a stylized synthesized image. The second bridging module is used to extract features from the stylized composite image corresponding to the composite image and perform statistical alignment based on the test image to obtain the target shallow feature map of the composite image. The extraction module is used to extract the corresponding deep features based on the target shallow feature map of each synthesized image, and to extract the deep features of each test image. The determination module is used to calculate a multi-dimensional loss based on each test image and its deep features in the joint image set, and each synthesized image and its corresponding stylized synthesized image and deep features, as a supervision signal for the target model at the current time step.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the adaptive supervision signal construction method for continuous model testing as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the adaptive supervision signal construction method for continuous model testing as described in any one of claims 1 to 7.

Images