Data processing method and apparatus, device and storage medium

Abstract

The embodiments of the present application belong to artificial intelligence technologies and disclose a data processing method and apparatus, a device and a storage medium. The method comprises: adding to a target service image random noise data corresponding to K time steps, so as to obtain K noisy images; outputting N candidate service tags and the K noisy images to a service diffusion model and, according to candidate service tag i, denoising the K noisy images of the target service image by means of the service diffusion model, so as to obtain denoised images corresponding to candidate service tag i respectively under the K time steps; fusing the denoised images corresponding to candidate service tag i respectively under the K time steps, so as to obtain a fused denoised image corresponding to candidate service tag i; and, according to the fused denoised images respectively corresponding to the N candidate service tags, determining a target service tag of the target service image from among the N candidate service tags. The present application can improve the tag identification efficiency of service diffusion models.

Description

Data processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 4, 2023, with application number 2023109769761 and invention name “Data processing method, device, equipment and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to fields such as artificial intelligence technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0003] Artificial intelligence software technologies primarily encompass computer vision, speech recognition, natural language processing, and machine learning / deep learning. Computer vision technologies typically encompass image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, and simultaneous localization and mapping. Image recognition involves identifying image labels based on trained image recognition models. Image labels reflect the categories of objects in the image. In practice, current image recognition models take a long time to identify labels, resulting in relatively low label recognition efficiency.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a data processing method, apparatus, device, and storage medium to reduce the time consumption of identifying tags in a business diffusion model and improve the efficiency of tag identification in the business diffusion model.

[0006] An embodiment of the present application provides a data processing method, including:

[0007] Obtain a target service image and N candidate service tags associated with the target service; N is a positive integer greater than 1;

[0008] Add random noise data corresponding to K time steps to the target business image to obtain K noisy images of the target business image; K is a positive integer greater than 1;

[0009] Output the N candidate service labels and the K noisy images of the target service image to a service diffusion model. De-noising is performed on the K noisy images of the target service image using the service diffusion model according to the candidate service label i, to obtain de-noised images corresponding to the candidate service label i at the K time steps. i is a positive integer less than or equal to N, and N is a positive integer greater than 1.

[0010] Fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i;

[0011] According to the fused denoised images corresponding to the N candidate service labels, the target service label of the target service image is determined from the N candidate service labels.

[0012] An embodiment of the present application provides a data processing device, including:

[0013] An acquisition module, configured to acquire a target service image and N candidate service tags associated with the target service; N is a positive integer greater than 1;

[0014] An adding module is used to add random noise data corresponding to K time steps to the target business image to obtain K noisy images of the target business image; K is a positive integer greater than 1;

[0015] a processing module, configured to output the N candidate service labels and the K noisy images of the target service image to a service diffusion model, and perform denoising on the K noisy images of the target service image according to the candidate service label i using the service diffusion model to obtain denoised images corresponding to the candidate service label i at the K time steps, where i is a positive integer less than or equal to N, and N is a positive integer greater than 1;

[0016] A fusion module is used to fuse the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i;

[0017] The determination module is configured to determine a target service label of the target service image from the N candidate service labels according to the fused denoised images corresponding to the N candidate service labels.

[0018] On one hand, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0019] On one hand, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0020] On one hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0021] In the present application, since the target business label of the target business image has a strong one-to-one correspondence with the space of the target business image (i.e., the image area), that is, one target business label corresponds to an image area in the target business image. Therefore, the computer device can simultaneously input multiple candidate business labels (i.e., N candidate business labels) into the business diffusion model. The business diffusion model determines the fused denoised image corresponding to the corresponding candidate business label based on the denoised images corresponding to each candidate business label at different time steps (i.e., K time steps). According to the fused denoised images corresponding to the N candidate business labels, the target business label of the target business image is determined from the N candidate business labels. It can be seen that there is no need to input multiple candidate business labels one by one, which is conducive to quickly identifying one or more target business labels of the target business image, reducing the time consumption of the business diffusion model in identifying labels, and improving the label recognition efficiency of the business diffusion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] FIG1 is a schematic diagram of a data processing system provided by the present application;

[0024] FIG2 is a schematic diagram of the model structure of a general diffusion model provided by the present application;

[0025] FIG3 is a schematic diagram of an interactive scenario of a data processing method provided by the present application;

[0026] FIG4 is a schematic diagram of an interaction scenario of another data processing method provided by the present application;

[0027] FIG5 is a flow chart of a data processing method provided by the present application;

[0028] FIG6 is a schematic diagram of a scenario for obtaining a denoised image based on candidate service tags provided by the present application;

[0029] FIG7 is a flow chart of another data processing method provided by the present application;

[0030] FIG8 is a schematic diagram of the interaction between an upsampling network of a general diffusion model and an upsampling network of an initial diffusion model provided by the present application;

[0031] FIG9 is a schematic diagram of the interaction between an upsampling block of a general diffusion model and an upsampling block of an initial diffusion model provided by the present application;

[0032] FIG10 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0033] FIG11 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0035] The embodiments of the present application may involve artificial intelligence technology and fields such as autonomous driving and smart transportation. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0037] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0038] For example, in this application, machine learning technology is used to construct a business diffusion model that can be used to identify business labels of images. For example, a computer device can simultaneously input multiple candidate business labels (i.e., N candidate business labels) into the business diffusion model. The business diffusion model determines the fused denoised image corresponding to each candidate business label based on the denoised images corresponding to each candidate business label at different time steps (i.e., K time steps). Based on the fused denoised images corresponding to the N candidate business labels, the target business label of the target business image is determined from the N candidate business labels. It can be seen that there is no need to input multiple candidate business labels one by one, which is conducive to quickly identifying one or more target business labels of the target business image, reducing the time consumed by the business diffusion model in identifying labels and improving the label recognition efficiency of the business diffusion model.

[0039] To facilitate a clearer understanding of this application, we first introduce the data processing system that implements this application. As shown in Figure 1, the data processing system includes a server 10 and a terminal cluster. The terminal cluster can include one or more terminals, and the number of terminals is not limited here. As shown in Figure 1, the terminal cluster can specifically include terminal 1, terminal 2, ..., terminal n; it is understood that terminal 1, terminal 2, terminal 3, ..., terminal n can all be connected to the server 10 through a network connection, so that each terminal can exchange data with the server 10 through the network connection.

[0040] One or more target applications are installed in the terminal. The target application here can refer to an application with image processing (such as downloading images, generating image labels) functions. For example, the target application can include an independent application, a web application, a small program in a host application, etc. The server 10 refers to a device that provides back-end services for the target application in the terminal. In one embodiment, the server can be used to train the initial diffusion model based on the trained general diffusion model to obtain a business diffusion model for identifying the business label of the target business, and send the business diffusion model to the terminal. The business diffusion model can be a functional module in the target application in the terminal. In actual applications, the terminal can call the business diffusion model to identify the business label of the target business image.

[0041] It is understandable that the server can be an independent physical server, or a server cluster or distributed system composed of at least two physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can specifically refer to a vehicle-mounted terminal, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a screen speaker, a smart watch, etc., but is not limited to this. Each terminal and server can be directly or indirectly connected through wired or wireless communication. At the same time, the number of terminals and servers can be one or at least two, and this application does not limit this.

[0042] It is understandable that the target business may refer to a business related to images, such as an image review business, which refers to identifying whether an image contains sensitive images, where sensitive images refer to data that cannot be directly displayed to users; or, the target business may refer to an image classification business, such as classifying images including animals, people, etc.; or, the target business may refer to a security detection business, such as preventing strangers from entering a certain community. Target business images and sample business images refer to images associated with the target business, that is, target business images and sample business images refer to images whose correlation with the target business is greater than a correlation threshold. That is, the target business image refers to the business image to be identified, and the sample business image is a business image used to train the initial diffusion model. For example, the target business may refer to an image review business, and the sample business image may include images with sensitive images and images without sensitive images. The target business image refers to an image to be reviewed, such as an image to be uploaded to a web page. The labeled business labels under the target business may refer to labels that mark whether the sample business images contain content of interest to the target business. For example, when the target business is an image review business, the labeled business labels may include nudity, violence, gore, normal, etc. The labeled business labels: nudity, violence, gore, etc. reflect that the sample business images contain sensitive images, and the labeled business label: normal, which reflects that the sample business images do not contain sensitive images. When the target business is an image classification business, the labeled business labels may include animals, people, buildings, plants, etc. More specifically, the labeled business labels may include cats, dogs, people, local buildings, flowers, trees, etc. When the target business is a security detection business, the labeled business labels may include users within the cell, users outside the cell, etc. Among them, the labeled business labels refer to those obtained by manually labeling the sample business images associated with the target business, the predicted business labels refer to those predicted by the initial diffusion model for the sample business images, and the target business labels refer to those identified by the business diffusion model for the target business images associated with the target business.

[0043] It is understandable that a general image may refer to an image whose correlation with the target business is less than or equal to a correlation threshold, such as the general image may be an image obtained from the Internet; a general tag may refer to an image obtained from a web page, or the general image tag may be a tagged business tag associated with other businesses, and other businesses refer to businesses other than the target business in this application. The general image tag may refer to the descriptive information of the general image; if the general image is an image including a dog, the general image tag of the general image is: high-definition pet dog photography image appreciation; if the general image is an image including a sunset, the general image tag of the general image is: beautiful sunset natural scenery picture. Of course, the language of these general image tags can be Chinese or other languages.

[0044] It should be noted that, by comparing the general image label with the annotated business label, it can be seen that the description form of the annotated label is relatively uniform, while the description form of the general image label is relatively diversified, and there is a certain degree of similarity between the general image label and the business label. For example, the general image is an image including a dog, and the general image label of the general image is: high-definition pet dog photography image appreciation; when the target business is an image classification business, and the sample business image is an image including a dog, the annotated business label of the sample business image is: dog, visible, and both the general image label and the annotated business label include the keyword: dog. Since the general diffusion model in this application is trained based on general images and general image labels, and there is a certain degree of similarity between the general image label and the annotated business label, it can be said that the general diffusion model has a rough ability to identify the business label of the target business image, and the business diffusion model can be obtained by fine-tuning based on the general diffusion model. In this way, while ensuring the accuracy of the business diffusion model, the business diffusion model can be trained with a small number of annotated business labels, thereby improving the training accuracy and training efficiency of the business diffusion model.

[0045] It is understandable that the general diffusion model, initial diffusion model, and business diffusion model in this application all belong to diffusion models (Diffusion Model). The diffusion model is also a generative model. The diffusion model can generate images by introducing noise and then trying to generate noise through denoising. The general diffusion model in this application can refer to the one trained based on a general image and a general image label. The initial diffusion model has the same model structure as the general diffusion model, such as the initial diffusion model and the general diffusion model both include a noise adding network, an upsampling network, and a downsampling network. The noise adding network can refer to a network for adding random noise data to an image, the downsampling network is used to downsample the noisy image obtained by the noise adding network, and the upsampling network is used to upsample the downsampled image obtained by the downsampling network to remove the random noise data in the noisy image. The weight of the noise adding network of the initial diffusion model is the same as the weight of the noise adding network of the general diffusion model, the weight of the downsampling network of the initial diffusion model is the same as the weight of the downsampling network of the general diffusion model, and the weight of the upsampling network of the initial diffusion model is the same as the weight of the upsampling network of the general diffusion model.

[0046] Optionally, the universal diffusion model may further include an image encoder, an image decoder, and a conditional generative network. The image encoder is used to map the universal image from pixel space to latent space to obtain a latent space representation of the universal image, and use the latent space representation of the universal image as the image feature of the universal image. In the application scenario where text data generates image data, the conditional generative network is used to extract features from text data (such as universal image labels, text prompts), and obtain text features corresponding to the text data. During the training phase of the universal diffusion model, the image decoder is used to map the denoised image features output by the upsampling network to the pixel space to obtain a universal image based on the noisy image feature recovery; during the inference process of the universal diffusion model, the image decoder is used to map the denoised image output by the upsampling network to the pixel space to obtain an image associated with the text prompt.

[0047] For example, as shown in Figure 2, the universal diffusion model may include an image encoder a, a denoising network, a conditional generative network, an upsampling network, a downsampling network, and an image decoder b. During the training of the universal diffusion model, the image encoder a is used to map a universal image x from pixel space to a latent space, obtaining image features z for the universal image x. The denoising network is used to add random noise data corresponding to a random step size t to the image features z, obtaining noisy image features corresponding to the universal image. The conditional generative network includes an image encoder c and a text encoder d, which are used to extract text features from the universal image label of the universal image x, obtaining universal text features θ for the universal image label. The downsampling network is used to predict predicted noise data corresponding to a random step size t based on the universal text features θ and the noisy image features. The universal diffusion model is trained based on the random noise data and the predicted noise data corresponding to the random step size t, obtaining a trained universal diffusion model. The Q, K, and V in the upsampling and downsampling networks indicate that both the upsampling and downsampling networks are attention-based networks, and the image encoder a, denoising network, conditional generative network, and relevance recognition network are pre-trained. The conditional generation network may refer to a clip network, which is used to manage information of two different dimensions: computer vision (image data) and human language (i.e., text data). The two use the same mathematical representation. For example, an image of a puppy and text data (a puppy photo) are input into the conditional generation network. The conditional generation network can output an image feature vector and a text feature vector, and the image feature vector matches the text feature vector. In the present application, the conditional generation network can be used to extract business text features of business text labels. In the present application, the general diffusion model including a noise adding network, an upsampling network, and a downsampling network is mainly used as an example for explanation. The upsampling network and the downsampling network can constitute a unet network. The unet network is used to denoise the noise adding network. The unet network can also be called a denoising network.

[0048] It should be noted that during the training of the initial diffusion model, the weights of the noise-adding network and upsampling network remain unchanged. The upsampling network of the initial diffusion model can be fine-tuned based on the sample business images and annotated business labels to obtain the business diffusion model. In other words, the computer device only needs to embed a small number of annotated business labels for the target business into the universal diffusion model to obtain a business diffusion model for identifying the business labels for the target business. This gives the business diffusion model the ability to refine the business labels of the target business images. In other words, the business diffusion model has the ability to refine the labels of the target business. This not only improves the label recognition accuracy of the business diffusion model, but also improves the training efficiency of the business diffusion model, reducing costs, and eliminating the need to retrain the business diffusion model from scratch based entirely on annotated business labels.

[0049] It should be noted that the time step in this application refers to the number of noise steps for adding random operation data to the image. Different time steps correspond to different numbers of noise steps. The time step can be represented by T. A single noise step can be represented by t, which can also be called the random step, that is, t can also be called the t-th noise step. The noise amount of the random noise data corresponding to different random steps is different, and the noise amount of the random noise data increases with the increase of random step t. For example, when T = 5, first, at t = 1, the random noise data corresponding to random step 1 is added to the target image to obtain the noisy image z1; at t = 2, the random noise data corresponding to random step 2 is added to the noisy image z1 to obtain the noisy image z2; and so on, until the noisy image z5 is obtained. That is, the noisy image z5 accumulates the noise data corresponding to random steps 1 to 5, with the random noise data corresponding to random step 1 having the least noise amount and the random noise data corresponding to random step 5 having the most noise amount. The noisy image z5 can be called the noisy image at time step 5.

[0050] The data processing system in FIG1 can be used to implement the data processing method of the present application, as shown in FIG3 and FIG4 , where the server 20a in FIG3 and FIG4 can refer to the server 10 in FIG1 , and the terminal 23a in FIG4 can refer to any terminal in the terminal cluster in FIG1 . FIG3 and FIG4 illustrate the target business by taking the image review business as an example, which detects the legitimacy of the image to be published by the user or the image in the video data to be published, such as detecting whether the image to be published by the user or the image in the video data to be published contains sensitive images.

[0051] As shown in Figure 3, the server 20a can download a universal diffusion model from a web page, or can obtain a universal image and a universal image label of the universal image from a web page, and train the universal diffusion model based on the universal image and the universal image label to obtain a trained universal diffusion model. The universal image here can include business images related to the target business, and can also include images that are not related to the target business. In Figure 3, taking the universal image 21a as an example, the universal image 21a can refer to an image including a person, and the universal image label of the universal image is: serious teacher xx. The universal diffusion model can include a noise addition network, an upsampling network, and a downsampling network. After obtaining the universal diffusion model, the server 20a can construct an initial business diffusion model based on the structure of the universal diffusion model. The structure of the initial business diffusion model is the same as that of the universal diffusion model, and the initial business diffusion model also has a noise addition network, an upsampling network, and a downsampling network. Among them, the initial business diffusion model is also referred to as the initial diffusion model.

[0052] Furthermore, server 20a can initialize the weights of the noise-adding network of the initial service diffusion model based on the weights of the noise-adding network of the universal diffusion model, initialize the weights of the upsampling network of the initial service diffusion model based on the weights of the upsampling network of the universal diffusion model, and initialize the weights of the downsampling network of the initial service diffusion model based on the weights of the downsampling network of the universal diffusion model. Specifically, the weights of the noise-adding network of the universal diffusion model are transferred to the noise-adding network of the initial service diffusion model, the weights of the upsampling network of the universal diffusion model are transferred to the upsampling network of the initial service diffusion model, and the weights of the downsampling network of the universal diffusion model are transferred to the downsampling network of the initial service diffusion model. In other words, the weights of the noise-adding network of the initial diffusion model are the same as the weights of the noise-adding network of the universal diffusion model, the weights of the upsampling network of the initial diffusion model are the same as the weights of the upsampling network of the universal diffusion model, and the weights of the downsampling network of the initial diffusion model are the same as the weights of the downsampling network of the universal diffusion model. Therefore, the initial diffusion model and the universal diffusion model have the same recognition capability. Server 20a can obtain sample business images associated with the image review service, as well as annotated business labels for the sample business images. The number of sample business images can be multiple. As shown in Figure 4, the sample business images include sample business image 22a as an example. Sample business image 22a is an image including a person. The annotated business label of sample business image 22a is normal (reflecting that the sample business image does not contain sensitive images). The annotated business label is obtained by manually annotating sample business image 22a. The annotated business label reflects the true label (actual label) of sample business image 22a under the image review service.

[0053] Next, server 20a can invoke the noise addition network of the general diffusion model to add noise data corresponding to the random step size t to general image 21a, thereby obtaining a noisy image of general image 21a. Furthermore, server 20a can invoke the downsampling network of the general diffusion model to downsample the noisy image of general image 21a, thereby obtaining a downsampled image of general image 21a. Next, server 20a can invoke the upsampling network of the general diffusion model to identify correlation features between general image 21a and the general image label based on the general label and the downsampled image of general image 21a, and determine the correlation features between general image 21a and the general image label as first-class correlation features between general image 21a and the general image label.

[0054] Similarly, as shown in Figure 4, server 20a can call the noise addition network of the initial diffusion model to add random noise data corresponding to the random step size t to the sample business image 22a, thereby obtaining a noisy image of the sample business image 22a. The server 20a can then call the downsampling network of the initial diffusion model to downsample the noisy image of the sample business image 22a, thereby obtaining a downsampled image of the sample business image 22a. Subsequently, the server 20a can call the upsampling network of the initial diffusion model to identify the correlation feature between the sample business image 22a and the labeled business label based on the labeled business label, the first-category correlation feature, and the downsampled image of the sample business image 22a. The correlation feature between the sample business image 22a and the labeled business label is then determined as a second-category correlation feature between the sample business image 22a and the labeled business label. Based on this second-category correlation feature, the server 20a can determine the predicted noise data in the noisy image of the sample business image 22a.

[0055] Server 20a can adjust the weights of the upsampling network of the initial diffusion model based on the predicted noise data and the random noise data corresponding to the random step size t. Server 20a can refer to the above steps and train the initial diffusion model multiple times using other sample service images to obtain a service diffusion model. During the training of the initial diffusion model, the parameters of the general diffusion model are frozen, meaning they remain unchanged. This allows the general diffusion model to be used to train recognition models associated with various services, improving the utility of the general diffusion model. Furthermore, the forward diffusion processes for different services are similar, involving the gradual addition of noise to image features. Therefore, during the training of the initial diffusion model, the weights of the noise addition network of the initial diffusion model are frozen, requiring only the parameters of the upsampling network of the initial diffusion model to be adjusted, thereby improving the training efficiency of the service diffusion model. Furthermore, by labeling service labels, the upsampling network of the initial diffusion model is adjusted, enabling the adjusted service diffusion model to recognize service labels associated with the target service, thereby improving the label recognition accuracy of the service diffusion model.

[0056] After training the service diffusion model, server 20a can send the service diffusion model to terminal 23a. After receiving the target service image to be published, terminal 23a can input the target service image and multiple candidate service tags under the image review service into the service diffusion model. The service diffusion model then identifies the candidate service tag of the target service image from the multiple candidate service tags and uses it as the target service tag of the target service image. Furthermore, based on the target service tag of the target service image, the legitimacy of the target service image is determined. If the target service tag of the target service image is "normal," the target service image is determined to be legitimate. If the target service tag of the target service image includes nudity, gore, violence, etc., the target service image is determined to be illegitimate. If the target service image is illegitimate, the publication of the target service image is suspended. If the target service image is legitimate, the target service image is published and can be published to the network. The service diffusion model ensures that the image review service is processed quickly and accurately, improving the processing efficiency and accuracy of the image review service.

[0057] It is understandable that the training process of the initial diffusion model may also be performed by the terminal 23a. For the specific implementation process, reference may be made to the training process of the initial diffusion model by the server 20a, and the repeated parts will not be repeated.

[0058] Further, please refer to Figure 5, which is a flow chart of a data processing method provided by an embodiment of the present application. As shown in Figure 5, the method can be executed by any terminal in the terminal cluster in Figure 1, or by the server in Figure 1, or by the terminals and servers in the terminal cluster in Figure 1 in collaboration. The devices used to execute the data processing method in this application can be collectively referred to as computer devices. The method may include the following steps:

[0059] S101. Obtain a target service image and N candidate service tags associated with a target service; N is a positive integer greater than 1.

[0060] In the present application, the computer device can obtain the target business image associated with the target business, and N candidate business tags associated with the target business; the target business image may refer to the image to be identified under the target business, and the candidate business tag may refer to the annotated business tag under the target business. For example, if the target business is an image review business, the target business image may refer to the image to be reviewed (such as an image to be posted to the Internet); the candidate business tags may include nudity, violence, blood, normal, etc. If the target business is an image classification business, the target business image may refer to the image to be classified, and the candidate business tags may include animals, people, buildings, plants, etc. More specifically, the candidate business tags include cats, dogs, people, buildings in a certain place, flowers, trees, etc.; when the target business is a security detection business, the candidate business tags may include users in the cell, users outside the cell, etc.

[0061] S102 : Add random noise data corresponding to K time steps to the target service image to obtain K noisy images of the target service image; K is a positive integer greater than 1.

[0062] In this application, the computer device can add random noise data corresponding to K time steps to the target business image to obtain K noisy images of the target business image, where K is a positive integer greater than 1. The K time steps are all different. For example, the K time steps can be: T1, T2, ..., Tc, c = K. Assuming T1 = 10, the computer device can accumulate the random noise data corresponding to random steps 1 to 10 to the target business image to obtain the noisy image z at time step T1. T1 Assuming that T2 = 20, the computer device can add random noise data corresponding to random steps 1 to 20 to the target business image to obtain the noisy image z at time step T2. T2 By analogy, the above steps are repeated until the noisy images corresponding to K time steps are obtained, and the noisy images corresponding to the K time steps are determined as the K noisy images of the target business image.

[0063] S103. Output the N candidate service labels and the K noisy images of the target service image to a service diffusion model. The service diffusion model performs denoising on the K noisy images of the target service image according to the candidate service label i to obtain denoised images corresponding to the candidate service label i at the K time steps. i is a positive integer less than or equal to N, and N is a positive integer greater than 1.

[0064] In this application, the computer device can simultaneously output the N candidate business labels and the K noisy images of the target business image to the business diffusion model. For the i-th denoising iteration, the computer device can call the business diffusion model to denoise the noisy image z at time step T1 according to the candidate business label i. T1 Perform T1 denoising processes to obtain the denoised image of the candidate service label i at time step T1, call the service diffusion model to perform the denoising process on the noisy image z at time step T2 according to the candidate service label i. T2 Perform T2 denoising processes to obtain the denoised image of the candidate service label i at time step T2. Repeat the above steps until the denoised images corresponding to the candidate service label i at K time steps are obtained. Further, for the i+1 denoising iteration, the computer device can call the service diffusion model to perform denoising on the noisy image z at time step T1 according to the candidate service label i+1. T1 Perform T1 denoising processes to obtain the denoised image of the candidate service label i+1 at time step T1, call the service diffusion model to perform the denoising process on the noisy image z at time step T2 according to the candidate service label i+1. T2 Perform T2 denoising processes to obtain the denoised image of candidate service label i+1 at time step T2. Repeat the above steps until the denoised images corresponding to candidate service label i+1 at K time steps are obtained. This process continues until the number of denoising iterations reaches N, at which point the denoised images corresponding to N candidate service labels at K time steps are obtained.

[0065] S104 , fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i.

[0066] In the present application, the computer device can fuse the denoised images corresponding to the candidate business label i at the K time steps to obtain the fused denoised image corresponding to the candidate business label i; and so on, repeat the above steps until the fused denoised images corresponding to N candidate business labels are obtained.

[0067] In one embodiment, the above-mentioned step S104 includes: the computer device may accumulate the denoised images corresponding to the candidate business label i at the K time steps to obtain an accumulated denoised image; and determine the accumulated denoised image as the fused denoised image corresponding to the candidate business label i.

[0068] In one embodiment, step S104 includes: the computer device may use an averaging algorithm to average the denoised images corresponding to the candidate service label i at the K time steps to obtain an average denoised image corresponding to the candidate service label i, and determine the average denoised image corresponding to the candidate service label i as the fused denoised image corresponding to the candidate service label i. The averaging algorithm may be any one of an arithmetic mean algorithm, a geometric mean algorithm, a square mean algorithm, and a harmonic mean algorithm.

[0069] It should be noted that since the target service label of the target service image has a strong one-to-one correspondence with the target service image space (i.e., image region), that is, one target service label corresponds to one image region in the target service image. Therefore, if the target service image includes content corresponding to candidate service label i, then candidate service label i can better guide the service diffusion model to perform denoising on the noisy images corresponding to each of the K time steps. In other words, the image quality of the denoised images corresponding to candidate service label i in each of the K time steps is relatively high. In other words, the image quality of the image region corresponding to candidate service label i in the denoised images corresponding to candidate service label i in each of the K time steps is relatively high. Furthermore, the image quality of the fused denoised image corresponding to candidate service label i is relatively high. If the target service image does not include content corresponding to candidate service label i, then candidate service label i cannot better guide the service diffusion model to perform denoising on the noisy images corresponding to each of the K time steps. In other words, the image quality of the denoised images corresponding to candidate service label i in each of the K time steps is relatively low. Furthermore, the image quality of the fused denoised image corresponding to candidate service label i is relatively low. Here, the image quality of the fused denoised image can be determined according to the pixel values ​​in the fused denoised image. For example, if the pixel values ​​in the fused denoised image are relatively large, the amount of information contained in the fused denoised image is relatively large, that is, the clarity of the fused denoised image is relatively high, and the image quality of the fused denoised image is relatively high. On the contrary, if the pixel values ​​in the fused denoised image are relatively low, the amount of information contained in the fused denoised image is relatively small, that is, the clarity of the fused denoised image is relatively low, and the image quality of the fused denoised image is relatively low.

[0070] For example, as shown in Figure 6, assume that the target service image includes a dinosaur and a tree, and one of the candidate service labels for the target service image is "dinosaur." After inputting these N candidate service labels and the target service image into the service diffusion model, the service diffusion model outputs a denoised image of the target service image at time step T1, as shown in Figure 6. As can be seen from Figure 6, the image quality of the target image region in the denoised image is higher than that of other regions in the denoised image. The target image region is the region indicated by the candidate service label (i.e., the region where the dinosaur is located). In other words, the candidate service label "dinosaur" can guide the diffusion service model to better denoise the target image region in the noisy image of the target service image.

[0071] S105 : Determine a target service label for the target service image from the N candidate service labels according to the fused denoised images corresponding to the N candidate service labels.

[0072] In the present application, the computer device can determine the target business label of the target business image from the N candidate business labels based on the fused denoised images corresponding to the N candidate business labels. The number of target business labels of the target image can be one or more.

[0073] Optionally, step S105 includes: the computer device may determine the initial service feature values ​​corresponding to the N candidate service tags based on the fused denoised images corresponding to the N candidate service tags. For example, the computer device may accumulate the pixel values ​​in the fused denoised image corresponding to candidate service tag i to obtain the total pixel value corresponding to candidate service tag i, and determine the total pixel value corresponding to candidate service tag i as the initial service feature value corresponding to candidate service tag i. The above steps are repeated in this manner until the initial service feature values ​​corresponding to the N candidate service tags are obtained. If the target service image includes content corresponding to candidate service tag i, then the image quality of the fused denoised image corresponding to candidate service tag i is relatively high. Consequently, the pixel values ​​in the fused denoised image corresponding to candidate service tag i are relatively large, and the initial service feature value corresponding to candidate service tag i is also relatively large. Conversely, if the target service image does not include content corresponding to candidate service tag i, then the image quality of the fused denoised image corresponding to candidate service tag i is relatively low. Consequently, the pixel values ​​in the fused denoised image corresponding to candidate service tag i are relatively small, and the initial service feature value corresponding to candidate service tag i is also relatively small. Therefore, the computer device can average the service feature values ​​corresponding to the N candidate service tags to obtain an average service feature value, and determine the candidate service tag among the N candidate service tags whose initial service feature value is greater than the average service feature value as the target service tag of the target service image. By identifying one or more target service tags in the target service image at one time, the efficiency of identifying the target service tags can be improved.

[0074] In the present application, since the target business label of the target business image has a strong one-to-one correspondence with the space of the target business image (i.e., the image area), that is, one target business label corresponds to an image area in the target business image. Therefore, the computer device can simultaneously input multiple candidate business labels (i.e., N candidate business labels) into the business diffusion model. The business diffusion model determines the fused denoised image corresponding to the corresponding candidate business label based on the denoised images corresponding to each candidate business label at different time steps (i.e., K time steps). According to the fused denoised images corresponding to the N candidate business labels, the target business label of the target business image is determined from the N candidate business labels. It can be seen that there is no need to input multiple candidate business labels one by one, which is conducive to quickly identifying one or more target business labels of the target business image, reducing the time consumption of the business diffusion model in identifying labels, and improving the label recognition efficiency of the business diffusion model.

[0075] Further, please refer to Figure 7, which is a flow chart of a data processing method provided by an embodiment of the present application. As shown in Figure 7, the method can be executed by any terminal in the terminal cluster in Figure 1, or by the server in Figure 1, or by the terminals and servers in the terminal cluster in Figure 1 in collaboration. The devices used to execute the data processing method in this application can be collectively referred to as computer devices. The method may include the following steps:

[0076] S201. Add random noise data corresponding to a random step size t to a sample business image associated with the target business to obtain a noisy image of the sample business image, and add random noise data corresponding to the random step size t to a general image to obtain a noisy image of the general image; t is a positive integer.

[0077] In this application, the random step length t refers to a single noise addition. Different values ​​of t result in different amounts of noise in the random noise data corresponding to the random step length t. As the value of the random step length t increases, the amount of noise in the corresponding random noise data also increases. The value of the random step length t is randomly selected from [1, T], where T is the time step and T can be an integer greater than 1. The computer device can add the random noise data corresponding to the random step length t to a sample business image associated with the target business to obtain a noisy image of the sample business image, and add the random noise data corresponding to the random step length t to a general image to obtain a noisy image of the general image.

[0078] S202 : Identify a first type of correlation feature between the general image and a general image label according to a noisy image of the general image using a general diffusion model.

[0079] In the present application, a computer device can call the universal diffusion model and identify the first type of correlation feature between the universal image and the universal image label based on the noisy image of the universal image. The first type of correlation feature is used to reflect the correlation between the universal image and the universal image label, that is, the first type of correlation feature is used to reflect the degree of association between the universal image and the universal image label.

[0080] Optionally, the upsampling network of the universal diffusion model includes L upsampling blocks, and the downsampling network of the universal diffusion model includes L downsampling blocks, where L is a positive integer greater than 1; the above step S202 includes: for the j+1th upsampling iteration, obtaining the upsampled image X of the universal image output by the upsampling block j of the universal diffusion model j ; The upsampled image X of the general image jis obtained based on the noisy image of the general image; j is a positive integer less than L; obtain the downsampled image of the general image output by the downsampling block Lj of the general diffusion model at the Ljth downsampling iteration; through the upsampling block j+1 of the general diffusion model, according to the general image label, the upsampled image X j And the downsampled image of the universal image at the Lj-th downsampling iteration, identifying the correlation features between the universal image at the j+1-th upsampling iteration and the universal image label; the correlation features between the universal image and the universal image label at the L upsampling iterations are determined as the first type of correlation features between the universal image and the universal image label.

[0081] For example, for the first downsampling iteration, the computer device may call downsampling block 1 of the universal diffusion model to perform downsampling processing on the noisy image of the universal image based on the universal image label, thereby obtaining a downsampled image of the universal image at the first downsampling iteration. For the second downsampling iteration, the computer device may call downsampling block 2 of the universal diffusion model to perform downsampling processing on the downsampled image of the universal image at the first downsampling iteration based on the universal image label, thereby obtaining a downsampled image of the universal image at the second downsampling iteration. The above steps are repeated in this manner until the downsampled images corresponding to the universal image at each of the L downsampling iterations are obtained.

[0082] Furthermore, when j=0, for the first upsampling iteration, the computer device may call upsampling block 1 of the universal diffusion model to identify, based on the universal image label and the downsampled image of the universal image at the Lth downsampling iteration, a correlation feature a1 between the universal image and the universal image label at the first upsampling iteration; and, based on the correlation feature a1, perform upsampling processing on the downsampled image of the universal image at the Lth downsampling iteration to obtain an upsampled image X1 of the universal image output by upsampling block 1 of the universal diffusion model. When j=1, for the second upsampling iteration, the computer device may call upsampling block 2 of the universal diffusion model to identify, based on the universal image label, the upsampled image X1, and the downsampled image of the universal image at the L-1th downsampling iteration, a correlation feature a2 between the universal image and the universal image label at the second upsampling iteration; and, based on the correlation feature a2, perform upsampling processing on the downsampled image of the universal image at the L-1th downsampling iteration to obtain an upsampled image X2 of the universal image output by upsampling block 2 of the universal diffusion model. And so on, repeat the above steps until the correlation features between the general image and the general image label under L upsampling iterations are obtained, and the correlation features between the general image and the general image label under L upsampling iterations are determined as the first type of correlation features between the general image and the above general image label.

[0083] Optionally, the upsampling block j+1 of the general diffusion model is configured according to the general image label and the upsampling image X. j and the downsampled image of the general image at the Lj-th downsampling iteration, identifying the correlation feature between the general image at the j+1-th upsampling iteration and the general image label, including: the computer device can use the upsampling block j+1 of the general diffusion model to perform the upsampled image X j The general image is fused with the downsampled image of the general image at the Lj-th downsampling iteration to obtain a general fused image. Furthermore, the text weight and image weight corresponding to upsampling block j+1 of the general diffusion model are obtained. The text weight and image weight may refer to the weight corresponding to upsampling block j+1. The text weight is then used to weight the text features corresponding to the general image label to obtain a weighted text vector. The image weight is used to weight the image features of the general fused image to obtain a weighted image vector. Based on the weighted text vector and the weighted image vector, the correlation feature between the general image and the general image label at the j+1-th upsampling iteration is determined.

[0084] Optionally, the above-mentioned determination of the correlation feature between the general image and the general image label at the j+1th upsampling iteration based on the weighted text vector and the weighted image vector includes: the computer device can perform multiplication processing on the weighted text vector and the weighted image vector to obtain the initial correlation feature between the general image and the general image label at the j+1th upsampling iteration, and then normalize the initial correlation feature between the general image and the general image label at the j+1th upsampling iteration to obtain the normalized correlation feature; that is, the initial correlation feature between the general image and the general image label at the j+1th upsampling iteration is adjusted to a range less than or equal to 1 to obtain the normalized correlation feature. Further, the correlation feature between the general image and the general image label at the j+1th upsampling iteration can be determined based on the normalized correlation feature and the position of the upsampling block j+1 in the general diffusion model.

[0085] For example, the computer device may use the following formula (1) to calculate the correlation feature between the general image and the general image label at the j+1th upsampling iteration:

[0086] In formula (1), represents the correlation feature between the general image and the general image label at the j+1th upsampling iteration, Normalized correlation features, It indicates the position of upsampling block j+1 in the universal diffusion model, for example, it can reflect that upsampling block j+1 is the j+1th upsampling block in the universal diffusion model. represents the weighted image vector, Represents a weighted text vector The transpose of .

[0087] S203 , using an initial diffusion model, based on the first type of correlation features and the annotated business label of the sample business image, identifying predicted noise data in the noisy image of the sample business image; the structure of the initial diffusion model is the same as that of the general diffusion model.

[0088] In the present application, the computer device can use the initial diffusion model to identify the predicted noise data in the noisy image of the sample business image based on the first type of correlation features and the annotated business label of the sample business image, that is, the initial diffusion model can identify the second type of correlation features between the sample business image and the annotated business label based on the first type of correlation features and the annotated business label, and identify the predicted noise data in the noisy image of the sample business image based on the second type of correlation features. This is conducive to enabling the initial diffusion model to have the ability to identify the correlation between the sample business image and the annotated business label, and further, to enable the initial diffusion model to have the ability to identify the business label of the target business image, and to enable the initial diffusion model to have the ability to process a certain business (target business), thereby realizing refined business processing and improving business processing accuracy.

[0089] For example, as shown in Figure 8 , the general diffusion model includes a downsampling network 81a, an upsampling network 82a, and a noise addition network (not shown in Figure 8 ); the service diffusion model includes a downsampling network 84a, an upsampling network 85a, and a noise addition network (not shown in Figure 8 ). The general image in Figure 8 includes a knife and fork, and the general image label for this general image is: "Image including tableware." The sample service image in Figure 8 includes a knife and fork and fruit, and the annotated service label for this sample service image is: "Knife and fork." First, the noise addition network of the general diffusion model can add random noise data corresponding to a random duration t to the general image to obtain a noisy image 80a of the general image. The downsampling network 81a can downsample the noisy image 80a of the general image based on the general image label to obtain a downsampled image of the general image. The upsampling network 82a can identify the first-class correlation feature between the general image and the general image label based on the downsampled image and the general image label.

[0090] Furthermore, as shown in Figure 8, the noise addition network of the initial diffusion model can add random noise data corresponding to a random duration t to the sample service image, obtaining a noisy image 83a of the sample service image. The downsampling network 84a can downsample the noisy image 83a of the sample service image based on the annotated service label, obtaining a downsampled image of the sample service image. The upsampling network 85a can identify a second-type correlation feature between the sample service image and the sample service label based on the downsampled image of the sample service image, the annotated service label, and the first-type correlation feature, and identify predicted noise data in the noisy image 83a of the sample service image based on the second-type correlation feature.

[0091] Optionally, the above-mentioned first-category correlation features include correlation features between the above-mentioned general image and the above-mentioned general image label under the above-mentioned j+1-th upsampling iteration, the upsampling network of the initial diffusion model includes L upsampling blocks, and the downsampling network of the initial diffusion model includes L downsampling blocks, L is a positive integer greater than 1, and j is a positive integer less than L.

[0092] Optionally, the above step S203 includes: for the j+1th upsampling iteration, obtaining the upsampled image Y of the sample service image output by the upsampling block j of the initial diffusion model j ; The upsampled image Y of the sample business image j It is obtained based on the noised image of the sample business image. Get the downsampled image of the business image output by the downsampling block Lj of the initial diffusion model at the Ljth downsampling iteration; through the upsampling block j+1 of the initial diffusion model, according to the marked business label, the upsampled image Y j and the correlation feature j+1, and the downsampled image of the sample business image at the Lj-th downsampling iteration, to identify the second type of correlation feature; the correlation feature j+j reflects the correlation feature between the general image and the general image label at the j+1-th upsampling iteration, and the second type of correlation feature reflects the correlation feature between the sample business image and the labeled business label at the j+1-th upsampling iteration; based on the second type of correlation feature obtained at the L-th upsampling iteration, determine the predicted noise data in the noisy image of the sample business image.

[0093] For example, for the first downsampling iteration, the computer device may call downsampling block 1 of the initial diffusion model to perform downsampling processing on the noisy image of the sample service image based on the annotated service label, thereby obtaining a downsampled image of the sample service image at the first downsampling iteration. For the second downsampling iteration, the computer device may call downsampling block 2 of the initial diffusion model to perform downsampling processing on the downsampled image of the sample service image at the first downsampling iteration based on the annotated service label, thereby obtaining a downsampled image of the sample service image at the second downsampling iteration. Similarly, the above steps are repeated until the downsampled images corresponding to the sample service image at each of the L downsampling iterations are obtained.

[0094] Further, j=0, for the first upsampling iteration, the computer device can call the upsampling block 1 of the initial diffusion model, and identify the second type of correlation feature b1 between the sample business image and the labeled business label at the first upsampling iteration based on the labeled business label, the above-mentioned correlation feature a1, and the downsampled image of the sample business image at the Lth downsampling iteration. Based on the second type of correlation feature b1, the downsampled image of the sample business image at the Lth downsampling iteration is upsampled to obtain the upsampled image Y1 of the sample business image output by the upsampling block 1 of the initial diffusion model. When j=1, for the second upsampling iteration, the computer device may call the upsampling block 2 of the initial diffusion model to identify the second type of correlation feature b2 between the sample business image and the labeled business label at the second upsampling iteration based on the labeled business label, the above-mentioned correlation feature a2, the upsampled image Y1, and the downsampled image of the sample business image at the L-1 downsampling iteration; based on the second type of correlation feature b2, the downsampled image of the sample business image at the L-1 downsampling iteration is upsampled to obtain the upsampled image Y2 of the sample business image output by the upsampling block 2 of the initial diffusion model. The above steps are repeated in this manner until the second type of correlation feature is obtained at the L-th upsampling iteration. Based on the second type of correlation feature obtained at the L-th upsampling iteration, the downsampled image of the sample business image at the 1st downsampling iteration is upsampled to obtain the upsampled image Y of the sample business image output by the upsampling block L of the initial diffusion model. L . Based on the upsampled image Y of the sample business image L , determine the predicted noise data in the noisy image of the sample business image.

[0095] Optionally, the upsampling block j+1 of the initial diffusion model is based on the above-mentioned business label and the upsampling image Y jand correlation feature j+1, and the downsampled image of the sample business image at the Ljth downsampling iteration, identifying the second type of correlation feature, including: the computer device can use the upsampling block j+1 of the initial diffusion model to perform the upsampling on the upsampled image Y j The downsampled image of the sample business image at the Ljth downsampling iteration is fused to obtain a business fusion image; further, the computing device can fuse the correlation feature j+1 with the image feature of the business fusion image to obtain an image fusion feature; the fusion here can refer to accumulation processing or averaging processing. Further, the computer device can identify the labeled business label and the image fusion feature through the upsampling block j+1 of the initial diffusion model to obtain a second type of correlation feature. By embedding (fusing) the first type of correlation feature of the general diffusion model in the process of training the initial diffusion model, it is beneficial to use less training data to train a business diffusion model for processing the target business, improve the label recognition accuracy of the business diffusion model, and improve the training efficiency of the business diffusion model.

[0096] Optionally, as shown in FIG9 , an upsampling block of the diffusion model and the initial diffusion model includes a residual network layer, an attention layer, and a cross-attention layer. The residual network layer is used to fuse the downsampled image output by the corresponding downsampled block with the upsampled image of the previous upsampled block. The self-attention layer is used to identify the correlation features between the image and the text label based on the fused image of the residual network layer. The cross-attention layer is used to upsample the downsampled image output by the corresponding downsampled block according to the correlation features output by the attention layer to obtain an upsampled image. For example, the correlation features obtained by the upsampling block of the general diffusion model will be fused into the corresponding upsampling block in the initial diffusion model. Take the upsampling block j+1 in the general diffusion model and the upsampling block j+1 in the initial diffusion model as examples for explanation. The residual network layer of the upsampling block j+1 in the general diffusion model is used to upsample the upsampled image X. jThe general image is fused with the downsampled image of the general image at the Lj-th downsampling iteration to obtain a general fused image; the self-attention layer of the upsampling block j+1 in the general diffusion model obtains the text weight and image weight corresponding to the upsampling block j+1 of the general diffusion model. The text weight and image weight can refer to the weight corresponding to the upsampling block j+1. Then, the text weight is used to weight the text features corresponding to the general image label to obtain a weighted text vector, and the image weight is used to weight the image features of the general fused image to obtain a weighted image vector. Based on the weighted text vector and the weighted image vector, the correlation feature between the general image and the general image label at the j+1-th upsampling iteration (labeled as correlation feature j+1) is determined. The cross-attention layer of the upsampling block j+1 in the general diffusion model upsamples the downsampled image of the general image at the Lj-th downsampling iteration based on the correlation feature j+1 to obtain the upsampled image X of the general image output by the upsampling block j+1 of the general diffusion model. j+1 .

[0097] Accordingly, the upsampled image Y is processed by the residual network layer of the upsampled block j+1 of the initial diffusion model. j The downsampled image of the sample business image at the Ljth downsampling iteration is fused to obtain a business fusion image; the correlation feature j+1 and the image feature of the business fusion image are fused to obtain an image fusion feature. The self-attention layer of the upsampling block j+1 of the initial diffusion model is used to identify the labeled business label and the image fusion feature to obtain a second type of correlation feature. The implementation process of the self-attention layer of the upsampling block j+1 of the initial diffusion model to obtain the second type of correlation feature can refer to the implementation process of the self-attention layer of the upsampling block j+1 in the general diffusion model to obtain the correlation feature j+1. The repeated parts will not be repeated. The downsampled image of the sample business image at the Ljth downsampling iteration is upsampled according to the second type of correlation feature through the cross-attention layer of the upsampling block j+1 of the initial diffusion model to obtain the upsampled image Y of the sample business image output by the upsampling block j+1 of the initial diffusion model. j+1 By analogy, the above steps are repeated until the upsampled image Y of the sample business image output by the upsampled block L of the initial diffusion model is obtained. L , upsampled image Y L It can be called the denoised image of the sample business image, and the predicted noise data in the noisy image of the sample business image is obtained by subtracting the noisy image of the sample business image from the denoised image of the sample business image.

[0098] S204 : Training the initial diffusion model according to the predicted noise data and the random noise data corresponding to the random step size t to obtain the service diffusion model.

[0099] Optionally, step S204 includes: the computer device may calculate a noise prediction error of the initial diffusion model based on the predicted noise data and the random noise data corresponding to the random step size t. The noise prediction error is used to reflect the denoising capability of the initial diffusion model, that is, the noise prediction error is used to reflect the initial diffusion model's ability to identify the correlation characteristics between the labeled business labels and the sample business images, that is, the noise prediction error is used to reflect the accuracy of the initial diffusion model in identifying the business labels for the target business. For example, a lower noise prediction error reflects a stronger ability of the initial diffusion model to identify the correlation characteristics between the labeled business labels and the sample business images, that is, the higher the accuracy of the initial diffusion model in identifying the business labels for the target business. Conversely, a higher noise prediction error reflects a weaker ability of the initial diffusion model to identify the correlation characteristics between the labeled business labels and the sample business images, that is, the lower the accuracy of the initial diffusion model in identifying the business labels for the target business. Therefore, the computer device can determine the convergence state of the initial diffusion model based on the noise prediction error. The convergence state can include an unconverged state or a converged state. The unconverged state indicates that the noise prediction error of the initial diffusion model is greater than an error threshold, while the converged state indicates that the noise prediction error of the initial diffusion model is less than or equal to the error threshold. The error threshold may be the minimum noise prediction error of the initial diffusion model, which is determined based on the loss function of the initial diffusion model. If the initial diffusion model is in a converged state, the initial diffusion model can be directly determined as the service diffusion model. If the initial diffusion model is in an unconverged state, the upsampling network of the initial diffusion model is trained based on the noise prediction error to obtain the aforementioned service diffusion model. In other words, during the training of the initial diffusion model, by embedding the second-category correlation features identified by the universal diffusion model, only the upsampling network of the initial diffusion model needs to be trained. That is, only fine-tuning of the upsampling network of the initial diffusion model is required to obtain the service diffusion model, thereby improving the training efficiency of the service diffusion model.

[0100] Optionally, if the initial diffusion model is in an unconverged state, then the upsampling network of the initial diffusion model is trained based on the noise prediction error to obtain the service diffusion model, including: if the initial diffusion model is in an unconverged state, the computer device may adjust the weights of the upsampling network of the initial diffusion model based on the noise prediction error to obtain an adjusted initial diffusion model; further, using the adjusted initial diffusion model, based on the first-category correlation feature and the annotated service label of the sample service image, noise data in the noisy image feature of the sample service image is identified, and the identified noise data is determined as updated noise data. Then, the upsampling network of the adjusted initial diffusion model is trained based on the updated noise data and the random noise data corresponding to the random step size t; until the adjusted initial diffusion model is in a converged state, the converged adjusted initial diffusion model is determined as the service diffusion model.

[0101] S205 : Acquire a target service image and N candidate service tags associated with the target service; N is a positive integer greater than 1.

[0102] S206 , adding random noise data corresponding to K time steps to the target service image to obtain K noisy images of the target service image; K is a positive integer greater than 1.

[0103] S207. Output the N candidate business labels and the K noisy images of the target business image to a business diffusion model. The business diffusion model performs denoising on the K noisy images of the target business image according to the candidate business label i to obtain denoised images corresponding to the candidate business label i at the K time steps. i is a positive integer less than or equal to N, and N is a positive integer greater than 1.

[0104] S208 : Fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i.

[0105] S209 : Determine a target service label for the target service image from the N candidate service labels according to the fused denoised images corresponding to the N candidate service labels.

[0106] It should be noted that, for the explanation of steps S205 to S209 , reference may be made to the above embodiments, and the repeated parts will not be described again.

[0107] In the present application, the computer device can obtain a pre-trained universal diffusion model, which is trained based on universal images and universal image labels, and has the ability to identify universal image labels of universal images. The universal image label has a certain similarity with the actual label under the target business (i.e., the labeled business label). In other words, the universal diffusion model has a rough ability to identify the labeled business label under the target business; therefore, the computer device only needs to embed a small amount of labeled business labels under the target business into the universal diffusion model to obtain a business diffusion model for identifying the business label under the target business, so that the business diffusion model has the refined ability to identify the labeled business label, that is, the business diffusion model has the refined ability to identify the label of the target business, which not only improves the label recognition accuracy of the business diffusion model, but also improves the training efficiency of the business diffusion model, reduces costs, and does not need to rely entirely on the labeled business label to retrain the label recognition model from scratch.

[0108] Please refer to Figure 10, which is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. As shown in Figure 10, the data processing device may include:

[0109] The acquisition module 1011 is used to acquire a target service image and N candidate service tags associated with the target service; N is a positive integer greater than 1;

[0110] An adding module 1012 is configured to add random noise data corresponding to K time steps to the target service image to obtain K noisy images of the target service image; K is a positive integer greater than 1;

[0111] The processing module 1013 is configured to output the N candidate service labels and the K noisy images of the target service image to a service diffusion model, and perform denoising on the K noisy images of the target service image based on the candidate service label i using the service diffusion model to obtain denoised images corresponding to the candidate service label i at the K time steps, where i is a positive integer less than or equal to N, and N is a positive integer greater than 1.

[0112] A fusion module 1014 is configured to fuse the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i;

[0113] The determination module 1015 is configured to determine a target service label of the target service image from the N candidate service labels according to the fused denoised images corresponding to the N candidate service labels.

[0114] Optionally, the fusion module 1014 may include a first averaging processing unit 101a and a first determining unit 102a;

[0115] The first averaging processing unit 101a is configured to perform averaging processing on the denoised images corresponding to the candidate service label i at the K time steps to obtain an average denoised image corresponding to the candidate service label i;

[0116] The first determining unit 102a is configured to determine the average denoised image corresponding to the candidate service label i as the fused denoised image corresponding to the candidate service label i.

[0117] Optionally, the determination module 1015 includes a second determination unit 103a, a second averaging processing unit 104a, and a third determination unit 105a;

[0118] The second determining unit 103a is configured to determine initial service feature values ​​corresponding to the N candidate service labels based on the fused denoised images corresponding to the N candidate service labels. For any one of the N candidate service labels, the initial service feature value corresponding to the candidate service label is used to measure the quality of the fused denoised image corresponding to the candidate service label.

[0119] The second averaging processing unit 104a is used to average the initial service characteristic values ​​corresponding to the N candidate service labels to obtain an average service characteristic value;

[0120] The third determining unit 105a is configured to determine a candidate service tag whose initial service characteristic value is greater than the average service characteristic value among the N candidate service tags as a target service tag for the target service image.

[0121] Optionally, the second determination unit 103a is specifically used to: accumulate the pixel values ​​in the fused denoised image corresponding to the candidate business label i to obtain the total pixel value corresponding to the candidate business label i; and determine the total pixel value corresponding to the candidate business label i as the initial business feature value corresponding to the candidate business label i.

[0122] An adding module 1012 is configured to add random noise data corresponding to a random step length t to a sample service image associated with the target service to obtain a noisy image of the sample service image, and to add random noise data corresponding to the random step length t to a general image to obtain a noisy image of the general image; t is a positive integer;

[0123] A first recognition module 1016 is configured to recognize a first type of correlation feature between the general image and a general image label of the general image based on the noisy image of the general image using a general diffusion model;

[0124] A second identification module 1017 is configured to identify predicted noise data in the noisy image of the sample business image based on the first type of correlation features and the annotated business label of the sample business image using an initial diffusion model. The structure of the initial diffusion model is the same as that of the general diffusion model.

[0125] The training module 1018 is configured to train the initial diffusion model according to the predicted noise data and the random noise data corresponding to the random step size t to obtain the service diffusion model.

[0126] Optionally, the upsampling network of the general diffusion model includes L upsampling blocks, and the downsampling network of the general diffusion model includes L downsampling blocks, where L is a positive integer greater than 1;

[0127] The first recognition module 1016 uses a universal diffusion model to identify a first type of correlation feature between the universal image and the universal image label of the universal image based on the noisy image of the universal image, including:

[0128] For the j+1th upsampling iteration, obtain the upsampled image X of the general image output by the upsampling block j of the general diffusion model j ; Upsampled image X of the above general image j It is obtained based on the noise-added image of the above general image; j is a positive integer less than L;

[0129] Obtaining a downsampled image of the general image output by the downsampling block Lj of the general diffusion model at the Ljth downsampling iteration;

[0130] Through the upsampling block j+1 of the general diffusion model, according to the general image label, the upsampled image X j and a downsampled image of the general image at the Lj-th downsampling iteration, identifying a correlation feature between the general image at the j+1-th upsampling iteration and the general image label;

[0131] The correlation features between the general image and the general image label under L upsampling iterations are determined as the first type of correlation features between the general image and the general image label.

[0132] Optionally, the first recognition module 1016 uses the upsampling block j+1 of the general diffusion model to identify the image according to the general image label and the upsampling image X. j and a downsampled image of the general image at the Lj-th downsampling iteration, identifying correlation features between the general image at the j+1-th upsampling iteration and the general image label, including:

[0133] The upsampled image X is processed by upsampling block j+1 of the general diffusion model. j Fusing the general image with the downsampled image obtained at the Lj-th downsampling iteration to obtain a general fused image;

[0134] Get the text weight and image weight corresponding to the upsampling block j+1 of the general diffusion model above;

[0135] The text features corresponding to the general image labels are weighted using the text weights to obtain a weighted text vector;

[0136] Using the above image weights to perform weighted processing on the image features of the above universal fusion image to obtain a weighted image vector;

[0137] According to the weighted text vector and the weighted image vector, a correlation feature between the general image and the general image label in the j+1th upsampling iteration is determined.

[0138] Optionally, the first recognition module 1016 determines, based on the weighted text vector and the weighted image vector, a correlation feature between the general image and the general image label at the j+1th upsampling iteration, including:

[0139] Performing a product process on the weighted text vector and the weighted image vector to obtain an initial correlation feature between the general image and the general image label under the j+1th upsampling iteration;

[0140] Normalizing the initial correlation feature between the general image and the general image label at the j+1th upsampling iteration to obtain a normalized correlation feature;

[0141] According to the normalized correlation feature and the position of the upsampling block j+1 in the universal diffusion model, the correlation feature between the universal image and the universal image label at the j+1th upsampling iteration is determined.

[0142] Optionally, the first type of correlation feature includes a correlation feature between the general image and the general image label in the j+1th upsampling iteration, the upsampling network of the initial diffusion model includes L upsampling blocks, and the downsampling network of the initial diffusion model includes L downsampling blocks, where L is a positive integer greater than 1, and j is a positive integer less than L;

[0143] The second identification module 1017 identifies predicted noise data in the noisy image of the sample business image based on the first type of correlation features and the annotated business label of the sample business image using the initial diffusion model, including:

[0144] For the j+1th upsampling iteration, obtain the upsampled image Y of the sample service image output by the upsampling block j of the initial diffusion model j ; The upsampled image Y of the sample business image j It is obtained based on the noise-added image of the above sample business image;

[0145] Obtaining a downsampled image of the service image output by the downsampling block Lj of the initial diffusion model at the Ljth downsampling iteration;

[0146] Through the upsampling block j+1 of the initial diffusion model, according to the above-mentioned marked business label, the above-mentioned upsampling image Y j and correlation feature j+1, and the downsampled image of the sample business image at the Lj-th downsampling iteration, to identify the second type of correlation feature; the correlation feature j+j reflects the correlation feature between the general image at the j+1-th upsampling iteration and the general image label, and the second type of correlation feature reflects the correlation feature between the sample business image at the j+1-th upsampling iteration and the annotated business label;

[0147] The predicted noise data in the noisy image of the sample service image is determined based on the second type of correlation features obtained by the L-th upsampling iteration.

[0148] Optionally, the second recognition module 1017 uses the upsampling block j+1 of the initial diffusion model to identify the service label and the upsampling image Y j And the correlation feature j+1, as well as the downsampled image of the above sample business image at the Ljth downsampling iteration, identify the second type of correlation features, including:

[0149] The upsampled image Y is processed by the upsampled block j+1 of the initial diffusion model. j Fuse the sample service image with the downsampled image of the Lj-th downsampling iteration to obtain a service fused image;

[0150] Fusing the correlation feature j+1 and the image feature of the service fusion image to obtain an image fusion feature;

[0151] The above-mentioned marked business labels and the above-mentioned image fusion features are identified through the upsampling block j+1 of the above-mentioned initial diffusion model to obtain the second type of correlation features.

[0152] Optionally, the initial diffusion model of the training module 1018 includes an upsampling network. The initial diffusion model is trained based on the predicted noise data and the random noise data corresponding to the random step size t to obtain the service diffusion model, including:

[0153] Calculating the noise prediction error of the initial diffusion model based on the predicted noise data and the random noise data corresponding to the random step size t;

[0154] determining a convergence state of the initial diffusion model based on the noise prediction error;

[0155] If the initial diffusion model is in an unconverged state, the upsampling network of the initial diffusion model is trained according to the noise prediction error to obtain the service diffusion model.

[0156] Optionally, if the initial diffusion model is in an unconverged state, the training module 1018 trains an upsampling network of the initial diffusion model according to the noise prediction error to obtain the service diffusion model, including:

[0157] If the initial diffusion model is in an unconverged state, adjusting the weights of the upsampling network of the initial diffusion model according to the noise prediction error to obtain an adjusted initial diffusion model;

[0158] Using the adjusted initial diffusion model, based on the first type of correlation features and the annotated business labels of the sample business images, updating noise data in the noisy image features of the sample business images is identified;

[0159] Training the upsampling network of the adjusted initial diffusion model based on the updated noise data and the random noise data corresponding to the random step size t;

[0160] Until the adjusted initial diffusion model is in a converged state, the adjusted initial diffusion model in the converged state is determined as the service diffusion model.

[0161] In the present application, since the target business label of the target business image has a strong one-to-one correspondence with the space of the target business image (i.e., the image area), that is, one target business label corresponds to an image area in the target business image. Therefore, the computer device can simultaneously input multiple candidate business labels (i.e., N candidate business labels) into the business diffusion model. The business diffusion model determines the fused denoised image corresponding to the corresponding candidate business label based on the denoised images corresponding to each candidate business label at different time steps (i.e., K time steps). According to the fused denoised images corresponding to the N candidate business labels, the target business label of the target business image is determined from the N candidate business labels. It can be seen that there is no need to input multiple candidate business labels one by one, which is conducive to quickly identifying one or more target business labels of the target business image, reducing the time consumption of the business diffusion model in identifying labels, and improving the label recognition efficiency of the business diffusion model.

[0162] Please refer to Figure 11, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 11, the above-mentioned computer device 1000 can be the first device in the above-mentioned method, specifically a terminal or server, including: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, in some embodiments, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile MeMory), such as at least one disk storage. The memory 1005 may also be at least one storage device away from the aforementioned processor 1001. As shown in FIG. 11 , the memory 1005 , which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a computer application program.

[0163] In the computer device 1000 shown in FIG11 , the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call the computer application stored in the memory 1005 to achieve:

[0164] Obtain a target service image and N candidate service tags associated with the target service; N is a positive integer greater than 1;

[0165] Add random noise data corresponding to K time steps to the target business image to obtain K noisy images of the target business image; K is a positive integer greater than 1;

[0166] Output the N candidate service labels and the K noisy images of the target service image to a service diffusion model. De-noising is performed on the K noisy images of the target service image using the service diffusion model according to the candidate service label i, to obtain de-noised images corresponding to the candidate service label i at the K time steps. i is a positive integer less than or equal to N, and N is a positive integer greater than 1.

[0167] Fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i;

[0168] According to the fused denoised images corresponding to the N candidate service labels, the target service label of the target service image is determined from the N candidate service labels.

[0169] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to fuse the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i, including:

[0170] Averaging the denoised images corresponding to the candidate service label i at the K time steps to obtain an average denoised image corresponding to the candidate service label i;

[0171] The average denoised image corresponding to the candidate service label i is determined as the fused denoised image corresponding to the candidate service label i.

[0172] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement, based on the fused denoised images corresponding to the N candidate service labels, determining a target service label for the target service image from the N candidate service labels, including:

[0173] Determining, based on the fused denoised images corresponding to the N candidate service labels, initial service feature values ​​corresponding to the N candidate service labels; wherein, for any one of the N candidate service labels, the initial service feature value corresponding to the candidate service label is used to measure the quality of the fused denoised image corresponding to the candidate service label;

[0174] Averaging the initial service characteristic values ​​corresponding to the N candidate service labels to obtain an average service characteristic value;

[0175] A candidate service label whose initial service characteristic value is greater than the average service characteristic value among the N candidate service labels is determined as a target service label of the target service image.

[0176] Optionally, the processor 1001 can be used to call a computer application stored in the memory 1005 to determine the initial business feature values ​​corresponding to the N candidate business tags based on the fused denoised images corresponding to the N candidate business tags, including: accumulating the pixel values ​​in the fused denoised image corresponding to the candidate business tag i to obtain the total pixel value corresponding to the candidate business tag i; and determining the total pixel value corresponding to the candidate business tag i as the initial business feature value corresponding to the candidate business tag i.

[0177] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement:

[0178] Adding random noise data corresponding to a random step length t to a sample service image associated with the target service to obtain a noisy image of the sample service image, and adding random noise data corresponding to the random step length t to a general image to obtain a noisy image of the general image; t is a positive integer;

[0179] identifying, by a universal diffusion model, a first type of correlation feature between the universal image and a universal image label of the universal image based on a noisy image of the universal image;

[0180] identifying predicted noise data in the noisy image of the sample business image based on the first type of correlation features and the annotated business label of the sample business image using an initial diffusion model; the structure of the initial diffusion model is the same as that of the general diffusion model;

[0181] The initial diffusion model is trained based on the predicted noise data and the random noise data corresponding to the random step size t to obtain the service diffusion model.

[0182] Optionally, the upsampling network of the universal diffusion model includes L upsampling blocks, and the downsampling network of the universal diffusion model includes L downsampling blocks, where L is a positive integer greater than 1;

[0183] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement, using a universal diffusion model, identifying a first type of correlation feature between the universal image and a universal image label of the universal image based on a noisy image of the universal image, including:

[0184] For the j+1th upsampling iteration, obtain the upsampled image X of the general image output by the upsampling block j of the general diffusion model j ; Upsampled image X of the above general image j It is obtained based on the noise-added image of the above general image; j is a positive integer less than L;

[0185] Obtaining a downsampled image of the general image output by the downsampling block Lj of the general diffusion model at the Ljth downsampling iteration;

[0186] Through the upsampling block j+1 of the general diffusion model, according to the general image label, the upsampled image X j and a downsampled image of the general image at the Lj-th downsampling iteration, identifying a correlation feature between the general image at the j+1-th upsampling iteration and the general image label;

[0187] The correlation features between the general image and the general image label under L upsampling iterations are determined as the first type of correlation features between the general image and the general image label.

[0188] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement the upsampling block j+1 of the general diffusion model, according to the general image label, the upsampling image X j and a downsampled image of the general image at the Lj-th downsampling iteration, identifying correlation features between the general image at the j+1-th upsampling iteration and the general image label, including:

[0189] The upsampled image X is processed by upsampling block j+1 of the general diffusion model. j Fusing the general image with the downsampled image obtained at the Lj-th downsampling iteration to obtain a general fused image;

[0190] Get the text weight and image weight corresponding to the upsampling block j+1 of the general diffusion model above;

[0191] The text features corresponding to the general image labels are weighted using the text weights to obtain a weighted text vector;

[0192] Using the above image weights to perform weighted processing on the image features of the above universal fusion image to obtain a weighted image vector;

[0193] According to the weighted text vector and the weighted image vector, a correlation feature between the general image and the general image label in the j+1th upsampling iteration is determined.

[0194] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to determine, based on the weighted text vector and the weighted image vector, a correlation feature between the general image at the j+1th upsampling iteration and the general image label, including:

[0195] Performing a product process on the weighted text vector and the weighted image vector to obtain an initial correlation feature between the general image and the general image label under the j+1th upsampling iteration;

[0196] Normalizing the initial correlation feature between the general image and the general image label at the j+1th upsampling iteration to obtain a normalized correlation feature;

[0197] According to the normalized correlation feature and the position of the upsampling block j+1 in the universal diffusion model, the correlation feature between the universal image and the universal image label at the j+1th upsampling iteration is determined.

[0198] Optionally, the first type of correlation feature includes a correlation feature between the general image and the general image label in the j+1th upsampling iteration, the upsampling network of the initial diffusion model includes L upsampling blocks, and the downsampling network of the initial diffusion model includes L downsampling blocks, where L is a positive integer greater than 1, and j is a positive integer less than L;

[0199] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement, through an initial diffusion model, identification of predicted noise data in the noisy image of the sample business image based on the first type of correlation feature and the annotated business label of the sample business image, including:

[0200] For the j+1th upsampling iteration, obtain the upsampled image Y of the sample service image output by the upsampling block j of the initial diffusion model j ; The upsampled image Y of the sample business image j It is obtained based on the noise-added image of the above sample business image;

[0201] Obtaining a downsampled image of the service image output by the downsampling block Lj of the initial diffusion model at the Ljth downsampling iteration;

[0202] Through the upsampling block j+1 of the initial diffusion model, according to the above-mentioned marked business label, the above-mentioned upsampling image Y jand correlation feature j+1, and the downsampled image of the sample business image at the Lj-th downsampling iteration, to identify the second type of correlation feature; the correlation feature j+j reflects the correlation feature between the general image at the j+1-th upsampling iteration and the general image label, and the second type of correlation feature reflects the correlation feature between the sample business image at the j+1-th upsampling iteration and the annotated business label;

[0203] The predicted noise data in the noisy image of the sample service image is determined based on the second type of correlation features obtained by the L-th upsampling iteration.

[0204] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement the upsampling block j+1 of the initial diffusion model, according to the above-mentioned marked business label, the above-mentioned upsampling image Y j And the correlation feature j+1, as well as the downsampled image of the above sample business image at the Ljth downsampling iteration, identify the second type of correlation features, including:

[0205] The upsampled image Y is processed by the upsampled block j+1 of the initial diffusion model. j Fuse the sample service image with the downsampled image of the Lj-th downsampling iteration to obtain a service fused image;

[0206] Fusing the correlation feature j+1 and the image feature of the service fusion image to obtain an image fusion feature;

[0207] The above-mentioned marked business labels and the above-mentioned image fusion features are identified through the upsampling block j+1 of the above-mentioned initial diffusion model to obtain the second type of correlation features.

[0208] Optionally, the initial diffusion model includes an upsampling network. The processor 1001 may be configured to call a computer application stored in the memory 1005 to train the initial diffusion model based on the predicted noise data and the random noise data corresponding to the random step size t, thereby obtaining the service diffusion model. The training step may include:

[0209] Calculating the noise prediction error of the initial diffusion model based on the predicted noise data and the random noise data corresponding to the random step size t;

[0210] determining a convergence state of the initial diffusion model based on the noise prediction error;

[0211] If the initial diffusion model is in an unconverged state, the upsampling network of the initial diffusion model is trained according to the noise prediction error to obtain the service diffusion model.

[0212] Optionally, the processor 1001 may be configured to call a computer application stored in the memory 1005 to implement, if the initial diffusion model is in an unconverged state, training an upsampling network of the initial diffusion model according to the noise prediction error to obtain the service diffusion model, including:

[0213] If the initial diffusion model is in an unconverged state, adjusting the weights of the upsampling network of the initial diffusion model according to the noise prediction error to obtain an adjusted initial diffusion model;

[0214] Using the adjusted initial diffusion model, based on the first type of correlation features and the annotated business labels of the sample business images, updating noise data in the noisy image features of the sample business images is identified;

[0215] Training the upsampling network of the adjusted initial diffusion model based on the updated noise data and the random noise data corresponding to the random step size t;

[0216] Until the adjusted initial diffusion model is in a converged state, the adjusted initial diffusion model in the converged state is determined as the service diffusion model.

[0217] In the present application, since the target business label of the target business image has a strong one-to-one correspondence with the space of the target business image (i.e., the image area), that is, one target business label corresponds to an image area in the target business image. Therefore, the computer device can simultaneously input multiple candidate business labels (i.e., N candidate business labels) into the business diffusion model. The business diffusion model determines the fused denoised image corresponding to the corresponding candidate business label based on the denoised images corresponding to each candidate business label at different time steps (i.e., K time steps). According to the fused denoised images corresponding to the N candidate business labels, the target business label of the target business image is determined from the N candidate business labels. It can be seen that there is no need to input multiple candidate business labels one by one, which is conducive to quickly identifying one or more target business labels of the target business image, reducing the time consumption of the business diffusion model in identifying labels, and improving the label recognition efficiency of the business diffusion model.

[0218] It should be understood that the computer device described in the embodiments of the present application can execute the description of the above-mentioned data processing method in the corresponding embodiments above, and can also execute the description of the above-mentioned data processing device in the corresponding embodiments above, and the description thereof will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here.

[0219] In addition, it should be noted that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned data processing device, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the data processing method in the corresponding embodiment above. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0220] As an example, the above program instructions may be deployed on a computer device for execution, or deployed on at least two computer devices at one location for execution, or executed on at least two computer devices distributed at at least two locations and interconnected via a communication network. The at least two computer devices distributed at at least two locations and interconnected via a communication network may constitute a blockchain network.

[0221] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the central storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMART Media card, SMC), a secure digital (SD) card, a flash memory card (flash card), etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the central storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0222] The terms "first," "second," and the like in the description, claims, and drawings of the embodiments of the present application are used to distinguish between contents in different media, rather than to describe a specific order. In addition, the terms "including" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other steps and units inherent to these processes, methods, apparatuses, products, or devices.

[0223] The collection and processing of the relevant data in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0224] The present application also provides a computer program product including a computer program / instruction. When the computer program / instruction is executed by a processor, the data processing method and decoding method described in the corresponding embodiment are implemented. Therefore, they will not be described in detail here. In addition, the description of the beneficial effects of the same method will not be described in detail. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiment of this application.

[0225] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0226] The methods and related devices provided in the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided in the embodiments of the present application. Specifically, each process and / or box in the method flow charts and / or structural diagrams, as well as the combination of the processes and / or boxes in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable network connection device to produce a machine, so that the instructions executed by the processor of the computer or other programmable network connection device generate a device for implementing the function specified in one or more processes in the flow chart and / or one or more boxes in the structural diagram. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable network connection device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a product including an instruction device, which implements the function specified in one or more processes in the flow chart and / or one or more boxes in the structural diagram. These computer program instructions can also be loaded onto a computer or other programmable network-connected device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the structural diagram.

[0227] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Obtain a target business image and N candidate business tags associated with the target business; N is a positive integer greater than 1; Add random noise data corresponding to K time steps to the target business image to obtain K noisy images of the target business image; K is a positive integer greater than 1; Output the N candidate service labels and the K noisy images of the target service image to the service diffusion model, and perform denoising on the K noisy images of the target service image according to the candidate service label i through the service diffusion model to obtain the denoised images corresponding to the candidate service label i at the K time steps, where i is a positive integer less than or equal to N, and N is a positive integer greater than 1; Fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i; According to the fused denoised images respectively corresponding to the N candidate service labels, a target service label of the target service image is determined from the N candidate service labels.

2. The method according to claim 1, characterized in that The step of fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i includes: Averaging the denoised images corresponding to the candidate service label i at the K time steps to obtain an average denoised image corresponding to the candidate service label i; The average denoised image corresponding to the candidate service label i is determined as the fused denoised image corresponding to the candidate service label i.

3. The method according to claim 1 or 2, characterized in that The step of determining the target service label of the target service image from the N candidate service labels according to the fused denoised images respectively corresponding to the N candidate service labels comprises: Determine the initial service feature values ​​corresponding to the N candidate service labels respectively according to the fused denoised images corresponding to the N candidate service labels; wherein, for any one of the N candidate service labels, the initial service feature value corresponding to the candidate service label is used to measure the quality of the fused denoised image corresponding to the candidate service label; Averaging the initial service characteristic values ​​corresponding to the N candidate service labels to obtain an average service characteristic value; A candidate service tag whose initial service characteristic value is greater than the average service characteristic value among the N candidate service tags is determined as a target service tag of the target service image.

4. The method according to claim 3, characterized in that The determining, according to the fused denoised images respectively corresponding to the N candidate service labels, the initial service feature values ​​respectively corresponding to the N candidate service labels comprises: Accumulating the pixel values ​​in the fused denoised image corresponding to the candidate service label i to obtain the total pixel value corresponding to the candidate service label i; The total pixel value corresponding to the candidate service tag i is determined as the initial service feature value corresponding to the candidate service tag i.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: Add random noise data corresponding to the random step length t to a sample service image associated with the target service to obtain a noisy image of the sample service image, and add random noise data corresponding to the random step length t to a general image to obtain a noisy image of the general image; t is a positive integer; identifying, by a universal diffusion model, a first type of correlation feature between the universal image and a universal image label of the universal image according to the noisy image of the universal image; identifying predicted noise data in the noisy image of the sample business image according to the first type of correlation features and the annotated business label of the sample business image through an initial diffusion model; the structure of the initial diffusion model is the same as that of the general diffusion model; The initial diffusion model is trained according to the predicted noise data and the random noise data corresponding to the random step length t to obtain the service diffusion model.

6. The method according to claim 5, characterized in that The upsampling network of the universal diffusion model includes L upsampling blocks, and the downsampling network of the universal diffusion model includes L downsampling blocks, where L is a positive integer greater than 1; The identifying, by using a universal diffusion model and according to the noisy image of the universal image, a first type of correlation feature between the universal image and a universal image label of the universal image comprises: For the j+1th upsampling iteration, obtain the upsampled image X of the general image output by the upsampling block j of the general diffusion model j ; The upsampled image X of the general image j is obtained based on the noisy image of the general image; j is a positive integer less than L; Obtaining a downsampled image of the universal image output by the downsampling block Lj of the universal diffusion model at the Ljth downsampling iteration; Through the upsampling block j+1 of the general diffusion model, according to the general image label, the upsampled image X j and a downsampled image of the general image at the Lj-th downsampling iteration, identifying a correlation feature between the general image at the j+1-th upsampling iteration and the general image label; The correlation features between the general image and the general image label under L upsampling iterations are determined as the first type of correlation features between the general image and the general image label.

7. The method according to claim 6, characterized in that The upsampling block j+1 through the universal diffusion model, according to the universal image label, the upsampling image X j and the downsampled image of the general image at the Lj-th downsampling iteration, identifying the correlation feature between the general image at the j+1-th upsampling iteration and the general image label, comprising: The upsampled image X is processed by the upsampling block j+1 of the general diffusion model. j Fusing the universal image with the downsampled image of the Lj-th downsampling iteration to obtain a universal fused image; Obtaining text weight and image weight corresponding to upsampling block j+1 of the universal diffusion model; Using the text weight to perform weighted processing on the text features corresponding to the universal image label to obtain a weighted text vector; Using the image weight to perform weighted processing on the image features of the universal fusion image to obtain a weighted image vector; Determine, based on the weighted text vector and the weighted image vector, a correlation feature between the general image and the general image label at the j+1th upsampling iteration.

8. The method according to claim 7, characterized in that The determining, according to the weighted text vector and the weighted image vector, a correlation feature between the general image and the general image label at the j+1th upsampling iteration comprises: Performing product processing on the weighted text vector and the weighted image vector to obtain an initial correlation feature between the general image and the general image label at the j+1th upsampling iteration; Normalizing the initial correlation feature between the general image and the general image label at the j+1th upsampling iteration to obtain a normalized correlation feature; The correlation feature between the universal image and the universal image label at the j+1th upsampling iteration is determined according to the normalized correlation feature and the position of the upsampling block j+1 in the universal diffusion model.

9. The method according to claim 5, characterized in that The first type of correlation features include correlation features between the general image and the general image label at the j+1th upsampling iteration, the upsampling network of the initial diffusion model includes L upsampling blocks, the downsampling network of the initial diffusion model includes L downsampling blocks, L is a positive integer greater than 1, and j is a positive integer less than L; The identifying predicted noise data in the noisy image of the sample business image according to the first type of correlation features and the annotated business label of the sample business image through the initial diffusion model includes: For the j+1th upsampling iteration, obtain the upsampled image Y of the sample service image output by the upsampling block j of the initial diffusion model j ; The upsampled image Y of the sample service image j is obtained based on a noise-added image of the sample business image; Obtaining a downsampled image of the business image output by the downsampling block Lj of the initial diffusion model at the Ljth downsampling iteration; Through the upsampling block j+1 of the initial diffusion model, according to the labeled business label, the upsampling image Y j and correlation feature j+1, and the downsampled image of the sample business image at the Ljth downsampling iteration, to identify the second type of correlation feature; the correlation feature j+1 reflects the correlation feature between the general image and the general image label at the j+1th upsampling iteration, and the second type of correlation feature reflects the correlation feature between the sample business image and the annotated business label at the j+1th upsampling iteration; The predicted noise data in the noisy image of the sample business image is determined according to the second type of correlation features obtained by the Lth upsampling iteration.

10. The method according to claim 9, characterized in that The up-sampling block j+1 of the initial diffusion model is based on the labeled service label and the up-sampled image Y j and correlation feature j+1, and the downsampled image of the sample business image at the Ljth downsampling iteration, and the second type of correlation features are identified, including: The upsampled image Y is processed by upsampling block j+1 of the initial diffusion model. j Fusing the sample service image with the downsampled image of the Lj-th downsampling iteration to obtain a service fused image; Fusing the correlation feature j+1 with the image feature of the business fusion image to obtain an image fusion feature; The annotated business label and the image fusion feature are identified through the upsampling block j+1 of the initial diffusion model to obtain a second type of correlation feature.

11. The method according to claim 5, characterized in that The initial diffusion model includes an upsampling network, and the initial diffusion model is trained according to the predicted noise data and the random noise data corresponding to the random step size t to obtain the service diffusion model, including: Calculating a noise prediction error of the initial diffusion model according to the predicted noise data and the random noise data corresponding to the random step size t; determining a convergence state of the initial diffusion model according to the noise prediction error; If the initial diffusion model is in an unconverged state, the upsampling network of the initial diffusion model is trained according to the noise prediction error to obtain the service diffusion model.

12. The method according to claim 11, characterized in that If the initial diffusion model is in an unconverged state, then if the initial diffusion model is in an unconverged state, training an upsampling network of the initial diffusion model according to the noise prediction error to obtain the service diffusion model includes: If the initial diffusion model is in an unconverged state, adjusting the weight of the upsampling network of the initial diffusion model according to the noise prediction error to obtain an adjusted initial diffusion model; Identify, by using the adjusted initial diffusion model, updating noise data in the noisy image features of the sample business image according to the first type of correlation features and the annotated business label of the sample business image; Training the upsampling network of the adjusted initial diffusion model according to the updated noise data and the random noise data corresponding to the random step size t; Until the adjusted initial diffusion model is in a converged state, the adjusted initial diffusion model in a converged state is determined as the service diffusion model.

13. A data processing device, characterized in that: include: An acquisition module, used to acquire a target service image and N candidate service tags associated with the target service; N is a positive integer greater than 1; An adding module is used to add random noise data corresponding to K time steps to the target business image to obtain K time steps of the target business image. Noised image; K is a positive integer greater than 1; A processing module, used for outputting the N candidate service labels and the K noisy images of the target service image to a service diffusion model, and performing denoising processing on the K noisy images of the target service image according to the candidate service label i through the service diffusion model to obtain denoised images corresponding to the candidate service label i at the K time steps, wherein i is a positive integer less than or equal to N, and N is a positive integer greater than 1; A fusion module, used for fusing the denoised images corresponding to the candidate service label i at the K time steps to obtain a fused denoised image corresponding to the candidate service label i; The determination module is used to determine the target service label of the target service image from the N candidate service labels according to the fused denoised images respectively corresponding to the N candidate service labels.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images