Data processing method and device, computer, storage medium and program product

Abstract

The embodiment of the invention discloses a data processing method and device, a computer, a storage medium and a program product, and the method comprises the steps: carrying out the first ordinary differential sampling of a first data feature through a text graph model for a sampling time step before a sliding window in a sampling period, and obtaining a first image feature; the first data feature is the feature of the first text data in the first sampling time step, and is the feature output by the text graph model in the previous sampling time step of the sampling time step in the non-first sampling time step; performing random differential sampling on the first data features through a text graph model for sampling time steps in the sliding window to obtain second image features; performing image decoding on the feature output by the last sampling time step to obtain first image data; and according to the first image data and the sampling features generated in the sliding window, adjusting the text graph model to obtain a target text graph model. By adopting the method, the data processing efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer, storage medium and program product. Background Technology

[0002] Inspired by proximal policy optimization, reinforcement learning is often incorporated into diffusion models when training text-to-image (TPE) models. The scoring function is optimized using policy gradient methods to train the diffusion model, resulting in a TPE model that can generate images more suited to user needs. This training process typically involves using offline data in the text-to-image generation task (i.e., training the TPE model). However, since offline positive and negative sample pairs can easily cause the model to deviate from the original distribution, online learning methods have emerged. Specifically, online learning methods introduce flow matching models, employing random sampling. Training a flow matching model with random sampling yields a TPE model, allowing for more diverse results. However, the use of random sampling requires processing a large amount of data during training, increasing the data volume and leading to significant time and resource consumption. Summary of the Invention

[0003] This application provides a data processing method, apparatus, computer, storage medium, and program product, which can improve the efficiency of data processing and reduce the resource consumption of data processing.

[0004] One embodiment of this application provides a data processing method, the method comprising:

[0005] When the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained; the sampling period is used to represent the N sampling time steps by which the text-image model converts text data into image data; N is a positive integer;

[0006] If the first sampling time step is before the sliding window, the first data feature is processed by the text image model using the first ordinary differential sampling to obtain the first image feature. When the first sampling time step is the first sampling time step among N sampling time steps, the first data feature is the feature of the first text data input to the text image model. When the first sampling time step is not the first sampling time step, the first data feature is the feature output by the text image model in the previous sampling time step.

[0007] If the first sampling time step is within the sliding window, the first data features are randomly differentiated and sampled using the Wensheng image model to obtain the second image features;

[0008] When the first sampling time step is the last sampling time step among N sampling time steps, the features output by the text-to-image model at the first sampling time step are processed by image decoding to obtain the first image data for the first text data.

[0009] Based on the first image data and the sampling features generated within the sliding window, the parameters of the text image model are adjusted to obtain the target text image model.

[0010] One embodiment of this application provides a data processing apparatus, the apparatus comprising:

[0011] The window processing module is used to obtain the range of sampling time steps corresponding to the sliding window in the sampling period when the first sampling time step in the sampling period is executed; the sampling period is used to represent the N sampling time steps in which the text-to-image model converts text data into image data; N is a positive integer;

[0012] The feature sampling module is used to perform first ordinary differential sampling on the first data features through the text image model to obtain the first image features if the first sampling time step is before the sliding window; when the first sampling time step is the first sampling time step among N sampling time steps, the first data features are the features of the first text data input to the text image model; when the first sampling time step is not the first sampling time step, the first data features are the features output by the text image model in the previous sampling time step.

[0013] The feature sampling module is also used to perform random differential sampling on the first data features through the Wensheng image model to obtain the second image features if the first sampling time step is within the sliding window;

[0014] The image decoding module is used to perform image decoding processing on the features output by the text-to-image model at the first sampling time step when the first sampling time step is the last sampling time step among N sampling time steps, so as to obtain the first image data for the first text data.

[0015] The parameter adjustment module is used to adjust the parameters of the text image model based on the first image data and the sampling features generated within the sliding window to obtain the target text image model.

[0016] The device also includes:

[0017] The window determination module is used to obtain the starting sampling time step within the sampling time step range corresponding to the sliding window in the sampling period, and to obtain the fixed number of window iterations corresponding to the starting sampling time step. The fixed number of window iterations is used to represent the number of loop iterations S required for the Wensheng graph model to be executed when the starting position of the sliding window is at the starting sampling time step. One loop iteration includes one or more sampling periods. S is a positive integer.

[0018] The window determination module is also used to count the number of loop iterations performed while the starting position of the sliding window is at the initial sampling time step, as the number of times the window has looped.

[0019] This parameter adjustment module can be used for:

[0020] Based on the first image data and the sampling features generated within the sliding window, the parameters of the text image model are adjusted to obtain a candidate text image model;

[0021] If the candidate text image model does not meet the model convergence condition, the sampling time step range corresponding to the sliding window in the next iteration cycle is determined based on the fixed number of window iterations and the number of window iterations. The iteration cycle refers to a cycle containing S iterations.

[0022] If a candidate text-to-image model satisfies the model convergence condition, then the candidate text-to-image model is determined as the target text-to-image model.

[0023] Specifically, when determining the sampling time step range corresponding to the sliding window in the next iteration cycle based on the fixed number of window iterations and the number of times the window has already iterated, this parameter adjustment module can be used for:

[0024] If the number of times the window has looped is less than the fixed number of times the window has looped, then the sampling time step range corresponding to the sliding window remains unchanged, and the number of times the window has looped is accumulated.

[0025] If the number of times the window has looped is equal to the number of times the window has been fixed, then the sliding window is slid-processed to obtain the sampling time step range corresponding to the sliding window in the next loop iteration cycle. The number of times the window has looped is initialized. Based on the starting sampling time step in the sampling time step range corresponding to the sliding window in the next loop iteration cycle, the number of times the window has been fixed in the next loop iteration cycle is determined.

[0026] Specifically, when determining the fixed number of window iterations for the next iteration cycle based on the starting sampling time step within the sampling time step range corresponding to the sliding window in the next iteration cycle, this parameter adjustment module can be used for:

[0027] Obtain the moving attenuation parameter, and perform exponential attenuation processing on the moving attenuation parameter according to the starting sampling time step in the sampling time step range corresponding to the sliding window in the next cycle iteration to obtain the cycle attenuation coefficient corresponding to the next cycle iteration.

[0028] The initial fixed number of iterations is adjusted using a round decay coefficient to obtain the fixed number of iterations for the next cycle.

[0029] One iteration consists of M sampling periods, and the Wensheng image model is used to generate a first image data in one sampling period; M is a positive integer;

[0030] When adjusting the parameters of the raw image model based on the first image data and the sampling features generated within the sliding window to obtain a candidate raw image model, the parameter adjustment module can be used for:

[0031] Obtain the i-th first image data; i is a positive integer less than or equal to M;

[0032] Based on the sampling features generated by the i-th first image data within the sliding window, the i-th first loss data is generated. The i-th first loss data is used to adjust the parameters of the intermediate text-to-image model to obtain the i-th adjusted text-to-image model. If i is 1, the intermediate text-to-image model is the text-to-image model. If i is greater than 1, the intermediate text-to-image model is the (i-1)-th adjusted text-to-image model corresponding to the (i-1)-th first image data.

[0033] When the adjusted text-to-image models corresponding to the M first image data are obtained, second loss data is generated based on the image quality corresponding to the M first image data. The parameters of the Mth adjusted text-to-image model are adjusted using the second loss data to obtain the candidate text-to-image model.

[0034] The device also includes:

[0035] The feature sampling module is also used to perform a second ordinary differential sampling process on the features output by the previous sampling time step of the first sampling time step through the Wensheng graph model if the first sampling time step is located after the sliding window, so as to obtain the third image feature.

[0036] Specifically, if the first sampling time step is after the sliding window, and the feature output from the previous sampling time step is processed by the Wensheng graph model using the second ordinary differential sampling to obtain the third image feature, this feature sampling module can be used for:

[0037] If the first sampling time step is after the sliding window, then the device performance of the computer device is obtained; the computer device is the device that processes the first sampling time step.

[0038] If the device performance falls within the first performance range, the first ordinary differential sampling is determined as the second ordinary differential sampling. The features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model to obtain the third image features.

[0039] If the device performance data falls within the second performance range, then the higher-order ordinary differential sampling is determined as the second ordinary differential sampling. The features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model using the second ordinary differential sampling to obtain the third image features. The minimum performance value of the second performance range is greater than the maximum performance value of the first performance range.

[0040] This parameter adjustment module can be used for:

[0041] Obtain the model sampling parameters corresponding to the Wensheng image model. Using the model sampling parameters, perform random differential sampling on the sampling features generated within the sliding window to obtain random differential features. Based on the sampling features generated within the sliding window and the random differential features, generate the first loss data.

[0042] The first image data is used to generate the second loss data. The first loss data and the second loss data are used to adjust the parameters of the text-based image model to obtain the target text-based image model.

[0043] The model sampling parameters include historical sampling parameters. When obtaining the model sampling parameters corresponding to the Wensheng image model, and using these parameters to perform random differential sampling on the sampling features generated within the sliding window to obtain random differential features, and generating the first loss data based on the sampling features generated within the sliding window and the random differential features, this parameter adjustment module can be used for:

[0044] Obtain the historical sampling parameters from the historical textual graph model corresponding to the textual graph model; the textual graph model is obtained by adjusting the parameters of the historical textual graph model.

[0045] Using historical sampling parameters, random differential sampling is performed on the sampling features generated within the sliding window to obtain random differential features; both the sampling features generated within the sliding window and the random differential features are probability distributions.

[0046] The probability distribution ratio between the sampled features and the random differential features generated within the sliding window is determined as the parameter optimization coefficient.

[0047] Based on the first image data, the parameter dominance values ​​of the sampling parameters in the Wensheng image model are determined, and the parameter dominance values ​​are adjusted using parameter optimization coefficients to obtain the first loss data.

[0048] The device also includes:

[0049] The data sampling module is used to input the second text data into the target text image model, and to perform feature sampling processing on the second text data through the target text image model to obtain the target image features.

[0050] The image generation module is used to decode the features of the target image to obtain the target image data.

[0051] Specifically, when performing feature sampling processing on the second text data using the target text-to-image model to obtain target image features, this data sampling module can be used for:

[0052] The device performance of the computer device running the target text image model is obtained, and a target feature sampler is selected based on the device performance. The target text image model includes multiple feature samplers, which have the same sampling parameters but use different sampling methods. The multiple feature samplers include the target feature sampler.

[0053] The target image features are obtained by performing feature sampling on the second text data using a target feature sampler.

[0054] Wherein, the number of target image features is A, and A is a positive integer;

[0055] This image generation module can be used for:

[0056] Decode the A target image features respectively to obtain the second image data corresponding to the A target image features;

[0057] Obtain the image quality corresponding to A second image data points, and determine the second image data with the highest image quality as the target image data.

[0058] One embodiment of this application provides a computer device, including a processor, a memory, and an input / output interface;

[0059] The processor is connected to a memory and an input / output interface, respectively. The input / output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device containing the processor executes the data processing method in one aspect of the embodiments of this application.

[0060] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the data processing method of one aspect of this application.

[0061] One aspect of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional embodiments of this application. In other words, when the computer instructions are executed by the processor, they implement the methods provided in various optional embodiments of this application.

[0062] Implementing the embodiments of this application will have the following beneficial effects:

[0063] In this embodiment, when the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained; the sampling period is used to represent N sampling time steps by which the Wensheng graph model converts text data into image data; N is a positive integer; if the first sampling time step is before the sliding window, the first data feature is processed by the Wensheng graph model using the first ordinary differential sampling to obtain the first image feature; when the first sampling time step is the first sampling time step in the N sampling time steps, the first data feature is the feature of the first text data input to the Wensheng graph model; when the first sampling time step is not the first sampling time step... During the step-by-step process, the first data feature is the feature output by the text-based image model in the previous sampling time step. If the first sampling time step is within a sliding window, the first data feature is processed by random differential sampling through the text-based image model to obtain the second image feature. When the first sampling time step is the last sampling time step among N sampling time steps, the feature output by the text-based image model in the first sampling time step is processed by image decoding to obtain the first image data for the first text data. Based on the first image data and the sampling features generated within the sliding window, the parameters of the text-based image model are adjusted to obtain the target text-based image model. Through the above process, two sampling techniques, first ordinary differential sampling and random differential sampling, are used in combination during model training. This allows the text-based image model to be trained by reducing the amount of data to be processed, thereby improving the efficiency of model training and reducing the resource consumption of data processing, through first ordinary differential sampling; and by random differential sampling, the text-based image model can achieve diversified sampling and random result generation, thereby improving the performance of model training. Meanwhile, the sampling time step range for random differential sampling is determined based on the sliding window, and only the sampling features generated within the sliding window are introduced into the parameter adjustment process of the Wensheng image model, while the sampling features generated outside the sliding window are not processed. This can effectively reduce the data processing overhead of model training and improve the efficiency of model training. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a network interaction architecture diagram for data processing provided in an embodiment of this application;

[0066] Figure 2 This is a schematic diagram of a data processing scenario provided in an embodiment of this application;

[0067] Figure 3 This is a flowchart of a data processing method provided in an embodiment of this application;

[0068] Figure 4 This is a flowchart of another data processing method provided in an embodiment of this application;

[0069] Figure 5 This is a flowchart illustrating a specific data processing method provided in an embodiment of this application;

[0070] Figure 6 This is a schematic diagram of a model parameter adjustment scenario provided in an embodiment of this application;

[0071] Figure 7 This is a flowchart of a method for generating text images according to an embodiment of this application;

[0072] Figure 8 This is a schematic diagram of a scene of a text-based image provided in an embodiment of this application;

[0073] Figure 9 This is a schematic diagram of a data processing device provided in an embodiment of this application;

[0074] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0076] If this application requires the collection of object data (such as user data), a prompt interface or pop-up window will be displayed before and during the collection process. This prompt interface or pop-up window is used to inform the user that certain data is being collected. The data acquisition steps will only begin after the user confirms the prompt interface or pop-up window; otherwise, the process will end. Furthermore, the acquired user data will be used in reasonable and legal scenarios or for legitimate purposes. Optionally, in scenarios where user data needs to be used but user authorization has not been obtained, authorization can be requested from the user, and the user data can only be used after authorization is granted.

[0077] In the embodiments of this application, please refer to Figure 1 , Figure 1 This is a network interaction architecture diagram for data processing provided in an embodiment of this application, such as... Figure 1 As shown, computer device 101 can acquire first text data and use it to adjust the parameters of the text-to-image model to obtain a target text-to-image model. The text-to-image model can be considered a flow-matching model, which learns to transform a "flow" of noise distribution into a continuous mapping of data distribution to achieve efficient feature sampling and image generation. Computer device 101 can acquire the first text data from local storage or from any one or more business devices (such as business devices 102a, 102b, or 102c, etc.), without limitation. For example, in one scenario, computer device 101 can respond to a text-to-image model training request from business device 102a, acquire the first text data, use it to adjust the parameters of the text-to-image model to obtain the target text-to-image model, and send the target text-to-image model to business device 102a. Any device containing the target text-to-image model (such as a computer device or business device, etc.) can use the target text-to-image model to convert text data into image data.

[0078] Specifically, during the sampling period, the computer device 101 performs ordinary differential sampling on the features in the input text image model at sampling time steps outside the sliding window; and performs random differential sampling on the features in the input text image model at sampling time steps within the sliding window. The sampling period represents the number of sampling processes required by the text image model to convert text data into image data. One sampling period includes N sampling time steps, each corresponding to one sampling process, where N is a positive integer. When sampling of the first text data is complete, i.e., the text image model has performed N sampling processes, the computer device 101 can perform image decoding on the last feature output by the text image model during the sampling period to obtain the first image data for the first text data. The computer device 101 can then adjust the parameters of the text image model based on the first image data and the features generated within the sliding window to obtain a target text image model, which is used to convert text data into image data. Ordinary differential sampling (ODS) refers to a sampling method that obtains a definite result after sampling the input features. In other words, given the initial conditions and sampling parameters, ODS will yield a unique result. Simply put, ODS refers to the continuous change of variables over time under deterministic laws, without noise terms. Random differential sampling, on the other hand, refers to a sampling method that obtains a random result after sampling the input features. In other words, given the same initial conditions and sampling parameters, the result may be different each time random differential sampling is performed. Simply put, random differential sampling is a sampling method that introduces noise terms in continuous time. It is often used to model dynamic systems with randomness and is used in generation tasks such as diffusion models or flow matching models to describe the evolution of data and noise. By integrating random differential sampling and ordinary differential sampling, this approach allows for flexible selection of the sampling method at different sampling stages. Random differential sampling enables the model to discover a wider variety of generated results, avoiding local optima and increasing the diversity of generated outcomes. Meanwhile, ordinary differential sampling reduces the amount of data required for model training, increasing the sampling speed at time steps outside the sliding window, thus improving training efficiency and reducing data processing resource consumption. Furthermore, by determining the sampling time step range for random differential sampling based on the sliding window, and only incorporating the sampling features generated within the sliding window into the parameter adjustment process of the text image model while ignoring sampling features generated outside the sliding window, the data processing overhead for model training can be effectively reduced, further improving training efficiency.

[0079] Specifically, when the first sampling time step in the sampling period is reached, the computer device can obtain the range of sampling time steps corresponding to the sliding window in the sampling period. This sampling period represents the number of sampling processes required for the text-to-image model to convert text data into image data. One sampling period includes N sampling time steps, where N is a positive integer. If the first sampling time step is outside the sliding window, the text-to-image model can perform ordinary differential sampling processing on the first data feature to obtain the first image feature. If the first sampling time step is within the sliding window, the text-to-image model can perform random sampling processing on the first data feature to obtain the second image feature. When the first sampling time step is the first of the N sampling time steps, the first data feature is the feature of the first text data input to the text-to-image model. When the first sampling time step is not the first sampling time step, the first data feature is the feature output by the text-to-image model in the previous sampling time step. Furthermore, when the first sampling time step is the last sampling time step out of N sampling time steps, that is, the features output by the textural image model at the last sampling time step are obtained, and image decoding processing is performed on the features output by the textural image model at the last sampling time step to obtain the first image data. Further, based on the first image data and the sampling features generated within the sliding window, the parameters of the textural image model can be adjusted to obtain the target textural image model.

[0080] For example, a continuous sampling method in the sampling selection process of this application can be seen in formula ①:

[0081]

[0082] As shown in formula ①, t represents the current sampling time step (denoted as the first sampling time step), S represents the range of sampling time steps corresponding to the sliding window, "t∈S" indicates that the first sampling time step is within the sliding window, and "otherwise" indicates that the first sampling time step is outside the sliding window. Here, g(t) represents the diffusion term in the first sampling time step, and dW represents Brownian motion, used to generate random noise. The diversity of model results is achieved through g(t)dW. Where X... t This is used to represent the state of the Wensheng graph model during sampling processing, including features input into the Wensheng graph model at the first sampling time step. Furthermore, the computer device can discretize this continuous sampling method to obtain a discrete sampling method. Based on the discrete sampling method, random differential sampling is performed at sampling time steps within the sliding window, and ordinary differential sampling is performed at sampling time steps outside the sliding window. For example, a possible discrete sampling method obtained based on formula ① can be seen in formula ②:

[0083]

[0084] As shown in formula ②, V θ (X t ,t) represents the deterministic feature output by the textural graph model at the first sampling time step t, θ represents the sampling parameters used in the textural graph model, and Δt represents the interval between two adjacent sampling time steps, i.e., "X t "+Δt" ​​is used to represent the result predicted by the Wensheng graph model at the first sampling time step t; and The random result is obtained by discretizing formula ①.

[0085] In other words, the sampling period can be divided into multiple time step ranges based on the range of sampling time steps corresponding to the sliding window within the sampling period. Specifically, when the starting sampling time step within the sampling time step range corresponding to the sliding window is the first sampling time step in the sampling period, the sampling period can be divided into a second time step range and a third time step range. When the starting sampling time step within the sampling time step range corresponding to the sliding window is not the first sampling time step in the sampling period, and the ending sampling time step within the sampling time step range corresponding to the sliding window is not the last sampling time step in the sampling period, the sampling period can be divided into a first time step range, a second time step range, and a third time step range. When the ending sampling time step within the sampling time step range corresponding to the sliding window is the last sampling time step in the sampling period, the sampling period can be divided into a first time step range and a second time step range. The first time step range includes the sampling time steps in the sampling period before the sliding window; the second time step range includes the sampling time steps in the sampling period within the sliding window; and the third time step range includes the sampling time steps in the sampling period after the sliding window. Specifically, within the first time step, the features of the input text-based image model are sampled using the first ordinary differential method; within the second time step, the features of the input text-based image model are sampled using the random differential method; and within the third time step, the features of the input text-based image model are sampled using the second ordinary differential method.

[0086] For example, please see Figure 2 , Figure 2 This is a schematic diagram of a data processing scenario provided in an embodiment of this application. For example... Figure 2As shown, assuming that the starting sampling time step in the sampling time step range corresponding to the sliding window is not the first sampling time step in the sampling period, and the ending sampling time step in the sampling time step range corresponding to the sliding window is not the last sampling time step in the sampling period, the sampling period 202 includes a first time step range 2021, a second time step range 2022, and a third time step range 2023. The sampling period 202 includes N sampling time steps, the first time step range 2021 includes sampling time steps t1 to t3, and the second time step range 2022 includes sampling time steps t4 to t5. 4+w-1 The third time step range of 2023 includes the sampling time step t. 4+w up to sampling time step t N , where w represents the window size of the sliding window 203, and w is a positive integer, that is, the range of sampling time steps corresponding to the sliding window 203 in any sampling period includes w sampling time steps.

[0087] At this time, during sampling period 202, the first text data 201 is input into the text graph model 204. At sampling time step t1, the text graph model 204 performs a first ordinary differential sampling process on the features of the first text data 201 to obtain the first sampled features output by the text graph model 204 at sampling time step t1. At this time, the first data feature is the feature of the first text data 201. At sampling time step t2, the text graph model 204 performs a first ordinary differential sampling process on the first sampled features corresponding to sampling time step t1 to obtain the first sampled features output by the text graph model 204 at sampling time step t2. At this time, the first data feature is the first sampled feature corresponding to sampling time step t1. Features; at sampling time step t3, the first sampling feature corresponding to sampling time step t2 is processed by the Wensheng graph model 204 using the first ordinary differential sampling process, resulting in the first sampling feature output by the Wensheng graph model 204 at sampling time step t3. At this time, the first data feature is the first sampling feature corresponding to sampling time step t2; at sampling time step t4, the first sampling feature corresponding to sampling time step t3 is processed by the Wensheng graph model 204 using the random differential sampling process, resulting in the second sampling feature output by the Wensheng graph model 204 at sampling time step t4. At this time, the first data feature is the first sampling feature corresponding to sampling time step t3; ..., similarly, the first sampling feature output by the Wensheng graph model 204 at sampling time step t4 is obtained. 4+w-1 The output second sampling feature; at sampling time step t 4+w The sampling time step t was analyzed using the Wensheng graph model 204. 4+w-1 The corresponding second sampling feature is subjected to second ordinary differential sampling processing to obtain the Wensheng graph model 204 at sampling time step t. 4+w The third sampling feature is output, where the first data feature is the sampling time step t. 4+w-1The corresponding second sampling feature; ..., similarly, the Wensheng graph model 204 is obtained at sampling time step t. N The third sampled feature is output. At this point, the last sampled time step t out of the N sampled time steps in the sampling period 202 is obtained. N The third sampling feature is that computer equipment can measure the sampling time step t. N The corresponding third sampling feature is used for image decoding to obtain the first image data 205 for the first text data 201. The first, second, and third sampling features are only used to distinguish the time step range in which the sampling feature was generated. For example, the first sampling feature represents the sampling feature generated within the first time step range, the second sampling feature represents the sampling feature generated within the second time step range, and the third sampling feature represents the sampling feature generated within the third time step range, etc. The computer device can adjust the parameters of the text-based image model based on the first image data 205 and the sampling features 206 generated within the sliding window 203 to obtain the target text-based image model. In this way, the framework used for training the text-based image model integrates both ordinary differential sampling and random differential sampling methods. This allows the model to retain its predictive diversity by using random differential sampling for only a portion of the sampling time steps, while using ordinary differential sampling for the rest. This reduces the number of sampling time steps using random differential sampling, thereby reducing the amount of data required for model training, lowering data processing resource consumption, and improving model training efficiency.

[0088] It is understood that the business equipment mentioned in the embodiments of this application can also be a computer device, and the computer device in the embodiments of this application includes, but is not limited to, terminal devices or servers. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. The terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, smart speakers, digital cameras, webcams, and other mobile internet devices (MIDs) with network access capabilities, or terminal devices in scenarios such as trains, ships, and flights. Figure 1 As shown, the terminal device can be a laptop computer (as shown in business device 102b), a mobile phone (as shown in business device 102c), or an in-vehicle device (as shown in business device 102a), etc. Figure 1This is just a partial list of devices. Optionally, business device 102a refers to the device located in vehicle 103. Business device 102a can be used to run text-based image models and process image data 1021, etc. The servers mentioned above can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-to-everything (V2X) communication, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0089] Optionally, the data involved in the embodiments of this application may be stored in a computer device, or may be stored based on cloud storage technology or a blockchain network, without limitation.

[0090] Further, please see Figure 3 , Figure 3 This is a flowchart of a data processing method provided in an embodiment of this application. Figure 3 As shown, the data processing procedure includes the following steps:

[0091] Step S301: When the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained.

[0092] In this embodiment, during the sampling period, the computer device can obtain the range of sampling time steps corresponding to the sliding window within the sampling period. The sampling period represents the N sampling time steps by which the Wensheng graph model converts text data into image data; N is a positive integer. The range of sampling time steps corresponding to the sliding window within the sampling period indicates the starting and ending sampling time steps of the sliding window within the sampling period. The number of sampling time steps included in the sliding window can be denoted as w, where w is a positive integer. A sliding window is a window with a fixed window size (w) that can slide progressively across a data sequence. It can be used to process local subsequences sequentially and is widely used in signal processing and time series analysis. In diffusion models or flow matching models, the model's result generation process is divided into multiple discrete time steps, each corresponding to a noise level or sampling stage. In this application, the Wensheng graph model's process of converting text data into image data is divided into N discrete sampling time steps, each corresponding to one sampling process. Optionally, the text-to-image model can generate one or more image data in one sampling period. That is, after the text-to-image model performs N sampling processes on the first text data, it can obtain one or more first image data. This is because the text-to-image model performs random differential sampling on the features in the input text-to-image model at the sampling time step within the sliding window. Based on the random differential sampling result, randomness is generated, which allows the text-to-image model to generate multiple image data in one sampling period.

[0093] Optionally, the computer device can acquire window state information corresponding to the sampling period, and obtain the sampling time step range corresponding to the sliding window in that sampling period from the window state information. The window state information corresponding to the sampling period includes the sampling time step range corresponding to the sliding window in that sampling period, the number of times the sliding window is fixed in that sampling period, and the number of times the window has looped. The window state information of the sliding window is updated once after each loop iteration; that is, the window state information used within a sampling period remains unchanged. One loop iteration includes one or more sampling periods. One loop iteration represents the complete process of converting text data into image data using the text-to-image model, adjusting the parameters of the text-to-image model based on this conversion process, and obtaining a new model that requires parameter convergence detection.

[0094] The sampling time step range corresponding to the sliding window in the sampling period is obtained by sliding the sliding window based on the initial sampling time step range. Specifically, when training the Wensheng graph model, the sampling time step range of the sliding window in the first iteration is the initial time step range, and the sampling time step range in the j-th iteration is the time step range obtained by sliding the initial time step range j-1 times, where j is a positive integer. For example, assuming the initial time step range is [t1, t2]... w If ], then the sampling time step range in the j-th iteration is [t] 1+(j-1)*step , t (j-1)*step+w ], where step represents the window sliding step size, that is, the number of sampling time steps that the sliding window slides once. The number of fixed window times in this sampling period is obtained based on the starting sampling time step within the sampling time step range of the sliding window in this sampling period, as shown in formula ⑥ below.

[0095] When generating the window state information of the sliding window, the computer device can obtain the starting sampling time step within the sampling time step range corresponding to the sliding window in the sampling period, and obtain the fixed number of window iterations corresponding to the starting sampling time step. The fixed number of window iterations is used to represent the number of loop iterations S required for the Wensheng graph model to execute when the starting position of the sliding window is at the starting sampling time step. One loop iteration includes one or more sampling periods; S is a positive integer. During the process when the starting position of the sliding window is at the starting sampling time step, the number of loop iterations executed is counted as the number of times the window has looped.

[0096] Furthermore, the computer device can detect the relative positional relationship between the first sampling time step and the range of sampling time steps corresponding to the sliding window within the sampling period. Based on this relative positional relationship, the first data feature is sampled and processed. This relative positional relationship includes being located before, within, and after the sliding window. Specifically, when the first sampling time step is the first of N sampling time steps, the first data feature is the feature of the first text data input to the text-to-image model; when the first sampling time step is not the first of N sampling time steps, the first data feature is the feature output by the text-to-image model in the previous sampling time step, such as... Figure 2 The relevant descriptions are shown below. Specifically, if the first sampling time step is before the sampling time step range, it means that the first sampling time step is before the sliding window, and step S302 can be executed; if the sampling time step range includes the first sampling time step, it means that the first sampling time step is within the sliding window, and step S303 can be executed. For example, assume the first sampling time step is t. kThe range of sampling time steps corresponding to the sliding window in the sampling period is [t]. l , t l+w-1 If k is less than l, it means the first sampling time step is before the sliding window, and step S302 is executed; if k is greater than or equal to l, and k is less than or equal to l+w-1, it means the first sampling time step is within the sliding window, and step S303 is executed. Here, k, l, and w are all positive integers less than or equal to N.

[0097] Step S302: If the first sampling time step is before the sliding window, the first data feature is processed by the first ordinary differential sampling through the Wensheng graph model to obtain the first image feature.

[0098] In this embodiment, if the first sampling time step is before the sliding window, the first data feature is processed by the text image model using the first ordinary differential sampling method to obtain the first image feature. Specifically, when the first sampling time step is the first of N sampling time steps, the first data feature is the feature of the first text data input to the text image model; when the first sampling time step is not the first sampling time step, the first data feature is the feature output by the text image model in the previous sampling time step. Furthermore, the computer device can execute the next sampling time step after the first sampling time step; that is, it determines the next sampling time step as the first sampling time step and returns to step S301.

[0099] Step S303: If the first sampling time step is within the sliding window, then the first data features are randomly differentiated and sampled using the Wensheng image model to obtain the second image features.

[0100] In this embodiment, if the first sampling time step is within a sliding window, the first data feature is randomly differentially sampled using the text-based image model to obtain the second image feature. Specifically, when the first sampling time step is the first of N sampling time steps, the first data feature is the feature of the first text data input to the text-based image model; when the first sampling time step is not the first sampling time step, the first data feature is the feature output by the text-based image model in the previous sampling time step. Furthermore, the computer device can detect the position of the first sampling time step among the N sampling time steps. When the first sampling time step is not the last of the N sampling time steps, the next sampling time step is executed; that is, the next sampling time step is determined as the first sampling time step, and the process returns to step S301. When the first sampling time step is the last of the N sampling time steps, step S304 is executed.

[0101] Step S304: When the first sampling time step is the last sampling time step among N sampling time steps, the features output by the text-to-image model in the first sampling time step are processed by image decoding to obtain the first image data for the first text data.

[0102] In this embodiment, when the first sampling time step is the last sampling time step among N sampling time steps, the computer device can use an image decoder to perform image decoding processing on the features output by the text-to-image model in the first sampling time step to obtain the first image data for the first text data. Further, step S305 can be executed.

[0103] Step S305: Based on the first image data and the sampling features generated within the sliding window, adjust the parameters of the raw image model to obtain the target raw image model.

[0104] In this embodiment, the computer device can obtain the model sampling parameters corresponding to the textural image model, and use the model sampling parameters to perform random differential sampling processing on the sampling features generated within the sliding window to obtain random differential features. Based on the sampling features generated within the sliding window, the random differential features, and the first image data, model loss data is generated, and the model loss data is used to adjust the parameters of the textural image model to obtain the target textural image model.

[0105] Specifically, in one loss processing method, the model loss data includes first loss data generated based on the sampling features, stochastic differential features, and first image data generated within the sliding window. This first loss data represents the relative optimization value of the strategy in the model and can be implemented through Group Relative Policy Optimization (GRPO). In other words, the computer device can generate only the first loss data and use it to adjust the parameters of the textural image model to obtain the target textural image model.

[0106] Alternatively, in one loss processing method, the model loss data includes first loss data generated based on the sampling features, random differential features, and first image data generated within the sliding window, and second loss data generated based on the image quality of the first image data. Specifically, the computer device can obtain the model sampling parameters corresponding to the textural image model, use the model sampling parameters to perform random differential sampling processing on the sampling features generated within the sliding window to obtain random differential features, generate first loss data based on the sampling features and random differential features generated within the sliding window, generate second loss data based on the first image data, and use the first loss data and second loss data to adjust the parameters of the textural image model to obtain the target textural image model.

[0107] Optionally, in any of the above loss processing methods, when generating the first loss data, the model sampling parameters include historical sampling parameters. These historical sampling parameters refer to the sampling parameters used in the previous historical text-based image model. It can be considered that the first sampling parameters used in the text-based image model are obtained by adjusting these historical sampling parameters. Specifically, the computer device can obtain the historical sampling parameters from the historical text-based image model corresponding to the text-based image model; the text-based image model is obtained by adjusting the parameters of the historical text-based image model. Using the historical sampling parameters, random differential sampling processing is performed on the sampling features generated within the sliding window to obtain random differential features. Both the sampling features generated within the sliding window and the random differential features are probability distributions. The probability distribution ratio between the sampling features generated within the sliding window and the random differential features is determined as the parameter optimization coefficient. Based on the first image data, the parameter dominance value of the sampling parameters in the text-based image model is determined, and the parameter optimization coefficient is used to adjust the parameter dominance value to obtain the first loss data. The number of first image data can be M, where M is a positive integer. These M first image data can be obtained by the text image model performing N sampling operations on the first text data within one sampling period; alternatively, the M first image data can be obtained by the text image model performing sampling operations on the first text data for M sampling periods, with each sampling period including N sampling time steps, and the text image model obtaining one first image data in each sampling period. An optional first loss data can be found in formula ③:

[0108]

[0109] As shown in formula ③, θ is used to represent the sampling parameters in the Wensheng diagram model; θ old Used to represent historical sampling parameters; S is used to represent the sampling time step range of the sliding window, and w is used to represent the window size of the sliding window; Used to represent the policy ratio between the sampling parameters of the i-th first image data in the Wensheng image model and the historical sampling parameters within the sliding window, and is the parameter optimization coefficient; A i The parameter advantage score is represented by 'c', where 'i' represents the first image data and 'c' represents the first text data. The `clip` function limits the parameter optimization coefficients to the range [1-ε, 1+ε], where ε represents the policy update magnitude (e.g., ε = 0.0001), which can be considered a hyperparameter that can be changed as needed. By using the `clip` function, the update magnitude of the sampling parameters in the model is limited, preventing over-optimization of a single sample from causing abrupt changes in the sampling parameters, thereby improving the accuracy of model training. This is used to represent the method for calculating the strategy optimization value of a text-based graph model within a sliding window when adjusting the parameters of the model based on the first text data. This is used to represent the parameter advantage value of the Wensheng graph model when using the sampling parameter θ, and is used to represent the first loss data mentioned above.

[0110] Alternatively, one possible method for determining the parameter optimization coefficients can be found in formula ④:

[0111]

[0112] As shown in formula ④, q θ (X t+1 |X t c), used to represent the text-generated image model trained on the first text data c, including the sampling parameter θ, in terms of features X. t Based on the prediction, feature X is obtained t+1 The probability distribution; Used to represent, when using the first text data c, for the sampled parameter θ old When training the text-based image model, the text-based image model in feature X t Based on the prediction, feature X is obtained t+1 The probability distribution of X. t X is used to represent the features in the input textural graph model. t+1 Used to represent the Wensheng graph model for feature X t The features obtained after sampling. In the process of generating the first loss data mentioned above, θ is the first sampling parameter in the textural graph model. old These are historical sampling parameters.

[0113] One possible way to determine the parameter dominance value is shown in formula ⑤:

[0114]

[0115] As shown in formula ⑤, This is used to represent the parameter reward corresponding to the i-th first image data. Specifically, the computer device can standardize the parameter rewards corresponding to the M first image data respectively to obtain the parameter advantage values ​​corresponding to the M first image data respectively. For example... Figure 3 As shown, the mean of the parameter reward can be obtained for each of the M first image data points. and parameter reward standard deviation The ratio of the difference between the parameter reward and the mean of the parameter rewards for the i-th first image data to the standard deviation of the parameter rewards is determined as the parameter advantage value corresponding to the i-th first image data. Optionally, when obtaining the parameter reward corresponding to the i-th first image data, multiple reward models can be used to obtain the image quality of the i-th first image data separately. The image quality corresponding to the i-th first image data under the multiple reward models is then integrated to obtain the parameter reward for the i-th first image data. The image quality corresponding to the i-th first image data under the multiple reward models can respectively reflect the semantic quality and aesthetic appeal of the i-th first image data, etc.

[0116] Formulas ③ to ⑤ above represent one possible process for generating the first loss data. Alternatively, the first loss data can be determined based on reward calculation methods that require the use of other relative optimization strategies. No restrictions are imposed here.

[0117] Optionally, the computer device may perform multiple iterations on the textural image data to train the textural image model. Specifically, the computer device can adjust the parameters of the textural image model based on the first image data and the sampling features generated within the sliding window to obtain candidate textural image models. If the candidate textural image model does not meet the model convergence condition, the sampling time step range corresponding to the sliding window in the next iteration cycle is determined based on the fixed number of window iterations and the number of iterations the window has already undergone. The iteration cycle refers to a cycle containing S iterations. If the candidate textural image model meets the model convergence condition, then the candidate textural image model is determined as the target textural image model.

[0118] In determining the sampling time step range of the sliding window in the next iteration cycle based on the fixed number of window iterations and the number of window iterations, if the number of window iterations is less than the fixed number of window iterations, the sampling time step range of the sliding window remains unchanged, and the number of window iterations is accumulated. If the number of window iterations equals the fixed number of window iterations, the sliding window is slidable to obtain the sampling time step range of the sliding window in the next iteration cycle. The number of window iterations is initialized, and the fixed number of window iterations for the next iteration cycle is determined based on the initial sampling time step in the sampling time step range of the sliding window in the next iteration cycle. When sliding the sliding window, a window sliding step size can be used. The window sliding step size represents the number of sampling time steps required for one sliding operation. The window sliding step size is the time step interval between the initial sampling time step in the sampling time step range of the sliding window in the next iteration cycle and the initial sampling time step in the sampling time step range of the sliding window in the current iteration cycle.

[0119] In determining the number of fixed window iterations for the next iteration cycle based on the initial sampling time step within the sampling time step range corresponding to the sliding window in the next iteration cycle, the computer device can acquire a moving attenuation parameter. Based on the initial sampling time step within the sampling time step range corresponding to the sliding window in the next iteration cycle, the moving attenuation parameter is subjected to exponential attenuation processing to obtain the round attenuation coefficient corresponding to the next iteration cycle. The round attenuation coefficient is then used to adjust the initial fixed number of iterations to obtain the number of fixed window iterations for the next iteration cycle. Here, it is assumed that the initial sampling time step within the sampling time step range corresponding to the sliding window in the current iteration cycle is t. l Then the starting sampling time step in the sampling time step range corresponding to the sliding window in the next iteration cycle is t. l+step `step` represents the window sliding step size. At this point, the window state information corresponding to the sliding window in the next iteration cycle can be obtained. The window state information corresponding to the next iteration cycle includes the sampling time step range "[t]" of the next iteration cycle. l+step , t l+step+w-1The values ​​are: τ(l+step)*, the number of times the window has been cyclically rotated in the next iteration cycle, and the number of times the window has already rotated in the next iteration cycle. The initial value for the number of times the window has rotated in the next iteration cycle is 1. For example, assuming w is 4, l is 3, and step is 1, the sampling time step range of the sliding window in the current iteration cycle is {t3, t4, t5, t6}. After sliding the window, the sampling time step range in the next iteration cycle is {t4, t5, t6, t7}. In simpler terms, taking the initial sampling time step of the sliding window as the l-th sampling time step out of N sampling time steps as an example, the sampling time step range of this sliding window can be denoted as sp(l) = {t... l , t l+1 , ..., t l+w-1}, l≤Nw.

[0120] Optionally, the process for determining the fixed number of windows corresponding to the next iteration cycle can be found in formula ⑥:

[0121] τ(l+step)=τ0·exp(-p·ReLU(l+step-λ thr )) ⑥

[0122] As shown in Formula ⑥, "τ(l+step)" represents the number of times the sliding window is fixed when the initial sampling time step is "l+step" within the sampling time step range; τ0 represents the initial number of fixed steps. The moving attenuation parameter can include the attenuation factor and the attenuation initiation threshold, where p represents the attenuation factor, and λ... thr The threshold value used to represent the decay initiation timing is exp(x); exp(x) is used to represent the exponential function and is used to calculate e. x ReLU(x) refers to the corrected linear unit, where ReLU(x) = max(0, x). This corrected linear unit controls when the decay process begins for a fixed number of sampling time steps of the sliding window. In other words, it controls when the sliding window slides to the λth sampling time step out of N sampling time steps. thr Only when the sampling time range of a certain sampling time step is reached will the decay process of the sliding window for a fixed number of times begin. This ensures that there are enough sampling periods for diverse model training, improving the accuracy of model training. Wherein, τ0, p, and λ... thrThis can be considered a hyperparameter. The movement along the sampling direction (i.e., the sampling direction over N sampling time steps) reflects the randomness of the probability flow from strong to weak. By determining the number of window fixations as shown in Equation ③, the number of window fixations can decrease as the sliding window slides over N sampling time steps. This allows the change in the number of window fixations during model training to align with the practice of assigning discount factors to rewards in reinforcement learning, i.e., assigning higher weights to rewards from a larger search space in the early stages. This allows the model training to focus more on optimization within a larger search space, thereby improving the accuracy of model training. The probability flow is a way of describing the continuous evolution of a probability distribution over time, used to smoothly transform a noise distribution into a data distribution through deterministic equations.

[0123] As shown in Equation ⑥, when the starting sampling time step corresponding to the sliding window is located between the first sampling time step and the λth sampling time step in the N sampling time steps... thr Within the range of λ sampling time steps, the number of fixed times corresponding to the sliding window is always the initial fixed number τ0. When the starting sampling time step corresponding to the sliding window is located at the λth sampling time step... thr After a certain number of sampling time steps, the fixed number of times the sliding window corresponds to will gradually decrease as the sliding window slides. For example, assuming τ0 is 20, p is 2, and λ... thr If the value is 5, then when the initial sampling time step of the sliding window is any one of the first to the fifth sampling time step among the N sampling time steps, the fixed number of the window corresponding to the sliding window is 20; when the initial sampling time step of the sliding window is the sixth sampling time step, the fixed number of the window corresponding to the sliding window is "20*exp(-2*ReLU(6-5))≈3"; when the initial sampling time step of the sliding window is the seventh sampling time step, the fixed number of the window corresponding to the sliding window is "20*exp(-2*ReLU(7-5))≈1".

[0124] Of course, formula ⑥ above is one possible way to determine the number of window fixations. A linear decay method can also be used to determine the number of window fixations, such as τ(l+step)=τ0 / (p·ReLU(l+step-λ)). thr )) etc., are not restricted here.

[0125] Optionally, one iteration includes M sampling periods, where the text-to-image model is used to generate a first image data in one sampling period; M is a positive integer. At this time, while adjusting the parameters of the text-to-image model based on the first image data and the sampling features generated within the sliding window to obtain a candidate text-to-image model, the computer device can acquire the i-th first image data; i is a positive integer less than or equal to M. Based on the sampling features generated within the sliding window using the i-th first image data, the i-th first loss data is generated. The i-th first loss data is used to adjust the parameters of the intermediate text-to-image model to obtain the i-th adjusted text-to-image model; if i is 1, the intermediate text-to-image model is the text-to-image model; if i is greater than 1, the intermediate text-to-image model is the (i-1)-th adjusted text-to-image model corresponding to the (i-1)-th first image data. When M adjusted raw image models corresponding to the first image data are obtained, second loss data is generated based on the image quality corresponding to each of the M first image data. The parameters of the Mth adjusted raw image model are then adjusted using the second loss data to obtain candidate raw image models. In other words, the parameters of the raw image model can be adjusted sequentially using the features corresponding to the M first image data within the sliding window to obtain a raw image model based on the features adjusted by the M first image data within the sliding window. Then, the parameters of the adjusted raw image model can be further adjusted using the M first image data to obtain candidate raw image models.

[0126] Alternatively, the computer device can acquire the i-th first image data. Based on the sampling features generated within the sliding window of the i-th first image data and the i-th first image data itself, the i-th first loss data is generated. The parameters of the intermediate raw image model are adjusted using the i-th first loss data to obtain the i-th adjusted raw image model. If i is 1, the intermediate raw image model is the raw image model; if i is greater than 1, the intermediate raw image model is the (i-1)-th adjusted raw image model corresponding to the (i-1)-th first image data. When the adjusted raw image models corresponding to M first image data are obtained, the M-th adjusted raw image model is determined as a candidate raw image model.

[0127] Specifically, when generating the i-th first loss data based on the sampling features generated within the sliding window of the i-th first image data and the i-th first image data itself, the strategy optimization coefficients can be determined based on the probability distribution ratio of the sampling features generated within the sliding window of the i-th first image data to the random differential features corresponding to the i-th first image data; the strategy advantage value corresponding to the i-th first image data is determined based on the image quality of the i-th first image data; and the strategy optimization coefficients and the strategy advantage value are combined to form the i-th first loss data. Wherein, when i is 1, the random differential features corresponding to the i-th first image data are the default features; when i is not 1, the random differential features corresponding to the i-th first image data are obtained by using the (i-1)th adjusted text-to-image model to perform random differential sampling processing on the features generated within the sliding window of the i-th first image data. Optionally, a possible method for obtaining the i-th first loss data can be found in Formula ⑦:

[0128]

[0129] As shown in formula ⑦, A i The parameter advantage value corresponding to the i-th first image data can be found in the implementation process of formula ⑤; This is used to represent the policy ratio between the sampling parameters of the i-th first image data in the textural image model and the historical sampling parameters within the sliding window. It is a parameter optimization coefficient, and its implementation can be found in formula ④. Here, θ represents the sampling parameters used in the intermediate textural image model. If i is 1 or 2, then θ... old For the sampling parameters used in the textural graph model, if i is greater than 2, then θ old These are the sampling parameters used for the (i-2)th adjusted text image model.

[0130] In this embodiment, when adjusting the parameters of the textural graph model, two sampling methods, random differential sampling and ordinary differential sampling, are integrated. A sliding window that moves along the sampling process is used. Random differential sampling is used within the sliding window, and ordinary differential sampling is used outside the sliding window. Optimization is performed only on the sampling time steps within the sliding window. This ensures the diversity of images generated by the textural graph model and allows the model to concentrate resources on parameter optimization based on features within the sliding window, thereby reducing the number of sampling time steps and resource consumption. Furthermore, due to the equivalence of random differential sampling and ordinary differential sampling in the probability flow—that is, random differential sampling and ordinary differential sampling have the same marginal distribution—it is ensured that the textural graph model will not deviate from the marginal distribution of a single ordinary differential sampling or a single random differential sampling at each sampling time step, thus improving the accuracy of model training. Here, marginal distribution refers to the probability distribution of the remaining variables after summing or integrating a subset of variables in a multidimensional probability distribution. Optionally, the sliding window can be decayed a fixed number of times to achieve a gradual increase in the speed of parameter adjustments in the Wensheng graph model. This sliding window allows for scheduling of the sampling time steps for random differential sampling, enabling a gradual increase in the difficulty of the search space. This ensures that the gradual increase in parameter adjustments aligns with the training principles of reinforcement learning, thereby improving the efficiency and accuracy of model training. Furthermore, since policy updates are only performed within the sliding window, even when using the second loss data, this data evaluates the model's results rather than the policy update itself. This allows for more focused gradient optimization during model training, improving the efficiency of model parameter optimization.

[0131] For example, suppose one iteration includes one sampling period. In the j-th iteration, assume the sliding window has a fixed window count of 5 and has already circulated 4 times, with a sampling time step range of [t2, t6], where j is a positive integer. At this point, the computer can use the first text data to adjust the parameters of the text graph model, obtaining a candidate text graph model. If the candidate text graph model parameters do not converge, the fixed window count and the already circulated window count are compared. Since the fixed window count and the already circulated window count are different, the already circulated window count can be accumulated, resulting in a window already circulated 5 times in the (j+1)-th iteration. The candidate text graph model is then determined as the text graph model in the (j+1)-th iteration, and the next iteration begins. In the (j+1)th iteration, the parameters of the text-based graph model are adjusted using the first text data to obtain a candidate text-based graph model. Assuming that the parameters of the candidate text-based graph model do not converge, the number of times the window is fixed is compared with the number of times the window has been looped. At this time, both the number of times the window has been looped and the number of times the window is fixed are 5, that is, the number of times the window has been looped and the number of times the window is fixed are the same. The computer device can slide the sliding window to obtain the sampling time step range of the sliding window in the (j+2)th iteration. Assuming that the window sliding step size is 1, the sampling time step range of the sliding window in the (j+2)th iteration is [t3, t7]. Based on the sampling time step t3, the number of times the window is fixed in the (j+2)th iteration is determined. Assuming that the number of times the window is fixed is 2, the number of times the window has been looped is initialized, and the candidate text-based graph model is determined as the text-based graph model in the (j+3)th iteration, and the next iteration is performed. Similarly, the sampling time step range of the sliding window in the (j+3)th to (j+4)th iteration is [t3, t7]. The sliding window is then slid-processed to obtain the sampling time step range of the sliding window in the (j+5)th iteration, until the candidate text graph model with converged parameters is obtained. The candidate text graph model with converged parameters is determined as the target text graph model.

[0132] Further, please see Figure 4 , Figure 4 This is a flowchart of another data processing method provided in an embodiment of this application. Figure 4 As shown, the data processing procedure includes the following steps:

[0133] Step S401: When the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained.

[0134] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S301. Optionally, in this process, if the first sampling time step is after the sampling time step range, it means that the first sampling time step is after the sliding window, and step S404 can be executed.

[0135] Step S402: If the first sampling time step is before the sliding window, the first data feature is processed by the first ordinary differential sampling through the Wensheng graph model to obtain the first image feature.

[0136] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S302. Here, the first ordinary differential sampling can be a conventional ordinary differential sampling, such as ODE (Ordinary Differential Equation).

[0137] Step S403: If the first sampling time step is within the sliding window, then the first data features are randomly differentiated and sampled using the Wensheng image model to obtain the second image features.

[0138] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S303. Among them, stochastic differential sampling (SDE) refers to differential sampling that introduces noise terms in continuous time.

[0139] Step S404: If the first sampling time step is after the sliding window, the features output by the previous sampling time step of the first sampling time step are processed by the Wensheng image model to obtain the third image features.

[0140] In this embodiment, the second ordinary differential sampling can be either the first ordinary differential sampling (ODE) or a higher-order ordinary differential sampling (DPMSolver++). Higher-order ordinary differential sampling can be considered an efficient diffusion model sampling algorithm that accelerates the data generation process through higher-order numerical methods, generating high-quality images with fewer sampling time steps. Since the sampling time steps after the sliding window only affect the calculation of parameter rewards and do not affect the policy optimization calculation process within the sliding window, a more efficient higher-order ordinary differential sampling can be used for the sampling time steps after the sliding window. This improves the efficiency of model training without affecting its accuracy, thereby reducing resource consumption during training, improving the balance between training overhead and model performance, and enhancing the stability and efficiency of model training.

[0141] In this process, after the sliding window, either the first ordinary differential sampling (ODS) can be directly determined as the second ODS, or the higher-order differential sampling can be directly determined as the second ODS. Optionally, the computer device can also select a sampling method for sampling at the sampling time step after the sliding window based on its device performance. Specifically, if the first sampling time step is after the sliding window, the device performance of the computer device is obtained; the computer device is the device that processes the first sampling time step. If the device performance falls within a first performance range, the first ODS is determined as the second ODS, and the features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model using the second ODS to obtain the third image feature. If the device performance falls within a second performance range, the higher-order ODS is determined as the second ODS, and the features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model using the second ODS to obtain the third image feature. The minimum performance value of the second performance range is greater than the maximum performance value of the first performance range. This allows for more efficient higher-order ordinary differential sampling at sampling time steps after the sliding window when the device performance is good, and first ordinary differential sampling when the device performance is poor and cannot support higher-order ordinary differential sampling. This enables adaptive selection of sampling methods and improves the performance of model training.

[0142] Step S405: When the first sampling time step is the last sampling time step among N sampling time steps, the features output by the text-to-image model in the first sampling time step are processed by image decoding to obtain the first image data for the first text data.

[0143] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S304.

[0144] Step S406: Based on the first image data and the sampling features generated within the sliding window, adjust the parameters of the raw image model to obtain the target raw image model.

[0145] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S305.

[0146] In this embodiment, ordinary differential sampling and random differential sampling can be decoupled so that the sampling time step after the sliding window will not affect the optimization detection of policy update. Thus, in one optional method, higher-order ordinary differential sampling can be used in the sampling time step after the sliding window to accelerate the sampling process of the sampling time step after the sliding window, thereby improving the efficiency of the model.

[0147] Further, please see Figure 5 , Figure 5 This is a flowchart illustrating a specific data processing method provided in an embodiment of this application. Figure 5 As shown, the computer device can acquire first text data 501, encode the first text data 501 using a text encoder 502 to obtain first text features, and input the first text features into the text graph model 503. Further, the computer device can initialize the window state information of the sliding window. This window state information includes an initial fixed number of iterations, an initial sampling time step range, and an initial number of window iterations. The initial fixed number of iterations can be denoted as τ0, representing the number of iterations required for the sliding window to be located within the initial sampling time step range when parameter adjustments to the text graph model 503 begin. The initial sampling time step range can be denoted as [t1, t2, t3, t4, t5, t6, t7, t8, t9, t1 ... w Furthermore, the computer device can also initialize the iteration number j and the position k of the current sampling time step within the sampling period.

[0148] Specifically, the data processing procedure includes the following steps:

[0149] Step S501: Detect the relative positional relationship between the k-th sampling time step and the sliding window.

[0150] In this embodiment, the computer device can detect the relative position of the k-th sampling time step and the sliding window. If the k-th sampling time step is before the sliding window, step S502 is executed; if the k-th sampling time step is within the sliding window, step S503 is executed; if the k-th sampling time step is after the sliding window, step S504 is executed. For details, please refer to... Figure 4 The relevant description in step S401.

[0151] Step S502, first ordinary differential sampling.

[0152] In this embodiment of the application, the computer device can perform a first ordinary differential sampling on the first data feature, as detailed in the following example. Figure 4 The relevant description in step S402.

[0153] Step S503, random differential sampling.

[0154] In this embodiment of the application, the computer device can perform random differential sampling on the first data feature, as detailed in the following article. Figure 4 The relevant description in step S403.

[0155] Step S504, second ordinary differential sampling.

[0156] In this embodiment of the application, the computer device can perform a second ordinary differential sampling on the features output by the Wensheng graph model at the (k-1)th sampling time step, as detailed in the following example. Figure 4 The relevant description in step S404.

[0157] Step S505: Detect whether the current sampling period has ended.

[0158] In this embodiment, the computer device can detect whether the current sampling period has ended, that is, whether N sampling time steps have been completed, which means it can determine whether k is N. Specifically, if the current sampling period is detected to have ended, image decoding processing is performed on the features output by the Wensheng graph model at the Nth sampling time step to obtain the first image data 505, and step S506 is executed; if the current sampling period is detected to have not ended, k is updated (i.e., k++ is executed), and the process returns to step S501, that is, the sampling processing for the next sampling time step is performed. This process can be found in [reference needed]. Figure 4 The relevant description in step S405.

[0159] Step S506: Obtain the random differential features corresponding to the sliding window.

[0160] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S305.

[0161] Step S507: Obtain model loss data.

[0162] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S305.

[0163] Step S508: Adjust the parameters.

[0164] In the embodiments of this application, this process can be referred to Figure 3 The relevant description of the candidate text-based graph model in step S305 is as follows: Specifically, after obtaining the model loss data, the computer device can determine the gradient of the sampling parameters in the text-based graph model through backpropagation based on the model loss data. Then, using an optimization algorithm and the gradient of the sampling parameters, the sampling parameters are adjusted to obtain the candidate text-based graph model. For example, the sampling parameters can be adjusted using a parameter optimizer to obtain the candidate text-based graph model; the gradient of the sampling parameters is statistically analyzed using a gradient accumulation module; and based on the gradient statistically analyzed by the gradient accumulation module (including the gradient of the sampling parameters), step S509 is executed to check whether the sampling parameters of the candidate text-based graph model have converged.

[0165] Step S509: Check whether the parameters have converged.

[0166] In the embodiments of this application, this process can be referred to Figure 3 The relevant description in step S305. Specifically, if parameter convergence is detected, the currently obtained text graph model is determined as the target text graph model 506; if parameter non-convergence is detected, the window state information of the sliding window is updated 504, k is initialized, and the next loop iteration begins.

[0167] Optionally, when the amount of the first image data is M, see [reference needed]. Figure 6 , Figure 6 This is a schematic diagram illustrating a model parameter adjustment scenario provided in an embodiment of this application. For example... Figure 6 As shown, in one iterative cycle, it is assumed that the iterative cycle includes M sampling periods, each sampling period includes N sampling time steps 602, and the N sampling time steps 602 in each sampling period are divided into a first time step range 6021, a second time step range 6022, and a third time step range 6023. In the first sampling period 61 of this iterative cycle, the computer device can predict the first text data 601 using the text-to-image model 607, and after N sampling processes, obtain the first image data 6031; ...; in the Mth sampling period 6M of this iterative cycle, the first text data 601 is predicted using the text-to-image model 607, and after N sampling processes, the first image data 603M is obtained. The computer device can use the sampling features 6061 generated by the text-to-image model 607 within the sliding window 603 in the first sampling period 61 to adjust the parameters of the text-to-image model 607, obtaining the i-th adjusted text-to-image model, where i is 1. Furthermore, it checks whether i is M. If i is not M, then the next first image data is processed, that is, i++ is executed, and the sampling features generated by the textural image model 607 within the sliding window 603 in the second sampling period are used to adjust the parameters of the (i-1)th adjusted textural image model to obtain the i-th adjusted textural image model. This process continues until the sampling features 606M generated by the textural image model 607 within the sliding window 603 in the Mth sampling period 6M are used to adjust the parameters of the (M-1)th adjusted textural image model to obtain the M-th adjusted textural image model. Furthermore, if the parameters of the M-th adjusted textural image model do not converge, then the next loop iteration is performed; if the parameters of the M-th adjusted textural image model converge, then the M-th adjusted textural image model is determined as the target textural image model.

[0168] In this embodiment, when the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained; the sampling period is used to represent N sampling time steps by which the Wensheng graph model converts text data into image data; N is a positive integer; if the first sampling time step is before the sliding window, the first data feature is processed by the Wensheng graph model using the first ordinary differential sampling to obtain the first image feature; when the first sampling time step is the first sampling time step in the N sampling time steps, the first data feature is the feature of the first text data input to the Wensheng graph model; when the first sampling time step is not the first sampling time step... During the step-by-step process, the first data feature is the feature output by the text-based image model in the previous sampling time step. If the first sampling time step is within a sliding window, the first data feature is processed by random differential sampling through the text-based image model to obtain the second image feature. When the first sampling time step is the last sampling time step among N sampling time steps, the feature output by the text-based image model in the first sampling time step is processed by image decoding to obtain the first image data for the first text data. Based on the first image data and the sampling features generated within the sliding window, the parameters of the text-based image model are adjusted to obtain the target text-based image model. Through the above process, two sampling techniques, first ordinary differential sampling and random differential sampling, are used in combination during model training. This allows the text-based image model to be trained by reducing the amount of data to be processed, thereby improving the efficiency of model training and reducing the resource consumption of data processing, through first ordinary differential sampling; and by random differential sampling, the text-based image model can achieve diversified sampling and random result generation, thereby improving the performance of model training. Meanwhile, the sampling time step range for random differential sampling is determined based on the sliding window, and only the sampling features generated within the sliding window are introduced into the parameter adjustment process of the Wensheng image model, while the sampling features generated outside the sliding window are not processed. This can effectively reduce the data processing overhead of model training and improve the efficiency of model training.

[0169] Further, please see Figure 7 , Figure 7 This is a flowchart illustrating a method for generating text-based images according to an embodiment of this application. Figure 7 As shown, the data processing procedure includes the following steps:

[0170] Step S701: Input the second text data into the target text-to-image model, and perform feature sampling processing on the second text data through the target text-to-image model to obtain the target image features.

[0171] In this embodiment of the application, the computer device can input the second text data into the target text image model, and perform feature sampling processing on the second text data through the target text image model to obtain the target image features.

[0172] After the target text image model is trained, the internal sampling method can be fixed (such as first ordinary differential sampling, random differential sampling, higher-order ordinary differential sampling, and accelerated sampling), or multiple feature samplers can be provided. The computer device can select a target feature sampler from the multiple feature samplers based on requirements to perform feature sampling processing on the second text data. For example, in one optional scheme, the computer device can obtain the device performance of the computer device running the target text image model and select the target feature sampler based on the device performance; the target text image model includes multiple feature samplers, all of which have the same sampling parameters but use different sampling methods; the multiple feature samplers include the target feature sampler. The target feature sampler performs feature sampling processing on the second text data to obtain the target image features. For example, the multiple feature samplers may include, but are not limited to, a sampler that uses first ordinary differential sampling, a sampler that uses random differential sampling, a sampler that uses higher-order ordinary differential sampling, and a sampler that uses accelerated sampling. Accelerated sampling refers to a sampling method that compresses the number of sampling steps. For example, the text-to-image model converts the second text data into target image features through N sampling time steps. This accelerated sampling can compress N into a positive integer less than N.

[0173] Step S702: Decode the target image features to obtain target image data.

[0174] In this embodiment, the number of target image features can be 1. In this case, the computer device can directly decode the target image features to obtain target image data and output the target image data. Alternatively, the number of target image features can be A, where A is a positive integer. In this case, the computer device can decode A target image features respectively to obtain second image data corresponding to each of the A target image features; obtain the image quality corresponding to each of the A second image data; and determine the second image data with the highest image quality as the target image data.

[0175] For example, see Figure 8 , Figure 8 This is a schematic diagram of a scenario for a text-based image provided in an embodiment of this application. For example... Figure 8As shown, the target text-to-image model includes a feature sampler, an image decoder, a reward model, and an image output layer. The computer device can input the second text data into a text encoder for encoding to obtain second text features; then, the target text-to-image model performs feature transformation on the second text features to obtain target image features, and finally performs image decoding on the target image features to obtain target image data. Optionally, the computer device can perform distributed resource scheduling on graphics processing unit (GPU) clusters and content delivery network (CDN) nodes to obtain scheduled resources, and use these resources to run the target text-to-image model to obtain target image data.

[0176] In this process, when using a target text-to-image model to transform the second text features to obtain target image features, and then performing image decoding on the target image features to obtain target image data, the computer device can sample the second text features using a target feature sampler to obtain target image features, and then use an image decoder to decode the target image features to obtain the second image data. The target feature sampler can be a pre-trained feature sampler used by the target text-to-image model, a randomly selected feature sampler from multiple feature samplers, or a feature sampler selected from multiple feature samplers based on scheduling resources or the performance of the computer device; no restrictions are placed here. Optionally, when the number of second image data is A, the image quality corresponding to each of the A second image data can be determined using a reward model, and the second image data with the highest image quality can be determined as the target image data. Furthermore, the target image data can be output to an interactive console through an image output layer, displayed on the interactive console, or sent to a service device requesting text-to-image processing of the second text data.

[0177] Furthermore, by conducting experiments using existing text-to-image models and the text-to-image model in this application, the model indices of the existing text-to-image models and the text-to-image model in this application can be obtained, as shown in Table 1. It can be seen that the performance of the text-to-image model in this application is significantly improved.

[0178] Table 1

[0179]

[0180] Further, please see Figure 9 , Figure 9This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus can be a computer program (including program code, etc.) running on a computer device; for example, the data processing apparatus can be application software. The apparatus can be used to execute corresponding steps in the methods provided in the embodiments of this application. Figure 9 As shown, the data processing device 900 can be used for Figure 3 Specifically, the computer device 900 in the corresponding embodiment may include: a window processing module 11, a feature sampling module 12, an image decoding module 13, and a parameter adjustment module 14.

[0181] Window processing module 11 is used to obtain the range of sampling time steps corresponding to the sliding window in the sampling period when the first sampling time step in the sampling period is executed; the sampling period is used to represent N sampling time steps for the text-to-image model to convert text data into image data; N is a positive integer;

[0182] The feature sampling module 12 is used to perform first ordinary differential sampling processing on the first data features through the text image model to obtain the first image features if the first sampling time step is before the sliding window; when the first sampling time step is the first sampling time step among N sampling time steps, the first data features are the features of the first text data input to the text image model; when the first sampling time step is not the first sampling time step, the first data features are the features output by the text image model in the previous sampling time step.

[0183] The feature sampling module 12 is also used to perform random differential sampling on the first data features through the Wensheng image model to obtain the second image features if the first sampling time step is within the sliding window;

[0184] Image decoding module 13 is used to perform image decoding processing on the features output by the text-to-image model at the first sampling time step when the first sampling time step is the last sampling time step among N sampling time steps, so as to obtain the first image data for the first text data.

[0185] The parameter adjustment module 14 is used to adjust the parameters of the text image model according to the first image data and the sampling features generated in the sliding window to obtain the target text image model.

[0186] The device 900 also includes:

[0187] The window determination module 15 is used to obtain the starting sampling time step within the sampling time step range corresponding to the sliding window in the sampling period, and to obtain the fixed number of windows corresponding to the starting sampling time step; the fixed number of windows is used to represent the number of loop iterations S required for the Wensheng graph model to be executed when the starting position of the sliding window is at the starting sampling time step; one loop iteration includes one or more sampling periods; S is a positive integer;

[0188] The window determination module 15 is also used to count the number of loop iterations performed during the initial sampling time step when the starting position of the sliding window is at the beginning of the window;

[0189] This parameter adjustment module 14 can be used for:

[0190] Based on the first image data and the sampling features generated within the sliding window, the parameters of the text image model are adjusted to obtain a candidate text image model;

[0191] If the candidate text image model does not meet the model convergence condition, the sampling time step range corresponding to the sliding window in the next iteration cycle is determined based on the fixed number of window iterations and the number of window iterations. The iteration cycle refers to a cycle containing S iterations.

[0192] If a candidate text-to-image model satisfies the model convergence condition, then the candidate text-to-image model is determined as the target text-to-image model.

[0193] Specifically, when determining the sampling time step range corresponding to the sliding window in the next iteration cycle based on the fixed number of window iterations and the number of window iterations, the parameter adjustment module 14 can be used for:

[0194] If the number of times the window has looped is less than the fixed number of times the window has looped, then the sampling time step range corresponding to the sliding window remains unchanged, and the number of times the window has looped is accumulated.

[0195] If the number of times the window has looped is equal to the number of times the window has been fixed, then the sliding window is slid-processed to obtain the sampling time step range corresponding to the sliding window in the next loop iteration cycle. The number of times the window has looped is initialized. Based on the starting sampling time step in the sampling time step range corresponding to the sliding window in the next loop iteration cycle, the number of times the window has been fixed in the next loop iteration cycle is determined.

[0196] Specifically, when determining the fixed number of window iterations for the next iteration cycle based on the starting sampling time step within the sampling time step range corresponding to the sliding window in the next iteration cycle, the parameter adjustment module 14 can be used for:

[0197] Obtain the moving attenuation parameter, and perform exponential attenuation processing on the moving attenuation parameter according to the starting sampling time step in the sampling time step range corresponding to the sliding window in the next cycle iteration to obtain the cycle attenuation coefficient corresponding to the next cycle iteration.

[0198] The initial fixed number of iterations is adjusted using a round decay coefficient to obtain the fixed number of iterations for the next cycle.

[0199] One iteration consists of M sampling periods, and the Wensheng image model is used to generate a first image data in one sampling period; M is a positive integer;

[0200] When adjusting the parameters of the raw image model based on the first image data and the sampling features generated within the sliding window to obtain a candidate raw image model, the parameter adjustment module 14 can be used for:

[0201] Obtain the i-th first image data; i is a positive integer less than or equal to M;

[0202] Based on the sampling features generated by the i-th first image data within the sliding window, the i-th first loss data is generated. The i-th first loss data is used to adjust the parameters of the intermediate text-to-image model to obtain the i-th adjusted text-to-image model. If i is 1, the intermediate text-to-image model is the text-to-image model. If i is greater than 1, the intermediate text-to-image model is the (i-1)-th adjusted text-to-image model corresponding to the (i-1)-th first image data.

[0203] When the adjusted text-to-image models corresponding to the M first image data are obtained, second loss data is generated based on the image quality corresponding to the M first image data. The parameters of the Mth adjusted text-to-image model are adjusted using the second loss data to obtain the candidate text-to-image model.

[0204] The device 900 also includes:

[0205] The feature sampling module 12 is also used to perform second ordinary differential sampling processing on the features output by the previous sampling time step of the first sampling time step through the Wensheng graph model if the first sampling time step is located after the sliding window, so as to obtain the third image feature.

[0206] Wherein, if the first sampling time step is after the sliding window, and the feature output from the previous sampling time step is processed by the Wensheng graph model to obtain the third image feature through the second ordinary differential sampling process, the feature sampling module 12 can be used for:

[0207] If the first sampling time step is after the sliding window, then the device performance of the computer device is obtained; the computer device is the device that processes the first sampling time step.

[0208] If the device performance falls within the first performance range, the first ordinary differential sampling is determined as the second ordinary differential sampling. The features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model to obtain the third image features.

[0209] If the device performance data falls within the second performance range, then the higher-order ordinary differential sampling is determined as the second ordinary differential sampling. The features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model using the second ordinary differential sampling to obtain the third image features. The minimum performance value of the second performance range is greater than the maximum performance value of the first performance range.

[0210] The parameter adjustment module 14 can be used for:

[0211] Obtain the model sampling parameters corresponding to the Wensheng image model. Using the model sampling parameters, perform random differential sampling on the sampling features generated within the sliding window to obtain random differential features. Based on the sampling features generated within the sliding window and the random differential features, generate the first loss data.

[0212] The first image data is used to generate the second loss data. The first loss data and the second loss data are used to adjust the parameters of the text-based image model to obtain the target text-based image model.

[0213] The model sampling parameters include historical sampling parameters. When obtaining the model sampling parameters corresponding to the Wensheng image model, and using these parameters to perform random differential sampling processing on the sampling features generated within the sliding window to obtain random differential features, and generating the first loss data based on the sampling features generated within the sliding window and the random differential features, the parameter adjustment module 14 can be used for:

[0214] Obtain the historical sampling parameters from the historical textual graph model corresponding to the textual graph model; the textual graph model is obtained by adjusting the parameters of the historical textual graph model.

[0215] Using historical sampling parameters, random differential sampling is performed on the sampling features generated within the sliding window to obtain random differential features; both the sampling features generated within the sliding window and the random differential features are probability distributions.

[0216] The probability distribution ratio between the sampled features and the random differential features generated within the sliding window is determined as the parameter optimization coefficient.

[0217] Based on the first image data, the parameter dominance values ​​of the sampling parameters in the Wensheng image model are determined, and the parameter dominance values ​​are adjusted using parameter optimization coefficients to obtain the first loss data.

[0218] The device 900 also includes:

[0219] The data sampling module 16 is used to input the second text data into the target text image model, and to perform feature sampling processing on the second text data through the target text image model to obtain the target image features;

[0220] The image generation module 17 is used to decode the features of the target image to obtain the target image data.

[0221] Specifically, when performing feature sampling processing on the second text data using the target text-to-image model to obtain target image features, the data sampling module 16 can be used for:

[0222] The device performance of the computer device running the target text image model is obtained, and a target feature sampler is selected based on the device performance. The target text image model includes multiple feature samplers, which have the same sampling parameters but use different sampling methods. The multiple feature samplers include the target feature sampler.

[0223] The target image features are obtained by performing feature sampling on the second text data using a target feature sampler.

[0224] Wherein, the number of target image features is A, and A is a positive integer;

[0225] The image generation module 17 can be used for:

[0226] Decode the A target image features respectively to obtain the second image data corresponding to the A target image features;

[0227] Obtain the image quality corresponding to A second image data points, and determine the second image data with the highest image quality as the target image data.

[0228] This application provides a data processing apparatus. When the apparatus reaches the first sampling time step in a sampling period, it acquires the range of sampling time steps corresponding to the sliding window in the sampling period. The sampling period represents N sampling time steps in which the text data is converted into image data by the Wensheng graph model; N is a positive integer. If the first sampling time step is before the sliding window, the first data feature is obtained by performing a first ordinary differential sampling process on the first data feature through the Wensheng graph model. When the first sampling time step is the first sampling time step in the N sampling time steps, the first data feature is the feature of the first text data input to the Wensheng graph model. When the first sampling time step is not the first sampling time step in the N sampling time steps, the first data feature is the feature of the first text data input to the Wensheng graph model. In the first sampling time step, the first data feature is the feature output by the text-to-image model in the previous sampling time step. If the first sampling time step is within a sliding window, the first data feature is processed by random differential sampling through the text-to-image model to obtain the second image feature. When the first sampling time step is the last sampling time step among N sampling time steps, the feature output by the text-to-image model in the first sampling time step is processed by image decoding to obtain the first image data for the first text data. Based on the first image data and the sampling features generated within the sliding window, the parameters of the text-to-image model are adjusted to obtain the target text-to-image model. Through the above process, two sampling techniques, first ordinary differential sampling and random differential sampling, are used in combination during model training. This allows the text-to-image model to be trained by reducing the amount of data to be processed, thereby improving the efficiency of model training and reducing the resource consumption of data processing, through first ordinary differential sampling; and by random differential sampling, the text-to-image model can achieve diversified sampling and random result generation, thereby improving the performance of model training. Meanwhile, the sampling time step range for random differential sampling is determined based on the sliding window, and only the sampling features generated within the sliding window are introduced into the parameter adjustment process of the Wensheng image model, while the sampling features generated outside the sliding window are not processed. This can effectively reduce the data processing overhead of model training and improve the efficiency of model training.

[0229] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 10As shown, the computer device in this embodiment may include one or more processors 1001, a memory 1002, and an input / output interface 1003. The processor 1001, memory 1002, and input / output interface 1003 are connected via a bus 1004. The memory 1002 stores a computer program, which includes program instructions. The input / output interface 1003 receives and outputs data, such as for data interaction between the host machine and the computer device, or for data interaction between various virtual machines within the host machine. The processor 1001 executes the program instructions stored in the memory 1002.

[0230] The processor 1001 can perform the following operations:

[0231] When the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained; the sampling period is used to represent the N sampling time steps by which the text-image model converts text data into image data; N is a positive integer;

[0232] If the first sampling time step is before the sliding window, the first data feature is processed by the text image model using the first ordinary differential sampling to obtain the first image feature. When the first sampling time step is the first sampling time step among N sampling time steps, the first data feature is the feature of the first text data input to the text image model. When the first sampling time step is not the first sampling time step, the first data feature is the feature output by the text image model in the previous sampling time step.

[0233] If the first sampling time step is within the sliding window, the first data features are randomly differentiated and sampled using the Wensheng image model to obtain the second image features;

[0234] When the first sampling time step is the last sampling time step among N sampling time steps, the features output by the text-to-image model at the first sampling time step are processed by image decoding to obtain the first image data for the first text data.

[0235] Based on the first image data and the sampling features generated within the sliding window, the parameters of the text image model are adjusted to obtain the target text image model.

[0236] In some feasible implementations, the processor 1001 may be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0237] The memory 1002 may include read-only memory and random access memory, and provides instructions and data to the processor 1001 and the input / output interface 1003. A portion of the memory 1002 may also include non-volatile random access memory. For example, the memory 1002 may also store device type information.

[0238] In practice, the computer device can perform actions such as these through its built-in functional modules. Figure 3 For details on the implementation methods provided for each step, please refer to [the relevant documentation / document / etc.]. Figure 3 The implementation methods provided for each step are not elaborated here.

[0239] This application provides a computer device including a processor, an input / output interface, and a memory. The processor retrieves a computer program from the memory and executes it. Figure 3Each step of the method shown performs data processing operations. This embodiment of the application implements obtaining the range of sampling time steps corresponding to the sliding window within the sampling period when the first sampling time step in the sampling period is reached; the sampling period is used to represent N sampling time steps by which the text data is converted into image data by the Wensheng graph model; N is a positive integer; if the first sampling time step is before the sliding window, the first data feature is obtained by performing first ordinary differential sampling processing on the first data feature through the Wensheng graph model; when the first sampling time step is the first sampling time step among the N sampling time steps, the first data feature is the feature of the first text data input to the Wensheng graph model; when the first sampling time step is not the first sampling... During the step-by-step process, the first data feature is the feature output by the text-based image model in the previous sampling time step. If the first sampling time step is within a sliding window, the first data feature is processed by random differential sampling through the text-based image model to obtain the second image feature. When the first sampling time step is the last sampling time step among N sampling time steps, the feature output by the text-based image model in the first sampling time step is processed by image decoding to obtain the first image data for the first text data. Based on the first image data and the sampling features generated within the sliding window, the parameters of the text-based image model are adjusted to obtain the target text-based image model. Through the above process, two sampling techniques, first ordinary differential sampling and random differential sampling, are used in combination during model training. This allows the text-based image model to be trained by reducing the amount of data to be processed, thereby improving the efficiency of model training and reducing the resource consumption of data processing, through first ordinary differential sampling; and by random differential sampling, the text-based image model can achieve diversified sampling and random result generation, thereby improving the performance of model training. Meanwhile, the sampling time step range for random differential sampling is determined based on the sliding window, and only the sampling features generated within the sliding window are introduced into the parameter adjustment process of the Wensheng image model, while the sampling features generated outside the sliding window are not processed. This can effectively reduce the data processing overhead of model training and improve the efficiency of model training.

[0240] This application also provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor. Figure 3 For details on the data processing methods provided in each step, please refer to the document. Figure 3The implementation methods provided for each step are not repeated here. Furthermore, the beneficial effects of using the same method are also not repeated. For technical details not disclosed in the computer-readable storage medium embodiments involved in this application, please refer to the description of the method embodiments of this application. As an example, a computer program may be deployed to execute on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.

[0241] The computer-readable storage medium can be the data processing apparatus provided in any of the foregoing embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0242] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform... Figure 3 The method provided among the various optional approaches achieves a hybrid use of first ordinary differential sampling and random differential sampling during model training. This allows for both reducing the amount of data required for training the textural graph model through first ordinary differential sampling, thereby improving training efficiency and reducing data processing resource consumption; and enabling diversified sampling through random differential sampling, allowing for the generation of random results and thus improving training performance. Furthermore, by determining the sampling time step range for random differential sampling based on a sliding window, and only incorporating the sampling features generated within the sliding window into the parameter adjustment process of the textural graph model, while ignoring sampling features generated outside the sliding window, the data processing overhead of model training can be effectively reduced, improving training efficiency.

[0243] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0244] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0245] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0246] The methods and related apparatuses provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0247] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0248] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0249] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, The method includes: When the first sampling time step in the sampling period is reached, the range of sampling time steps corresponding to the sliding window in the sampling period is obtained; the sampling period is used to represent N sampling time steps in which the text-to-image model converts text data into image data; N is a positive integer. If the first sampling time step is before the sliding window, the first data feature is processed by the text image model using the first ordinary differential sampling to obtain the first image feature; when the first sampling time step is the first sampling time step among the N sampling time steps, the first data feature is the feature of the first text data input to the text image model; when the first sampling time step is not the first sampling time step, the first data feature is the feature output by the text image model in the previous sampling time step. If the first sampling time step is within the sliding window, then the first data feature is subjected to random differential sampling processing through the text image model to obtain the second image feature; When the first sampling time step is the last sampling time step among the N sampling time steps, the features output by the text image model at the first sampling time step are processed by image decoding to obtain the first image data for the first text data. Based on the first image data and the sampling features generated within the sliding window, the parameters of the text image model are adjusted to obtain the target text image model.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the starting sampling time step within the sampling time step range corresponding to the sliding window in the sampling period, and obtain the fixed number of window iterations corresponding to the starting sampling time step; the fixed number of window iterations is used to represent the number of loop iterations S required for the Wensheng graph model to be executed when the starting position of the sliding window is at the starting sampling time step; one loop iteration includes one or more of the sampling periods; S is a positive integer; During the process where the starting position of the sliding window is at the starting sampling time step, the number of times the loop iteration has been performed is counted, which is taken as the number of times the window has looped. The step of adjusting the parameters of the raw image model based on the first image data and the sampling features generated within the sliding window to obtain the target raw image model includes: Based on the first image data and the sampling features generated within the sliding window, the parameters of the text image model are adjusted to obtain a candidate text image model; If the candidate text image model does not meet the model convergence condition, then the sampling time step range corresponding to the sliding window in the next iteration cycle is determined based on the fixed number of times the window has been fixed and the number of times the window has been iterated; the iteration cycle refers to a cycle containing S iterations. If the candidate text-to-image model satisfies the model convergence condition, then the candidate text-to-image model is determined as the target text-to-image model.

3. The method according to claim 2, characterized in that, The step of determining the sampling time step range corresponding to the sliding window in the next iteration cycle based on the fixed number of times the window has been used and the number of times the window has been iterated includes: If the number of times the window has been circulated is less than the fixed number of times the window has been circulated, then the sampling time step range corresponding to the sliding window remains unchanged, and the number of times the window has been circulated is accumulated. If the number of times the window has been looped is equal to the number of times the window has been fixed, then the sliding window is slid-processed to obtain the sampling time step range corresponding to the sliding window in the next loop iteration cycle. The number of times the window has been looped is initialized. Based on the starting sampling time step in the sampling time step range corresponding to the sliding window in the next loop iteration cycle, the number of times the window has been fixed in the next loop iteration cycle is determined.

4. The method according to claim 3, characterized in that, The step of determining the fixed number of times the window corresponds to the next iteration cycle based on the starting sampling time step within the sampling time step range corresponding to the sliding window in the next iteration cycle includes: Obtain the moving attenuation parameter, and perform exponential attenuation processing on the moving attenuation parameter according to the starting sampling time step in the sampling time step range corresponding to the sliding window in the next cycle iteration period to obtain the round attenuation coefficient corresponding to the next cycle iteration period. The initial fixed number of iterations is adjusted using the round decay coefficient to obtain the fixed number of iterations for the next cycle.

5. The method according to claim 2, characterized in that, One iteration consists of M sampling periods, and the text-based image model is used to generate a first image data in one sampling period; M is a positive integer; The step of adjusting the parameters of the raw image model based on the first image data and the sampling features generated within the sliding window to obtain a candidate raw image model includes: Obtain the i-th first image data; i is a positive integer less than or equal to M; Based on the sampling features generated by the i-th first image data within the sliding window, the i-th first loss data is generated. The i-th first loss data is used to adjust the parameters of the intermediate text-to-image model to obtain the i-th adjusted text-to-image model. If i is 1, the intermediate text-to-image model is the text-to-image model. If i is greater than 1, the intermediate text-to-image model is the (i-1)-th adjusted text-to-image model corresponding to the (i-1)-th first image data. When the adjusted text-to-image models corresponding to M first image data are obtained, second loss data is generated according to the image quality corresponding to the M first image data. The parameters of the Mth adjusted text-to-image model are adjusted using the second loss data to obtain the candidate text-to-image model.

6. The method according to claim 1, characterized in that, The method further includes: If the first sampling time step is located after the sliding window, the features output by the previous sampling time step of the first sampling time step are processed by the Wensheng graph model to obtain the third image features through the second ordinary differential sampling process.

7. The method according to claim 6, characterized in that, If the first sampling time step is after the sliding window, then the features output by the previous sampling time step of the first sampling time step are subjected to second ordinary differential sampling processing through the text image model to obtain the third image features, including: If the first sampling time step is after the sliding window, then the device performance of the computer device is obtained; the computer device is the device that processes the first sampling time step. If the device performance falls within the first performance range, the first ordinary differential sampling is determined as the second ordinary differential sampling. The features output by the previous sampling time step of the first sampling time step are processed by the Wensheng image model to obtain the third image features. If the device performance data falls within the second performance range, then the higher-order ordinary differential sampling is determined as the second ordinary differential sampling. The features output by the previous sampling time step of the first sampling time step are processed by the Wensheng image model using the second ordinary differential sampling to obtain the third image features. The minimum performance value of the second performance range is greater than the maximum performance value of the first performance range.

8. The method according to claim 1, characterized in that, The step of adjusting the parameters of the raw image model based on the first image data and the sampling features generated within the sliding window to obtain the target raw image model includes: Obtain the model sampling parameters corresponding to the text image model, use the model sampling parameters to perform random differential sampling processing on the sampling features generated in the sliding window to obtain random differential features, and generate first loss data based on the sampling features generated in the sliding window and the random differential features; The first image data is used to generate the second loss data. The first loss data and the second loss data are used to adjust the parameters of the text image model to obtain the target text image model.

9. The method according to claim 8, characterized in that, The model sampling parameters include historical sampling parameters; the process of obtaining the model sampling parameters corresponding to the text image model, using the model sampling parameters to perform random differential sampling processing on the sampling features generated within the sliding window to obtain random differential features, and generating first loss data based on the sampling features generated within the sliding window and the random differential features, includes: Obtain the historical sampling parameters from the historical textual graph model corresponding to the textual graph model; the textual graph model is obtained by adjusting the parameters of the historical textual graph model. Using the historical sampling parameters, the sampling features generated within the sliding window are subjected to the random differential sampling process to obtain random differential features; both the sampling features generated within the sliding window and the random differential features are probability distributions. The probability distribution ratio between the sampled features generated within the sliding window and the random differential features is determined as the parameter optimization coefficient; Based on the first image data, the parameter dominance values ​​of the sampling parameters in the text image model are determined, and the parameter dominance values ​​are adjusted using the parameter optimization coefficients to obtain the first loss data.

10. The method according to claim 1, characterized in that, The method further includes: The second text data is input into the target text-to-image model, and the target text-to-image model is used to perform feature sampling processing on the second text data to obtain the target image features. The target image features are decoded to obtain target image data.

11. The method according to claim 10, characterized in that, The step of performing feature sampling processing on the second text data using the target text-to-image model to obtain target image features includes: The device performance of the computer device running the target text image model is obtained, and a target feature sampler is selected based on the device performance; the target text image model includes multiple feature samplers, the multiple feature samplers include the same sampling parameters, and different feature samplers use different sampling methods; the multiple feature samplers include the target feature sampler; The target image features are obtained by performing feature sampling processing on the second text data using the target feature sampler.

12. The method according to claim 10, characterized in that, The number of features in the target image is A, where A is a positive integer; The process of decoding the target image features to obtain target image data includes: Decode each of the A target image features to obtain the second image data corresponding to each of the A target image features; Obtain the image quality corresponding to A second image data points, and determine the second image data with the highest image quality as the target image data.

13. A data processing apparatus, characterized in that, The device includes: The window processing module is used to obtain the range of sampling time steps corresponding to the sliding window in the sampling period when the first sampling time step in the sampling period is executed; the sampling period is used to represent N sampling time steps in which the text-to-image model converts text data into image data; N is a positive integer; The feature sampling module is used to perform first ordinary differential sampling processing on the first data features through the text image model to obtain first image features if the first sampling time step is before the sliding window; when the first sampling time step is the first sampling time step among the N sampling time steps, the first data features are the features of the first text data input to the text image model; when the first sampling time step is not the first sampling time step, the first data features are the features output by the text image model in the previous sampling time step. The feature sampling module is further configured to perform random differential sampling on the first data features through the text image model to obtain the second image features if the first sampling time step is located within the sliding window; The image decoding module is used to perform image decoding processing on the features output by the text image model at the first sampling time step when the first sampling time step is the last sampling time step among the N sampling time steps, so as to obtain the first image data for the first text data. The parameter adjustment module is used to adjust the parameters of the text image model based on the first image data and the sampling features generated within the sliding window to obtain the target text image model.

14. A computer device, characterized in that, Includes processor, memory, and input / output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

16. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method described in any one of claims 1-12.

Images