Image processing method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the field of artificial intelligence and the field of financial science and technology, and relates to an artificial intelligence-based image processing method, which comprises the steps of receiving a noise image input by a user; obtaining time step length information and a category label corresponding to the noise image; calling a pre-constructed image processing model; performing prediction processing on the noise image, the time step length information and the category label based on the image processing model to obtain a corresponding noise prediction result; performing denoising processing on the noise image based on the noise prediction result until a preset generation quality condition is met, and obtaining a corresponding target image; and sending the target image to the user. The invention further provides an image processing device based on artificial intelligence, computer equipment and a storage medium. In addition, the target image can be stored in the block chain. The image processing efficiency of the noise image is improved, and the image quality of the generated target image is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence development technology and financial technology, and in particular to an image processing method, apparatus, computer equipment and storage medium based on artificial intelligence. Background Art

[0002] With the rapid development of financial technology, the financial industry such as banks and securities has become increasingly dependent on image data. These image data include but are not limited to customer identity authentication, transaction credentials, market monitoring and other aspects. Their quality and processing efficiency directly affect the accuracy and security of financial services. However, in practical applications, due to various reasons such as equipment aging, environmental interference, data transmission errors, etc., the generated image data often contains varying degrees of noise, which brings huge challenges to subsequent image processing, feature extraction and data analysis.

[0003] At present, the financial industry mainly relies on traditional image processing technologies such as filtering, edge detection, sharpening, contrast adjustment, etc. when processing noisy image data. Although these technologies can improve image quality to a certain extent, they often have problems of low processing efficiency and limited quality improvement. For example, although spatial domain filtering methods such as mean filtering and median filtering can effectively remove noise, they will also cause blurring of image details and affect image quality; and although frequency domain filtering methods such as Gaussian filtering can retain more image details, their computational complexity is high and it is difficult to meet the requirements of the financial industry for processing efficiency. Therefore, the existing image generation methods in the banking, securities and other industries have problems of low processing efficiency and low quality when processing noisy images. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose an image processing method, device, computer equipment and storage medium based on artificial intelligence to solve the technical problems of low processing efficiency and low quality in existing image generation methods when processing noisy images.

[0005] In order to solve the above technical problems, the embodiment of the present application provides an image processing method based on artificial intelligence, which adopts the following technical solutions:

[0006] Receive a noise image input by a user;

[0007] Obtaining time step information and category labels corresponding to the noise image;

[0008] Calling a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model;

[0009] Based on the image processing model, prediction processing is performed on the noise image, the time step information, and the category label to obtain a corresponding noise prediction result;

[0010] Based on the noise prediction result, the noise image is denoised until it meets a preset generation quality condition, and a corresponding target image is obtained;

[0011] The target image is sent to the user.

[0012] Furthermore, before the step of calling the pre-built image processing model, the method further includes:

[0013] Call the original Transformer model;

[0014] Replacing the standard layer norm layer in the original Transformer model based on a preset adaptive layer norm to obtain a corresponding first Transformer model;

[0015] Add a multi-head cross attention layer at the first specified position in the first Transformer model to obtain a corresponding second Transformer model;

[0016] Adding a linear decoder at a second specified position in the second Transformer model to obtain a corresponding third Transformer model;

[0017] The third Transformer model is used as the target Transformer model.

[0018] Furthermore, before the step of calling the pre-built image processing model, the method further includes:

[0019] Acquiring pre-collected image data;

[0020] Preprocessing the image data to obtain corresponding sample data;

[0021] Dividing the sample data into a training image set and a test image set;

[0022] Obtain the training strategy corresponding to the target loss function and the target optimization algorithm;

[0023] Based on the training strategy, the target Transformer model is trained using the training image set to obtain a corresponding first model;

[0024] Performing model evaluation on the first model based on the test image set;

[0025] If the first model passes the model evaluation, the first model is used as the image processing model.

[0026] Furthermore, the step of performing model evaluation on the first model based on the test image set specifically includes:

[0027] Inputting the test image set into the first model for performance testing, and obtaining a performance index value of the first model; wherein the performance index value is a numerical value corresponding to a preset performance index, and the number of the performance indexes includes a plurality;

[0028] Determine whether all the performance indicator values ​​are greater than the corresponding performance indicator thresholds;

[0029] If so, determining that the first model passes the model evaluation;

[0030] If not, it is determined that the first model fails the model evaluation.

[0031] Furthermore, after the step of performing model evaluation on the first model based on the test image set, the method further includes:

[0032] If the first model fails the model evaluation, obtaining a preset model adjustment strategy;

[0033] Get the preset training optimization strategy;

[0034] Based on the model adjustment strategy and the training optimization strategy, the first model is improved until a third model that meets the construction expectations is obtained;

[0035] The third model is used as the image processing model.

[0036] Furthermore, after the step of using the first model as the image processing model, the method further includes:

[0037] Get the preset model deployment format;

[0038] Performing model export processing on the image processing model based on the model deployment format to obtain a corresponding target image processing model;

[0039] The target image processing model is integrated and stored.

[0040] Furthermore, the step of sending the target image to the user specifically includes:

[0041] Obtaining user information of the user;

[0042] Acquire a preferred push mode corresponding to the user based on the user information;

[0043] Based on the preferred push mode, the target image is sent to the user.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides an image processing device based on artificial intelligence, which adopts the following technical solution:

[0045] A receiving module, used for receiving a noise image input by a user;

[0046] A first acquisition module, used to acquire time step information and category labels corresponding to the noise image;

[0047] A first calling module is used to call a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model;

[0048] A prediction module, used for performing prediction processing on the noise image, the time step information and the category label based on the image processing model to obtain a corresponding noise prediction result;

[0049] A denoising module, used for denoising the noisy image based on the noise prediction result until it meets a preset generation quality condition and obtains a corresponding target image;

[0050] A sending module is used to send the target image to the user.

[0051] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0052] Receive a noise image input by a user;

[0053] Obtaining time step information and category labels corresponding to the noise image;

[0054] Calling a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model;

[0055] Based on the image processing model, prediction processing is performed on the noise image, the time step information, and the category label to obtain a corresponding noise prediction result;

[0056] Based on the noise prediction result, the noise image is denoised until it meets a preset generation quality condition, and a corresponding target image is obtained;

[0057] The target image is sent to the user.

[0058] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0059] Receive a noise image input by a user;

[0060] Obtaining time step information and category labels corresponding to the noise image;

[0061] Calling a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model;

[0062] Based on the image processing model, prediction processing is performed on the noise image, the time step information, and the category label to obtain a corresponding noise prediction result;

[0063] Based on the noise prediction result, the noise image is denoised until it meets a preset generation quality condition, and a corresponding target image is obtained;

[0064] The target image is sent to the user.

[0065] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0066] The present application first receives a noise image input by a user; then obtains the time step information and category label corresponding to the noise image; and then calls a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained after performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model; subsequently, based on the image processing model, the noise image, the time step information and the category label are predicted and processed to obtain a corresponding noise prediction result; further, based on the noise prediction result, the noise image is denoised until it meets the preset generation quality conditions, and a corresponding target image is obtained; finally, the target image is sent to the user. The present application receives a noise image input by a user, obtains the time step information and category label corresponding to the noise image, and then predicts the noise image, the time step information and the category label based on the use of an image processing model to obtain a noise prediction result. Subsequently, based on the noise prediction result, the noise image can be quickly and intelligently denoised until it meets the preset generation quality conditions and obtains the corresponding target image, effectively improving the image processing efficiency of the noise image and the image quality of the obtained target image. Furthermore, by returning the target image to the user, it is beneficial to improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0069] Figure 2 A flowchart of an embodiment of an artificial intelligence-based image processing method according to the present application;

[0070] Figure 3 is a schematic structural diagram of an embodiment of an image processing device based on artificial intelligence according to the present application;

[0071] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0073] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0074] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0075] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0076] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0077] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0078] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0079] It should be noted that the artificial intelligence-based image processing method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based image processing device is generally set in the server / terminal device.

[0080] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0081] Continue to refer Figure 2 , showing a flow chart of an embodiment of an artificial intelligence-based image processing method according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The artificial intelligence-based image processing method provided in the embodiment of the present application can be applied to any scenario where image generation is required, and the artificial intelligence-based image processing method can be applied to products in these scenarios, for example, image generation in the financial insurance field. The artificial intelligence-based image processing method comprises the following steps:

[0082] Step S201: receiving a noise image input by a user.

[0083] In this embodiment, the electronic device (eg, Figure 1The server / terminal device shown in the figure) can obtain the noise image through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. The executor of the present application may be an image processing model, or simply referred to as a model. The above-mentioned noise image is usually of the same size as the image used in training the image processing model, but contains a large amount of random noise. Among them, in the business scenario of noise image processing in financial insurance, the noise image may include low-resolution medical imaging data corresponding to the medical insurance business, such as CT, MRI, etc.

[0084] Step S202, obtaining time step information and category labels corresponding to the noise image.

[0085] In this embodiment, in addition to the noise image, it is also necessary to further obtain the time step of the noise image currently being processed. The time step information is crucial for the model to predict and remove the corresponding noise, because it determines the level and distribution of the noise in the current image. In addition, it is also necessary to obtain the conditional information corresponding to the noise image, that is, the category label. By embedding the category label into the model as an additional input, the conditional information can help the model generate images that better meet user needs.

[0086] Here, the vector embedding of the time step information and the class label is appended to the input sequence as two additional tokens and treated indiscriminately with the image token (noise image). Similar to the cls token in ViTs, the standard ViT block can be used without modification. After the last block, the present invention removes the conditional token from the sequence.

[0087] Step S203, calling a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model.

[0088] In this embodiment, the construction process of the above-mentioned image processing model and the construction process of the above-mentioned target Transformer model will be further described in detail in subsequent specific embodiments, and will not be elaborated in detail here.

[0089] Step S204: performing prediction processing on the noise image, the time step information and the category label based on the image processing model to obtain a corresponding noise prediction result.

[0090] In this embodiment, the above-mentioned prediction processing process based on the image processing model includes: for each time step (starting from the maximum time step and gradually decreasing): inputting the noise image, the current time step and the category label into the trained image processing model, and outputting the noise prediction result of the current time step. Among them, the noise prediction result may include predicted noise and may also include predicted covariance. Specifically, the image token sequence constructed by the noise image, the time step information and the category label can be decoded into a predicted noise and a diagonal covariance prediction (i.e., predicted covariance) through a linear decoder in the image processing model. Both outputs have the same shape as the original spatial input. Specifically, a linear decoder is used to linearly decode each token into a tensor, and then the decoded tokens are rearranged into their original spatial layout to obtain the output predicted noise and predicted covariance.

[0091] Step S205: De-noising the noise image based on the noise prediction result until it meets a preset generation quality condition, and obtains a corresponding target image.

[0092] In this embodiment, after obtaining the noise prediction result of the current time step, the predicted noise is subtracted from the noise image to perform denoising, and the noise image is updated to the denoised result. Alternatively, a more complex update is performed using the predicted noise and the predicted covariance to obtain the denoised result. Thereafter, the time step is gradually reduced and the above denoising process is repeated until the minimum time step is reached or a certain generation quality condition is met, and after all time step iterations are completed, the denoised image obtained is the final generated image, i.e., the above target image.

[0093] Step S206: sending the target image to the user.

[0094] In this embodiment, the specific implementation process of sending the target image to the user will be described in further detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0095] The present application first receives a noise image input by a user; then obtains the time step information and category label corresponding to the noise image; and then calls a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained after performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model; subsequently, based on the image processing model, the noise image, the time step information and the category label are predicted and processed to obtain a corresponding noise prediction result; further, based on the noise prediction result, the noise image is denoised until it meets the preset generation quality conditions, and a corresponding target image is obtained; finally, the target image is sent to the user. The present application receives a noise image input by a user, obtains the time step information and category label corresponding to the noise image, and then predicts the noise image, the time step information and the category label based on the use of an image processing model to obtain a noise prediction result. Subsequently, based on the noise prediction result, the noise image can be quickly and intelligently denoised until it meets the preset generation quality conditions and obtains the corresponding target image, effectively improving the image processing efficiency of the noise image and the image quality of the obtained target image. Furthermore, by returning the target image to the user, it is beneficial to improve the user experience.

[0096] In some optional implementations, before step S203, the electronic device may further perform the following steps:

[0097] Call the original Transformer model.

[0098] In this embodiment, the above-mentioned original Transformer model is a standard Transformer architecture, or a Transformer module. The standard Transformer architecture includes: using multiple Transformer blocks to process the input token sequence. Each block contains a multi-head self-attention mechanism and a feedforward neural network. Among them, the standard Transformer architecture implements a multi-head self-attention mechanism, allowing the model to process the interaction between input tokens. And a feedforward neural network is applied, which includes two linear layers with a ReLU activation function in the middle. And residual connections and layer norms are added.

[0099] In addition, the original Transformer model includes patch embedding and position embedding. Patch embedding: Each patch is converted into a d-dimensional token sequence through linear embedding to form an input token set. Specifically, a linear layer (fully connected layer) is defined to map the original pixel value of each block to a d-dimensional space.

[0100] The standard layer norm layer in the original Transformer model is replaced based on a preset adaptive layer norm to obtain a corresponding first Transformer model.

[0101] In this embodiment, the standard layer norm layer in the original Transformer model can be replaced by an adaptive layer norm layer (adaLN). Instead of learning the dimension scale and translation parameters directly, regression is performed from the sum of the embedding vectors of t (time step) and c (category label). Among the three block designs explored, the adaptive layer norm layer (adaLN) adds the least triggers and is therefore the most computationally efficient.

[0102] Add a multi-head cross attention layer to the first specified position in the first Transformer model to obtain the corresponding second Transformer model.

[0103] In this embodiment, the Transformer module is modified to add an additional multi-head cross attention layer after the multi-head self-attention module to process the interaction between the conditional token and the image token to enhance the model's ability to process conditional information. Cross attention adds the most Gflops (floating point operations) to the model, saving about 15% of the overhead, which can effectively reduce the processing cost of the model.

[0104] A linear decoder is added to the second specified position in the second Transformer model to obtain a corresponding third Transformer model.

[0105] In this embodiment, a standard linear decoder is used to decode the token sequence back to the spatial representation at the end of the Transformer module. The linear decoder outputs the predicted noise and covariance, i.e., the noise prediction result. In addition, the final image noise and covariance prediction can be obtained by rearranging the decoded tokens into their original spatial layout.

[0106] The third Transformer model is used as the target Transformer model.

[0107] The present application calls the original Transformer model; then replaces the standard layer norm layer in the original Transformer model based on a preset adaptive layer norm to obtain a corresponding first Transformer model; then adds a multi-head cross-attention layer at a first designated position in the first Transformer model to obtain a corresponding second Transformer model; and adds a linear decoder at a second designated position in the second Transformer model to obtain a corresponding third Transformer model; and subsequently uses the third Transformer model as the target Transformer model. The present application replaces the standard layer norm layer in the original Transformer model based on a preset adaptive layer norm, and adds a multi-head cross-attention layer and a linear decoder at a designated position in the original Transformer model, so as to intelligently and accurately complete the optimization processing of the original Transformer model and obtain the corresponding target Transformer model, thereby effectively improving the visual recognition ability of the obtained target Transformer model.

[0108] In some optional implementations of this embodiment, before step S203, the electronic device may further perform the following steps:

[0109] Acquire pre-collected image data.

[0110] In this embodiment, a large amount of high-quality image data is collected in advance as basic data for constructing an image processing model.

[0111] The image data is preprocessed to obtain corresponding sample data.

[0112] In this embodiment, the above preprocessing may include resizing, normalization and other processing to convert sample data into a format that can be processed by the model. Specifically, he converts the input image (such as 256x256x3 pixels) into a format that can be processed by the model through the patchify operation, divides the image into multiple small patches (patches), multiple patches are a token sequence, and the size of each patch is determined by the patch size hyperparameter, and uses the converted spatial representation (tensor with a shape of 32x32x4) as the input of the model.

[0113] The sample data is divided into a training image set and a test image set.

[0114] In this embodiment, the sample data may be divided into a training image set and a test image set according to a preset division ratio value, wherein the division ratio value is not specifically limited and may be set according to actual use requirements, for example, may be set to 7:3.

[0115] Get the training strategy corresponding to the target loss function and the target optimization algorithm.

[0116] In this embodiment, the above-mentioned target loss function can specifically adopt L2 loss. The above-mentioned target optimization algorithm can specifically adopt gradient descent algorithm. Correspondingly, the strategy content of the above-mentioned training strategy includes: the training goal is to learn an inverse diffusion process, that is, given a noisy image, the current time step t, and conditional information (category label), the model can predict and remove the noise of the time step, thereby gradually restoring the original image. It is specifically achieved by minimizing the difference between the predicted noise and the real noise (such as L2 loss). The training process includes: for each training image, first generate a series of images with different noise levels through a forward diffusion process. Then, these noisy images and their corresponding time steps and conditional information are used as input to train the target Transformer model to predict the noise of each time step. During the training process, back propagation and gradient descent are used to update the parameters of the model to minimize the loss, thereby completing the model training.

[0117] Based on the training strategy, the target Transformer model is trained using the training image set to obtain a corresponding first model.

[0118] In this embodiment, the target Transformer model is initialized, including Patch embedding: each patch is converted into a token sequence through linear embedding to form an input token sequence. Position embedding: ViT (Vision Transformer) frequency-based position embedding is applied to all input tokens to preserve spatial information. Specifically, a learnable position embedding matrix is ​​created and the position embedding is added to the output of the Patch embedding to preserve spatial information. Similar to the cls token in ViTs, the standard ViT block can be used without modification. After the last block, the conditional token is removed from the sequence.

[0119] In addition, the corresponding noise time steps and category labels are obtained as vector embeddings using different embedding layers and appended as additional tokens to the input token sequence. Specifically, these embedded vectors can be appended as additional tokens to the beginning or end of the input token sequence. Similar to the cls token in ViTs, the standard ViT block can be used without modification. After the last block, the conditional token is removed from the input token sequence (i.e., the embeddings of the noise time steps and category labels are spliced ​​into a sequence of length 2).

[0120] Subsequently, the target Transformer model may be trained using the training image set according to the strategy content corresponding to the above training strategy, thereby obtaining a corresponding trained first model.

[0121] A model evaluation is performed on the first model based on the test image set.

[0122] In this embodiment, the specific implementation process of the model evaluation of the first model based on the test image set will be described in further detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0123] If the first model passes the model evaluation, the first model is used as the image processing model.

[0124] The present application obtains pre-collected image data; and pre-processes the image data to obtain corresponding sample data; then divides the sample data into a training image set and a test image set; then obtains a training strategy corresponding to the target loss function and the target optimization algorithm; subsequently, based on the training strategy, uses the training image set to train the target Transformer model to obtain the corresponding first model; further, based on the test image set, the first model is evaluated; if the first model passes the model evaluation, the first model is used as the image processing model. The present application obtains sample data by pre-processing the pre-collected image data, and then, based on the training strategy corresponding to the target loss function and the target optimization algorithm, uses the sample data to train and evaluate the target Transformer model to obtain the final image processing model, which effectively improves the construction efficiency of the image processing model and ensures the image processing effect of the obtained image processing model.

[0125] In some optional implementations, performing model evaluation on the first model based on the test image set comprises the following steps:

[0126] The test image set is input into the first model for performance testing, and a performance index value of the first model is obtained; wherein the performance index value is a numerical value corresponding to a preset performance index, and the number of the performance indexes includes multiple ones.

[0127] In this embodiment, the performance test can be performed by inputting the test image set into the first model, and after the performance test of the first model is completed, the performance indicator value corresponding to the preset performance indicator is recorded. The above performance indicators specifically include FID (Fréchet Inception Distance), Inception Score and other indicators for evaluating the quality of the generated image. Correspondingly, the above performance indicator values ​​include FID value and Inception Score value.

[0128] Determine whether all of the performance indicator values ​​are greater than the corresponding performance indicator thresholds.

[0129] In this embodiment, according to the actual model performance test requirements, a performance indicator threshold corresponding to the normal performance expectation of the model is pre-constructed for each performance indicator. Specifically, a first performance indicator threshold matching the FID indicator and a second performance indicator threshold matching the Inception Score indicator are pre-constructed.

[0130] If so, it is determined that the first model passes the model evaluation.

[0131] In this embodiment, if all the performance indicator values ​​are greater than the corresponding performance indicator thresholds, it indicates that the model effect (FID and Inception Score) of the generated first model has achieved the expected performance effect, and it is determined that the first model has passed the model evaluation.

[0132] If not, it is determined that the first model fails the model evaluation.

[0133] In this embodiment, if at least one performance indicator value among all the performance indicator values ​​is smaller than the corresponding performance indicator threshold, it is determined that the first model fails the model evaluation.

[0134] The present application performs a performance test by inputting the test image set into the first model and obtains the performance index value of the first model; wherein the performance index value is a numerical value corresponding to a preset performance index, and the number of the performance indexes includes multiple values; then determines whether all the performance index values ​​are greater than the corresponding performance index threshold; if so, determines that the first model has passed the model evaluation; if not, determines that the first model has not passed the model evaluation. The present application performs a model evaluation on the obtained performance index value of the first model corresponding to the performance index based on the use of the performance index threshold, and then can accurately generate a model evaluation result corresponding to the first model, effectively ensuring the accuracy of the obtained model evaluation result.

[0135] In some optional implementations, after the step of performing model evaluation on the first model based on the test image set, the electronic device may further perform the following steps:

[0136] If the first model fails the model evaluation, a preset model adjustment strategy is obtained.

[0137] In this embodiment, the above-mentioned model adjustment strategy may include a strategy for adjusting model parameters or a strategy for adjusting model structure. Among them, the strategy for adjusting model parameters includes: adjusting the learning rate, the number of Transformer blocks, the embedding dimension, etc. to optimize the quality and efficiency of the model-generated image. The strategy for adjusting the model structure includes: adding / removing layers, changing the attention mechanism, etc.

[0138] Get the preset training optimization strategy.

[0139] In this embodiment, the above-mentioned training optimization strategy may include: using data enhancement strategy, optimizer, etc. to optimize the quality and efficiency of the model to generate images.

[0140] The first model is improved based on the model adjustment strategy and the training optimization strategy until a third model that meets the construction expectations is obtained.

[0141] In this embodiment, the first model can be improved according to the policy content of the above-mentioned model adjustment strategy and the above-mentioned training optimization strategy to obtain a model that passes the model evaluation, that is, a third model that meets the construction expectations.

[0142] The third model is used as the image processing model.

[0143] If the present application detects that the first model has not passed the model evaluation, a preset model adjustment strategy is obtained; and a preset training optimization strategy is obtained; then the first model is improved based on the model adjustment strategy and the training optimization strategy until a third model that meets the construction expectations is obtained; and the third model is subsequently used as the image processing model. After detecting that the first model has not passed the model evaluation, the present application will intelligently improve the first model based on the use of the model adjustment strategy and the training optimization strategy, so as to obtain an image processing model that meets the construction expectations, effectively optimizing the quality and efficiency of the image generated by the image processing model, and ensuring the model effect of the generated image processing model.

[0144] In some optional implementations of this embodiment, after the step of using the first model as the image processing model, the electronic device may further perform the following steps:

[0145] Get the preset model deployment format.

[0146] In this embodiment, there is no specific limitation on the selection of the above-mentioned model deployment format, which can be set according to the actual model deployment requirements, for example, it can include deployable formats such as .pth, .h5, etc.

[0147] The image processing model is exported based on the model deployment format to obtain a corresponding target image processing model.

[0148] In this embodiment, the image processing model is exported into a format corresponding to the model deployment format, thereby obtaining a corresponding target image processing model.

[0149] The target image processing model is integrated and stored.

[0150] In this embodiment, the target image processing model can be stored by using a preset storage method. The above storage method is not specifically limited, for example, local database storage, blockchain storage, cloud server storage, etc. can be used. In addition, the target image processing model can be integrated into actual application scenarios, such as image processing software, mobile applications, etc., so as to complete the integration of the target image processing model.

[0151] This application obtains a preset model deployment format; then, based on the model deployment format, the image processing model is exported to obtain a corresponding target image processing model; and the target image processing model is subsequently integrated and stored. After completing the construction of the image processing model, this application will also intelligently export the image processing model based on the acquired model deployment format to obtain a corresponding target image processing model, and then store the target image processing model to ensure the security of the generated target image processing model. In addition, the target image processing model will also be integrated so that the target image processing model can be smoothly applied in actual application scenarios.

[0152] In some optional implementations of this embodiment, step S206 includes the following steps:

[0153] Get user information of the user.

[0154] In this embodiment, the user information may refer to the user's name.

[0155] A preferred push mode corresponding to the user is acquired based on the user information.

[0156] In this embodiment, the user database can be queried according to the user information to find the preferred push method matching the user information from the user database. The preferred push method refers to the user's preferred method of receiving push data, such as email push, interface display, MMS push, etc.

[0157] Based on the preferred push mode, the target image is sent to the user.

[0158] In this embodiment, the target image is sent to the user by adopting a preferred push method that meets the user's preferences, which is beneficial to improving the user's usage experience.

[0159] The present application obtains the user information of the user; then obtains the preferred push mode corresponding to the user based on the user information; and then sends the target image to the user based on the preferred push mode. The present application obtains the preferred push mode corresponding to the user based on the user information of the user, and then sends the target image to the user based on the preferred push mode, thereby effectively improving the push intelligence of the target image and helping to improve the user experience.

[0160] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0161] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0162] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0163] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target image, the above-mentioned target image can also be stored in a node of a blockchain.

[0164] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0165] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0166] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0168] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0169] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of an image processing device based on artificial intelligence. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0170] like Figure 3As shown, the artificial intelligence-based image processing device 300 described in this embodiment includes: a receiving module 301, a first acquisition module 302, a first calling module 303, a prediction module 304, a denoising module 305 and a sending module 306. Among them:

[0171] The receiving module 301 is used to receive a noise image input by a user;

[0172] A first acquisition module 302, used to acquire time step information and category labels corresponding to the noise image;

[0173] The first calling module 303 is used to call a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model;

[0174] A prediction module 304 is used to perform prediction processing on the noise image, the time step information and the category label based on the image processing model to obtain a corresponding noise prediction result;

[0175] A denoising module 305 is used to perform denoising processing on the noise image based on the noise prediction result until the preset generation quality condition is met, and obtain a corresponding target image;

[0176] The sending module 306 is configured to send the target image to the user.

[0177] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0178] In some optional implementations of this embodiment, the image processing device based on artificial intelligence further includes:

[0179] A second calling module, used to call the original Transformer model;

[0180] A replacement module, used to replace the standard layer norm layer in the original Transformer model based on a preset adaptive layer norm to obtain a corresponding first Transformer model;

[0181] A first adding module is used to add a multi-head cross attention layer at a first specified position in the first Transformer model to obtain a corresponding second Transformer model;

[0182] A second adding module, used for adding a linear decoder at a second specified position in the second Transformer model to obtain a corresponding third Transformer model;

[0183] The first determination module is used to use the third Transformer model as the target Transformer model.

[0184] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0185] In some optional implementations of this embodiment, the image processing device based on artificial intelligence further includes:

[0186] A second acquisition module is used to acquire pre-collected image data;

[0187] A preprocessing module, used for preprocessing the image data to obtain corresponding sample data;

[0188] A division module, used for dividing the sample data into a training image set and a test image set;

[0189] The third acquisition module is used to obtain a training strategy corresponding to the target loss function and the target optimization algorithm;

[0190] A training module, configured to train the target Transformer model using the training image set based on the training strategy to obtain a corresponding first model;

[0191] An evaluation module, configured to perform model evaluation on the first model based on the test image set;

[0192] The second determination module is used to use the first model as the image processing model if the first model passes the model evaluation.

[0193] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0194] In some optional implementations of this embodiment, the evaluation module includes:

[0195] A testing submodule, used for inputting the test image set into the first model for performance testing, and obtaining a performance index value of the first model; wherein the performance index value is a numerical value corresponding to a preset performance index, and the number of the performance indexes includes a plurality;

[0196] A judgment submodule, used to judge whether all the performance indicator values ​​are greater than the corresponding performance indicator threshold;

[0197] A first determination submodule, configured to determine that if so, the first model passes the model evaluation;

[0198] The second determination submodule is used to determine if the first model fails the model evaluation.

[0199] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0200] In some optional implementations of this embodiment, the image processing device based on artificial intelligence further includes:

[0201] A fourth acquisition module, used for acquiring a preset model adjustment strategy if the first model fails the model evaluation;

[0202] A fifth acquisition module is used to acquire a preset training optimization strategy;

[0203] An improvement module, used for improving the first model based on the model adjustment strategy and the training optimization strategy, until a third model that meets the construction expectations is obtained;

[0204] The third determining module is used to use the third model as the image processing model.

[0205] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0206] In some optional implementations of this embodiment, the image processing device based on artificial intelligence further includes:

[0207] A sixth acquisition module, used to acquire a preset model deployment format;

[0208] An export module, used to perform model export processing on the image processing model based on the model deployment format to obtain a corresponding target image processing model;

[0209] The processing module is used to integrate and store the target image processing model.

[0210] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0211] In some optional implementations of this embodiment, the sending module 306 includes:

[0212] A first acquisition submodule is used to acquire user information of the user;

[0213] A second acquisition submodule is used to acquire a preferred push mode corresponding to the user based on the user information;

[0214] The sending submodule is used to send the target image to the user based on the preferred push mode.

[0215] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based image processing method in the aforementioned implementation, and will not be repeated here.

[0216] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0217] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0218] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0219] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of an image processing method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0220] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the artificial intelligence-based image processing method.

[0221] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0222] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0223] In the embodiment of the present application, by receiving the noise image input by the user, and obtaining the time step information and category label corresponding to the noise image, and then predicting the noise image, the time step information and the category label based on the use of the image processing model to obtain the noise prediction result, the noise image can be quickly and intelligently denoised based on the noise prediction result until it meets the preset generation quality conditions and obtains the corresponding target image, which effectively improves the image processing efficiency of the noise image and effectively improves the image quality of the obtained noise image. Then, by returning the target image to the user, it is beneficial to improve the user experience.

[0224] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based image processing method.

[0225] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0226] In the embodiment of the present application, by receiving the noise image input by the user, and obtaining the time step information and category label corresponding to the noise image, and then predicting the noise image, the time step information and the category label based on the use of the image processing model to obtain the noise prediction result, the noise image can be quickly and intelligently denoised based on the noise prediction result until it meets the preset generation quality conditions and obtains the corresponding target image, which effectively improves the image processing efficiency of the noise image and effectively improves the image quality of the obtained noise image. Then, by returning the target image to the user, it is beneficial to improve the user experience.

[0227] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0228] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An image processing method based on artificial intelligence, characterized in that: The steps include: Receive a noise image input by a user; Obtaining time step information and category labels corresponding to the noise image; Calling a pre-built image processing model; wherein the image processing model is obtained by training an optimized target Transformer model using preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model; Based on the image processing model, prediction processing is performed on the noise image, the time step information, and the category label to obtain a corresponding noise prediction result; Based on the noise prediction result, the noise image is denoised until it meets a preset generation quality condition, and a corresponding target image is obtained; The target image is sent to the user.

2. The image processing method based on artificial intelligence according to claim 1, characterized in that: Before the step of calling the pre-built image processing model, the method further includes: Calling the original Transformer model; Replacing the standard layer norm layer in the original Transformer model based on a preset adaptive layer norm to obtain a corresponding first Transformer model; Add a multi-head cross attention layer at the first specified position in the first Transformer model to obtain the corresponding second Transformer model; Add a linear decoder at a second specified position in the second Transformer model to obtain a corresponding third Transformer model; The third Transformer model is used as the target Transformer model.

3. The image processing method based on artificial intelligence according to claim 1 or 2, characterized in that: Before the step of calling the pre-built image processing model, the method further includes: Acquiring pre-collected image data; Preprocessing the image data to obtain corresponding sample data; Dividing the sample data into a training image set and a test image set; Obtain the training strategy corresponding to the target loss function and the target optimization algorithm; Based on the training strategy, the target Transformer model is trained using the training image set to obtain a corresponding first model; Performing model evaluation on the first model based on the test image set; If the first model passes the model evaluation, the first model is used as the image processing model.

4. The image processing method based on artificial intelligence according to claim 3, characterized in that: The step of performing model evaluation on the first model based on the test image set specifically includes: Inputting the test image set into the first model for performance testing, and obtaining a performance index value of the first model; wherein the performance index value is a numerical value corresponding to a preset performance index, and the number of the performance indexes includes a plurality; Determine whether all the performance indicator values ​​are greater than the corresponding performance indicator thresholds; If so, determining that the first model passes the model evaluation; If not, it is determined that the first model fails the model evaluation.

5. The image processing method based on artificial intelligence according to claim 3, characterized in that: After the step of performing model evaluation on the first model based on the test image set, the method further includes: If the first model fails the model evaluation, obtaining a preset model adjustment strategy; Get the preset training optimization strategy; Based on the model adjustment strategy and the training optimization strategy, the first model is improved until a third model that meets the construction expectations is obtained; The third model is used as the image processing model.

6. The image processing method based on artificial intelligence according to claim 3, characterized in that: After the step of using the first model as the image processing model, the method further includes: Get the preset model deployment format; Performing model export processing on the image processing model based on the model deployment format to obtain a corresponding target image processing model; The target image processing model is integrated and stored.

7. The image processing method based on artificial intelligence according to claim 1, characterized in that: The step of sending the target image to the user specifically includes: Obtaining user information of the user; Acquire a preferred push mode corresponding to the user based on the user information; Based on the preferred push mode, the target image is sent to the user.

8. An image processing device based on artificial intelligence, characterized in that: include: A receiving module, used for receiving a noise image input by a user; A first acquisition module, used to acquire time step information and category labels corresponding to the noise image; The first calling module is used to call a pre-built image processing model; wherein the image processing model is obtained by training the optimized target Transformer model with preset sample data, and the target Transformer model is a model obtained by performing standard layer norm layer replacement processing and preset layer addition processing on the original Transformer model; A prediction module, used for performing prediction processing on the noise image, the time step information and the category label based on the image processing model to obtain a corresponding noise prediction result; A denoising module, used for denoising the noisy image based on the noise prediction result until it meets a preset generation quality condition and obtains a corresponding target image; A sending module is used to send the target image to the user.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the artificial intelligence-based image processing method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based image processing method according to any one of claims 1 to 7.

Images