Image text recognition method and device based on diffusion model and computer equipment

Abstract

The invention relates to an image text recognition method and device based on a diffusion model and computer equipment, and the method comprises the steps: carrying out the reconstruction of an input image through a trained complete diffusion model based on the visual features of the input image, and obtaining a corresponding target image; the training of the diffusion model comprises the following steps: acquiring a training data set; sample images in the training data set correspond to different image distortion conditions; training a diffusion model based on each sample image to obtain a completely trained diffusion model; a diffusion model in the training process is used for performing forward diffusion on each sample image to obtain a noise image, and a real image is reconstructed through learning and gradual denoising to realize generation of a target image; and performing text recognition on the target image to obtain a text character sequence in the target image. By means of the text recognition method and device, the problem that accurate text recognition cannot be conducted in a complex scene is solved, and accurate text recognition in the complex scene is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image text recognition method, device and computer equipment based on a diffusion model. Background Art

[0002] Image text recognition technology is widely used in many fields. For example, in autonomous driving scenarios, image text recognition technology is used to identify text information in traffic signs and signals to help the autonomous driving system make decisions; in visual question-answering systems, image text recognition technology is used to extract text information in images as a basis for answer generation and reasoning.

[0003] In existing image text recognition methods, feature extraction based on convolutional neural networks is usually used, and image enhancement, deblurring, super-resolution and other technologies are combined to improve recognition accuracy. However, this method performs poorly when dealing with complex scenes such as text distortion, low resolution, and blurred background, and cannot perform accurate text recognition in complex scenes.

[0004] Currently, no effective solution has been proposed to address the problem that related technologies cannot accurately recognize text in complex scenarios. Summary of the invention

[0005] In this embodiment, a diffusion model-based image text recognition method, apparatus, and computer device are provided to solve the problem in the related art that accurate text recognition cannot be performed in complex scenarios.

[0006] In a first aspect, a method for image text recognition based on a diffusion model is provided in this embodiment, and the method includes:

[0007] Extracting features from an input image to obtain visual features of the input image;

[0008] Based on the visual features, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; the diffusion model is trained based on each sample image to obtain a fully trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image;

[0009] Perform text recognition on the target image to obtain a text character sequence in the target image.

[0010] In some embodiments, the training of the diffusion model based on each of the sample images includes:

[0011] Based on the visual features of each of the sample images, forward diffusion and reverse diffusion are performed on the sample images through the diffusion model to obtain a corresponding first standard image;

[0012] Constructing a target loss function according to the difference between the first standard image and the preset standard image;

[0013] The diffusion model is trained based on the target loss function.

[0014] In some embodiments, forward diffusion of each sample image to obtain a corresponding noise image includes:

[0015] A preset noise injection mechanism is used to add noise to the sample image to obtain the corresponding noise image.

[0016] In some of the embodiments, the preset noise injection mechanism adopts Bernoulli distribution.

[0017] In some embodiments, reconstructing the input image based on the visual features by training a complete diffusion model to obtain a corresponding target image includes:

[0018] Determining a feature vector corresponding to the visual feature;

[0019] Based on the feature vector, the input image is reconstructed by training the complete diffusion model to obtain the corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

[0020] In some of the embodiments, after determining the feature vector corresponding to the visual feature, the method further includes:

[0021] Extracting semantic information from the input image;

[0022] Based on the feature vector and the semantic information, the input image is reconstructed by training the complete diffusion model to obtain the corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

[0023] In some embodiments, performing text recognition on the target image to obtain a text character sequence in the target image includes:

[0024] Inputting the target image into a pre-trained text recognition model for processing to obtain a text character sequence in the target image;

[0025] Among them, the text recognition model adopts an encoder-decoder architecture; the encoder is used to convert the image features of the target image into a corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation.

[0026] In a second aspect, in this embodiment, a device for image text recognition based on a diffusion model is provided, the device comprising: an extraction module, a processing module and a recognition module;

[0027] The extraction module is used to extract features from the input image to obtain visual features of the input image;

[0028] The processing module is used to reconstruct the input image based on the visual features through a well-trained diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; the diffusion model is trained based on each sample image to obtain a well-trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image;

[0029] The recognition module is used to perform text recognition on the target image to obtain a text character sequence in the target image.

[0030] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the diffusion model-based image text recognition method described in the first aspect is implemented.

[0031] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the image text recognition method based on the diffusion model described in the first aspect is implemented.

[0032] Compared with the related art, the image text recognition method, device and computer equipment based on the diffusion model provided in this embodiment obtain the visual features of the input image by extracting features from the input image; based on the visual features, the input image is reconstructed by training a complete diffusion model to obtain the corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion situation; the diffusion model is trained based on each sample image to obtain a fully trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image; further, text recognition is performed on the target image to obtain a text character sequence in the target image, which solves the problem of being unable to perform accurate text recognition in complex scenes and realizes accurate text recognition in complex scenes.

[0033] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0035] Figure 1 It is a hardware structure block diagram of a terminal device of an image text recognition method based on a diffusion model provided in an embodiment of the present application;

[0036] Figure 2 is a flow chart of an image text recognition method based on a diffusion model provided by an embodiment of the present application;

[0037] Figure 3 is a schematic diagram of the effect of reverse diffusion provided by an embodiment of the present application;

[0038] Figure 4 is a schematic diagram of the forward diffusion and reverse diffusion process provided by an embodiment of the present application;

[0039] Figure 5 is a flow chart of an image text recognition method based on a diffusion model provided by a preferred embodiment of the present application;

[0040] Figure 6 It is a structural block diagram of an image text recognition device based on a diffusion model provided in one embodiment of the present application.

[0041] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, extraction module; 20, processing module; 30, recognition module. DETAILED DESCRIPTION

[0042] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0044] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure block diagram of a terminal of the method for image text recognition based on a diffusion model of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0045] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image text recognition method based on the diffusion model in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0047] In this embodiment, a method for image text recognition based on a diffusion model is provided. Figure 2 is a flow chart of the image text recognition method based on the diffusion model of this embodiment. Figure 2 As shown, the process includes the following steps:

[0048] Step S210, extracting features from the input image to obtain visual features of the input image;

[0049] Step S220, based on visual features, the input image is reconstructed by a well-trained diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; the diffusion model is trained based on each sample image to obtain a well-trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image;

[0050] Step S230, performing text recognition on the target image to obtain a text character sequence in the target image.

[0051] Specifically, the pre-trained visual encoder extracts features from the input image to obtain visual features of the input image, so as to capture important detail information in the input image, such as edge, shape, color, size, font, distortion, etc. of the text in the input image. In this embodiment, the visual encoder may adopt a convolutional neural network, a vision transformer (ViT), etc.

[0052] Furthermore, the extracted visual features are converted into deep feature vectors, which contain rich visual information and provide a basis for subsequent noise removal and recognition processes. Based on the converted feature vectors, the input image is reconstructed by training a complete diffusion model, that is, the real image is gradually denoised and reconstructed to achieve the generation of the target image, such as Figure 3 As shown, the diffusion model gradually eliminates the noise and distortion in the input image through multiple iterations, such as removing blur and correcting the distortion of characters in the image, so as to gradually restore a target image that is close to the real one.

[0053] When training the above diffusion model, a training data set is obtained in advance. Each sample image in the training data set corresponds to a different image distortion condition. The image distortion condition includes but is not limited to text distortion and low resolution, and each sample image may contain multiple distortion conditions, which is helpful for subsequent unified processing of multiple distortion combinations and significantly improves the recognition effect in complex scenes. The diffusion model is trained based on each sample image to obtain a fully trained diffusion model. The diffusion model in the training process is used to forward diffuse each sample image to obtain the corresponding noise image, and reverse diffuse the noise image. Through repeated iterative training, the diffusion model learns to recover standardized text images from noise images in complex distortion scenarios. Figure 4 As shown, the forward diffusion process refers to gradually adding noise to the sample image, adding a small amount of noise to the sample image at each step until the image is degraded into a completely random noise image, while the reverse diffusion process refers to gradually removing image noise, restoring a clear and standardized target image from random noise, so as to eliminate image noise and correct distortion problems such as text distortion and blur. It can be understood that this embodiment can reduce the difference between training data and test data and improve cross-domain recognition performance by standardizing the image generation process, thereby solving the common domain difference problem in text recognition.

[0054] Afterwards, the target image is input into a pre-trained text recognition model for processing, and the text character sequence in the target image is recognized and output. The text recognition model may adopt a deep learning model, a Transformer model, etc. For example, an encoder-decoder architecture is adopted, where the encoder is used to convert the image features of the target image into a corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation.

[0055] In existing image text recognition methods, feature extraction based on convolutional neural networks is usually used, and image enhancement, deblurring, super-resolution and other technologies are combined to improve recognition accuracy. However, this method performs poorly when dealing with complex scenes such as text distortion, low resolution, and blurred background, and cannot perform accurate text recognition in complex scenes.

[0056] Compared with the prior art, the present application performs feature extraction on an input image to obtain visual features of the input image; based on the visual features, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image; the diffusion model is trained as follows: a training data set is obtained; each sample image in the training data set corresponds to a different image distortion condition; the diffusion model is trained based on each sample image to obtain a fully trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reversely diffuse the noise image, that is, each sample image is gradually subjected to noise processing, and a denoising process of restoring an image from noise is learned; further, text recognition is performed on the target image to obtain a text character sequence in the target image. Based on this, by training the diffusion model, the training model can learn to recover standardized text images from noisy images, and use the reverse diffusion process of the diffusion model to restore the distorted text images to clear and standardized text images, reducing the impact of distortion on text recognition, which helps to improve the accuracy of subsequent image text recognition, thereby solving the problem of inability to perform accurate text recognition in complex scenes, and achieving accurate text recognition in complex scenes, significantly improving the text recognition effect in complex distorted scenes, and making it more suitable for natural scenes.

[0057] In some embodiments, the step S220 of training the diffusion model based on each sample image includes the following steps:

[0058] Based on the visual features of each sample image, the sample image is forward diffused and reverse diffused through a diffusion model to obtain a corresponding first standard image;

[0059] Constructing a target loss function according to the difference between the first standard image and the preset standard image;

[0060] The diffusion model is trained based on the target loss function.

[0061] Specifically, the sample image is forward diffused through the diffusion model. During the forward diffusion process, noise is gradually added to the sample image, with a small amount of noise added to the sample image at each step until the image is degraded into a completely random noise image. The sample image is reversely diffused through the diffusion model. During the reverse diffusion process, the image noise is gradually removed. Based on the visual features of the sample image, a clear and standardized image, namely the first standard image, is restored from the random noise, so that the restored image matches the visual features, while eliminating image noise and correcting distortion problems such as text distortion and blur.

[0062] Furthermore, according to the difference between the first standard image and the preset standard image, a target loss function is constructed, and the diffusion model is trained based on the target loss function. The target loss function may adopt a binary cross entropy loss (BCE), a mean squared error loss (MSE), or the like.

[0063] Through this embodiment, based on the visual features of each sample image, the sample image is forward diffused and reverse diffused through the diffusion model to obtain the corresponding first standard image, and according to the difference between the first standard image and the preset standard image, the target loss function is constructed, and based on the target loss function, the diffusion model is trained so that the diffusion model learns to recover the standardized text image from the noisy image.

[0064] In some embodiments, the step S220 of forward diffusing each sample image to obtain a corresponding noise image includes the following steps:

[0065] A preset noise injection mechanism is used to add noise to the sample image to obtain the corresponding noise image.

[0066] Specifically, a preset noise injection mechanism is used to gradually add noise to the sample image, and a small amount of noise is added to the sample image at each step to degrade the image into a completely random noise image. Among them, a statistical distribution such as Gaussian distribution, Bernoulli distribution, uniform distribution, etc. can be used as the noise injection mechanism.

[0067] Among them, Bernoulli distribution is preferably used as the noise injection mechanism. Bernoulli distribution is more suitable for processing binary text features, such as character contours and boundaries in black and white images. The application of Bernoulli distribution makes the addition and removal of noise more accurate, which helps to improve the training efficiency of the model.

[0068] Through this embodiment, a preset noise injection mechanism is used to add noise to the sample image to obtain a corresponding noise image, thereby realizing the degradation process of the sample image.

[0069] In some embodiments, the step S220 reconstructs the input image based on the visual features by training a complete diffusion model to obtain a corresponding target image, including the following steps:

[0070] Determine the feature vector corresponding to the visual feature;

[0071] Based on the feature vector, the input image is reconstructed by training a complete diffusion model to obtain the corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

[0072] Specifically, the extracted visual features are converted into deep feature vectors. Based on the feature vectors, the input image is reconstructed by training a complete diffusion model. The diffusion model gradually eliminates the noise and distortion in the input image through multiple iterations, such as removing blur and correcting character distortion in the image, thereby gradually restoring a target image that is close to the real one.

[0073] It should be noted that since the feature vector contains rich visual information, reverse diffusion based on the feature vector can ensure that the target image matches the feature vector, thereby reconstructing the visual details and achieving high-quality image restoration.

[0074] Through this embodiment, the feature vector corresponding to the visual feature is determined, and based on the feature vector, the input image is reconstructed by training a complete diffusion model, and the noise and distortion in the input image are removed to obtain the corresponding target image. In this way, the complex and distorted text image is automatically restored to a standard text image by utilizing the inverse degradation process, thereby significantly reducing the impact of distortion and noise on subsequent text recognition.

[0075] In some of the embodiments, after determining the feature vector corresponding to the visual feature, the method further comprises the following steps:

[0076] Extract semantic information from input images;

[0077] Based on the feature vector and semantic information, the input image is reconstructed by training a complete diffusion model to obtain the corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

[0078] Specifically, optical character recognition, deep learning models and other methods are used to extract semantic information from the input image. In the reverse diffusion process of the diffusion model, the input image is reversely diffused in combination with the feature vector and the semantic information, and the target image that matches the feature vector and conforms to the semantic information is gradually restored, while effectively removing noise and distortion in the input image.

[0079] It is understandable that by fusing visual features and semantic information to guide the reverse diffusion process, not only can visual details be reconstructed, but also the text content in the restored target image is consistent with the original text. For example, when the text in the input image is severely distorted due to perspective problems, the diffusion model can automatically correct the distortion and restore the standard text form, which helps to improve the robustness and accuracy of subsequent text recognition.

[0080] Through this embodiment, semantic information in the input image is extracted, and based on the feature vector and the semantic information, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image, thereby removing noise and distortion in the input image and making the target image easier to recognize.

[0081] In some embodiments, performing text recognition on the target image in step S230 to obtain a text character sequence in the target image includes the following steps:

[0082] Input the target image into the pre-trained text recognition model for processing to obtain the text character sequence in the target image;

[0083] Among them, the text recognition model adopts an encoder-decoder architecture; the encoder is used to convert the image features of the target image into the corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation.

[0084] Specifically, the target image is input into a pre-trained text recognition model for processing, and the text character sequence in the target image is recognized. The text recognition model may adopt a deep learning model, a Transformer model, etc. Exemplarily, an encoder-decoder architecture is used for text recognition, where the encoder is used to convert the image features of the target image into a corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation.

[0085] In the training process of the text recognition model, a corresponding loss function is constructed, and the loss function is used to optimize the matching degree between the recognized text character sequence and the standard character sequence to ensure that the recognized text character sequence is accurate.

[0086] Through this embodiment, the target image is input into a pre-trained text recognition model for processing to obtain a text character sequence in the target image, wherein the text recognition model can adopt an encoder-decoder architecture, where the encoder is used to convert the image features of the target image into a corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation, so as to improve the accuracy of image text recognition.

[0087] The present embodiment is described and illustrated below through preferred embodiments.

[0088] Figure 5 is a flow chart of the image text recognition method based on the diffusion model of the preferred embodiment. Figure 5 As shown, the image text recognition method based on the diffusion model includes the following steps:

[0089] Step S510, extracting features of the input image using a pre-trained visual encoder to obtain visual features of the input image;

[0090] Step S520, obtaining a training data set; each sample image in the training data set corresponds to a different image distortion condition;

[0091] Step S530, training the diffusion model based on each sample image to obtain a fully trained diffusion model; the diffusion model in the training process is used to perform forward diffusion on each sample image to obtain a corresponding noise image, and reverse diffusion on the noise image;

[0092] Step S540, based on the visual features and the semantic information in the input image, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image;

[0093] Step S550: input the target image into a pre-trained text recognition model for processing to obtain a text character sequence in the target image.

[0094] Through this embodiment, the input image is feature extracted by a pre-trained visual encoder to obtain the visual features of the input image. A training data set is obtained, in which each sample image corresponds to a different image distortion situation, and a diffusion model is trained based on each sample image to obtain a fully trained diffusion model. During the training process, the diffusion model is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image so that the diffusion model learns to recover a standardized text image from the noise image.

[0095] Furthermore, based on the visual features and semantic information in the input image, the input image is reconstructed by training a complete diffusion model to obtain the corresponding target image, which is then input into a pre-trained text recognition model for processing to obtain the text character sequence in the target image. This solves the problem of inability to perform accurate text recognition in complex scenes and achieves accurate text recognition in complex scenes.

[0096] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0097] In this embodiment, a diffusion model-based image text recognition device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0098] Figure 6 is a structural block diagram of the image text recognition device based on the diffusion model of this embodiment. Figure 6 As shown, the device includes: an extraction module 10, a processing module 20 and an identification module 30;

[0099] The extraction module 10 is used to extract features from the input image to obtain visual features of the input image;

[0100] The processing module 20 is used to reconstruct the input image based on the visual features through a well-trained diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; the diffusion model is trained based on each sample image to obtain a well-trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image;

[0101] The recognition module 30 is used to perform text recognition on the target image to obtain a text character sequence in the target image.

[0102] Through the device provided in this embodiment, feature extraction is performed on the input image to obtain the visual features of the input image; based on the visual features, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: a training data set is obtained; each sample image in the training data set corresponds to a different image distortion situation; the diffusion model is trained based on each sample image to obtain a fully trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image; further, text recognition is performed on the target image to obtain a text character sequence in the target image, which solves the problem of being unable to perform accurate text recognition in complex scenes and realizes accurate text recognition in complex scenes.

[0103] In some of the embodiments, the processing module 20 is also used to perform forward diffusion and reverse diffusion on the sample images through a diffusion model based on the visual features of each sample image to obtain a corresponding first standard image; construct a target loss function based on the difference between the first standard image and the preset standard image; and train the diffusion model based on the target loss function.

[0104] In some of the embodiments, the processing module 20 is further configured to add noise to the sample image using a preset noise injection mechanism to obtain a corresponding noise image.

[0105] In some of the embodiments, the processing module 20 is further used to determine a feature vector corresponding to the visual feature; based on the feature vector, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

[0106] In some of the embodiments, the processing module 20 is also used to extract semantic information from the input image; based on the feature vector and the semantic information, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

[0107] In some of the embodiments, the recognition module 30 is further used to input the target image into a pre-trained text recognition model for processing to obtain a text character sequence in the target image; wherein the text recognition model adopts an encoder-decoder architecture; the encoder is used to convert the image features of the target image into a corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation.

[0108] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0109] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0110] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0111] S1, extracting features from the input image to obtain visual features of the input image;

[0112] S2, based on visual features, reconstruct the input image by training a complete diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; training the diffusion model based on each sample image to obtain a trained complete diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image;

[0113] S3, performing text recognition on the target image to obtain a text character sequence in the target image.

[0114] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0115] In addition, in combination with the image text recognition method based on the diffusion model provided in the above embodiment, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by the processor, any one of the image text recognition methods based on the diffusion model in the above embodiment is implemented.

[0116] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0117] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0118] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0119] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. An image text recognition method based on a diffusion model, characterized in that: The method comprises: Extracting features from an input image to obtain visual features of the input image; Based on the visual features, the input image is reconstructed by training a complete diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; the diffusion model is trained based on each sample image to obtain a fully trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image; Perform text recognition on the target image to obtain a text character sequence in the target image.

2. The image text recognition method based on diffusion model according to claim 1 is characterized in that: The training of the diffusion model based on each of the sample images includes: Based on the visual features of each of the sample images, forward diffusion and reverse diffusion are performed on the sample images through the diffusion model to obtain a corresponding first standard image; Constructing a target loss function according to the difference between the first standard image and the preset standard image; The diffusion model is trained based on the target loss function.

3. The image text recognition method based on diffusion model according to claim 1 is characterized in that: The forward diffusion of each sample image to obtain a corresponding noise image includes: A preset noise injection mechanism is used to add noise to the sample image to obtain the corresponding noise image.

4. The image text recognition method based on diffusion model according to claim 3 is characterized in that: The preset noise injection mechanism adopts Bernoulli distribution.

5. The image text recognition method based on diffusion model according to claim 1 is characterized in that: The step of reconstructing the input image based on the visual features by training a complete diffusion model to obtain a corresponding target image includes: Determining a feature vector corresponding to the visual feature; Based on the feature vector, the input image is reconstructed by training the complete diffusion model to obtain the corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

6. The image text recognition method based on diffusion model according to claim 5 is characterized in that: After determining the feature vector corresponding to the visual feature, the method further includes: Extracting semantic information from the input image; Based on the feature vector and the semantic information, the input image is reconstructed by training the complete diffusion model to obtain the corresponding target image; the reconstruction is used to remove noise and distortion in the input image.

7. The image text recognition method based on diffusion model according to claim 1 is characterized in that: The performing text recognition on the target image to obtain a text character sequence in the target image includes: Inputting the target image into a pre-trained text recognition model for processing to obtain a text character sequence in the target image; Among them, the text recognition model adopts an encoder-decoder architecture; the encoder is used to convert the image features of the target image into a corresponding intermediate representation, and the decoder is used to generate a text character sequence corresponding to the intermediate representation.

8. An image text recognition device based on a diffusion model, characterized in that: The device comprises: an extraction module, a processing module and a recognition module; The extraction module is used to extract features from the input image to obtain visual features of the input image; The processing module is used to reconstruct the input image based on the visual features through a well-trained diffusion model to obtain a corresponding target image; the training of the diffusion model is as follows: obtaining a training data set; each sample image in the training data set corresponds to a different image distortion; the diffusion model is trained based on each sample image to obtain a well-trained diffusion model; the diffusion model in the training process is used to forward diffuse each sample image to obtain a corresponding noise image, and reverse diffuse the noise image; The recognition module is used to perform text recognition on the target image to obtain a text character sequence in the target image.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the image text recognition method based on the diffusion model according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image text recognition method based on a diffusion model according to any one of claims 1 to 7 are implemented.

Images