Image processing method and device

Abstract

The present application discloses an image processing method, comprising: obtaining an image to be processed including characters; inputting the image to be processed into the character recognition model to obtain the characters included in the image to be processed; wherein: the character recognition model is used to: extract a first feature of the image to be processed; classify the first feature to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature; weight the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature to obtain a second feature, wherein the weight of the features of the corresponding background noise in the first feature is less than 1 and greater than or equal to 0; according to the second feature, obtain the characters included in the image to be processed. Compared with the first feature, the second feature weakens the features of the corresponding background noise; therefore, the characters in the image to be processed obtained according to the second feature are more accurate.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and device. Background Art

[0002] In some scenarios, it is necessary to recognize characters in an image. However, the current methods for recognizing characters in an image cannot accurately recognize the characters in the image.

[0003] Therefore, there is an urgent need for a solution that can accurately recognize characters in images. Summary of the invention

[0004] The technical problem to be solved by the present application is how to accurately recognize characters in an image, and to provide an image processing method and device.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Acquire an image to be processed including characters;

[0007] The image to be processed is input into the character recognition model to obtain the characters included in the image to be processed; wherein:

[0008] The character recognition model is used for:

[0009] Extracting a first feature of the image to be processed;

[0010] Classifying the first features to determine features corresponding to characters in the first features and features corresponding to background noise in the first features;

[0011] Weighting a feature corresponding to the character in the first feature and a feature corresponding to the background noise in the first feature to obtain a second feature, wherein the weight of the feature corresponding to the background noise in the first feature is less than 1 and greater than or equal to 0;

[0012] According to the second feature, characters included in the image to be processed are obtained.

[0013] Optionally, the character recognition model includes N encoders, a feature classification processing module and a decoder;

[0014] The first i encoders of the N encoders are connected in series, and the first i encoders are used to obtain the first feature according to the image to be processed, and the first feature is the output of the i-th encoder, where i is a positive integer less than N;

[0015] The feature classification processing module is used to classify the first feature to determine the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature; and to weight the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature to obtain the second feature;

[0016] The latter (Ni) encoders are connected in series, and the latter (Ni) encoders are used to process the second feature to obtain the third feature;

[0017] The decoder is used to obtain the character prediction result according to the third feature.

[0018] Optionally, the feature classification processing module includes: an aggregate cross entropy ACE module, and the ACE module is used to classify the first feature to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature.

[0019] Optionally, the feature corresponding to the background noise in the first feature includes a first sub-feature, and the weight of the first sub-feature is determined according to the confidence that the first sub-feature is the feature corresponding to the background noise.

[0020] Optionally, the character recognition model is trained in the following manner:

[0021] Acquire a training image and a label corresponding to the training image, wherein the label corresponding to the training image is used to indicate a character included in the training image;

[0022] Based on the training image and the label corresponding to the training image, a character recognition model is trained, and the character recognition model is used to recognize characters in the image; wherein:

[0023] The step of training a character recognition model based on the training image and the label corresponding to the training image includes:

[0024] Extracting a first feature of the training image;

[0025] Classifying the first feature of the training image to determine a feature of a character corresponding to the first feature of the training image and a feature of a background noise corresponding to the first feature of the training image;

[0026] Weighting a feature of the first feature of the training image corresponding to the character and a feature of the first feature of the training image corresponding to the background noise to obtain a second feature of the training image, wherein the weight of the feature of the first feature of the training image corresponding to the background noise is less than 1 and greater than or equal to 0;

[0027] Obtaining a character prediction result according to the second feature of the training image;

[0028] Based on the character prediction results and the labels corresponding to the training images, the parameters of the character recognition model are updated.

[0029] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0030] An acquisition unit, used for acquiring an image to be processed including characters;

[0031] A processing unit is used to input the image to be processed into the character recognition model to obtain the characters included in the image to be processed; wherein:

[0032] The character recognition model is used for:

[0033] Extracting a first feature of the image to be processed;

[0034] Classifying the first features to determine features corresponding to characters in the first features and features corresponding to background noise in the first features;

[0035] Weighting a feature corresponding to the character in the first feature and a feature corresponding to the background noise in the first feature to obtain a second feature, wherein the weight of the feature corresponding to the background noise in the first feature is less than 1 and greater than or equal to 0;

[0036] According to the second feature, characters included in the image to be processed are obtained.

[0037] Optionally, the character recognition model includes N encoders, a feature classification processing module and a decoder;

[0038] The first i encoders of the N encoders are connected in series, and the first i encoders are used to obtain the first feature according to the image to be processed, and the first feature is the output of the i-th encoder, where i is a positive integer less than N;

[0039] The feature classification processing module is used to classify the first feature to determine the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature; and to weight the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature to obtain the second feature;

[0040] The latter (Ni) encoders are connected in series, and the latter (Ni) encoders are used to process the second feature to obtain the third feature;

[0041] The decoder is used to obtain the character prediction result according to the third feature.

[0042] Optionally, the feature classification processing module includes: an aggregate cross entropy ACE module, and the ACE module is used to classify the first feature to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature.

[0043] Optionally, the feature corresponding to the background noise in the first feature includes a first sub-feature, and the weight of the first sub-feature is determined according to the confidence that the first sub-feature is the feature corresponding to the background noise.

[0044] Optionally, the character recognition model is trained in the following manner:

[0045] Acquire a training image and a label corresponding to the training image, wherein the label corresponding to the training image is used to indicate a character included in the training image;

[0046] Based on the training image and the label corresponding to the training image, a character recognition model is trained, and the character recognition model is used to recognize characters in the image; wherein:

[0047] The step of training a character recognition model based on the training image and the label corresponding to the training image includes:

[0048] Extracting a first feature of the training image;

[0049] Classifying the first feature of the training image to determine a feature of a character corresponding to the first feature of the training image and a feature of a background noise corresponding to the first feature of the training image;

[0050] Weighting a feature of the first feature of the training image corresponding to the character and a feature of the first feature of the training image corresponding to the background noise to obtain a second feature of the training image, wherein the weight of the feature of the first feature of the training image corresponding to the background noise is less than 1 and greater than or equal to 0;

[0051] Obtaining a character prediction result according to the second feature of the training image;

[0052] Based on the character prediction results and the labels corresponding to the training images, the parameters of the character recognition model are updated.

[0053] In a third aspect, an embodiment of the present application provides a device, the device comprising a processor and a memory;

[0054] The processor is configured to execute instructions stored in the memory, so that the device executes any one of the methods described in the first aspect above.

[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, wherein the instructions instruct a device to execute a method as described in any one of the above first aspects.

[0056] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0057] Compared with the prior art, the embodiments of the present application have the following advantages:

[0058] An embodiment of the present application provides an image processing method, comprising: acquiring an image to be processed including characters; inputting the image to be processed into a character recognition model to obtain the characters included in the image to be processed; wherein: the character recognition model is used to: extract a first feature of the image to be processed; classify the first feature to determine features of the first feature corresponding to the characters and features of the first feature corresponding to background noise; weight the features of the first feature corresponding to the characters and the features of the first feature corresponding to the background noise to obtain a second feature, wherein the weight of the features of the first feature corresponding to the background noise is less than 1 and greater than or equal to 0; and obtain the characters included in the image to be processed based on the second feature.

[0059] It can be understood that, compared with the first characteristic, the second characteristic weakens the characteristic corresponding to the background noise; therefore, the characters in the image to be processed obtained according to the second characteristic are more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0062] Figure 2 This is a structural diagram of a character recognition model in an embodiment of the present application;

[0063] Figure 3 A flowchart of an image processing method provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] The inventors of the present application have discovered through research that, currently, a machine learning model can be pre-trained, and the trained machine learning model can be used to recognize images, thereby determining the characters included in the image.

[0067] In one example, the machine learning model can be a Transformer model, which includes an encoder and a decoder. The encoder is used to encode the image to obtain image features, and the decoder is used to decode the features output by the encoder to obtain the characters included in the image.

[0068] The encoder of the Transformer model includes a multi-head attention module, which is used to obtain the global features of the image. Because the multi-head attention module is used to obtain the global features of the image, when there is a lot of background noise in the image, the global features will be greatly affected by the background noise, making the character recognition result inaccurate.

[0069] In order to solve the above problems, the embodiments of the present application provide a model training method and device.

[0070] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0071] Exemplary Methods

[0072] An embodiment of the present application provides an image processing method, in which a pre-trained character recognition model can be used to identify characters in an image to be processed, and when the character recognition model identifies characters in the image to be processed, the influence of background noise can be weakened, so that the character recognition model can accurately identify the characters in the image to be processed.

[0073] Next, the training process of the character recognition model is introduced.

[0074] See also Figure 1 , which is a flow chart of a model training method provided in an embodiment of the present application. In this embodiment, the method can be executed by a terminal or a server, and the embodiment of the present application does not make specific limitations.

[0075] Figure 1 The method shown may, for example, include the following steps: S101 - S102 .

[0076] It should be noted that the model training process is a process of multiple iterative calculations. The model parameters can be adjusted in each iteration, and the adjusted parameters participate in the next round of iterative calculations.

[0077] Figure 1 Taking a training image as an example, a round of iteration process in training the character recognition model is introduced. It can be understood that there are many groups of training images used to train the character recognition model. When training the formula recognition model, each group of training images is processed in a similar manner. After training with multiple groups of training images, a character recognition model with accuracy that meets the requirements can be obtained.

[0078] S101: Acquire a training image and a label corresponding to the training image, where the label corresponding to the training image is used to indicate characters included in the training image.

[0079] In one example, the training image may be an image including characters. The training image may be obtained by photographing a photographing device, or obtained from a network resource, or obtained by other means, which is not specifically limited in the embodiments of the present application.

[0080] In one example, an original image may be obtained, and then the original image may be processed to obtain a training image. The original image may be processed, for example, by changing the size of the original image. In one example, the width and height of the original image may be scaled proportionally so that the height of the processed image is a preset height (for example, 32).

[0081] In one example, the labels corresponding to the training images may be manually annotated.

[0082] S102: Based on the training image and the label corresponding to the training image, a character recognition model is trained, where the character recognition model is used to recognize characters in the image.

[0083] When S102 is specifically implemented, it may include the following S1021-S1025.

[0084] S1021: Extracting a first feature of the training image.

[0085] In one example, the character recognition model may include a feature extraction module, and the feature extraction module is used to extract a first feature of the training image.

[0086] In one example, the feature extraction module may include i encoders, i is an integer greater than or equal to 1, and the first feature is the output of the i-th encoder. When i is greater than 1, the i encoders are connected in series, the first encoder among the i encoders is used to process the training image, and the output of the j-th encoder is the input of the (j+1)-th encoder. Wherein, (j+1) is less than or equal to i. The encoder mentioned here may be a native encoder of a traditional Transformer model, which is not described in detail here.

[0087] S1022: Classify the first feature to determine features of the first feature corresponding to the character and features of the first feature corresponding to the background noise.

[0088] S1023: Weighting the feature of the first feature corresponding to the character and the feature of the first feature corresponding to the background noise to obtain a second feature, wherein the weight of the feature of the first feature corresponding to the background noise is less than 1 and greater than or equal to 0.

[0089] S1024: Obtain a character prediction result based on the second feature.

[0090] In an embodiment of the present application, in order to reduce the influence of background noise on character recognition, the first feature can be classified to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature, and then the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature are weighted to obtain a second feature. It is not difficult to understand that since the weight of the features corresponding to the background noise in the first feature is less than 1 and greater than or equal to 0, the second feature weakens the features corresponding to the background noise compared to the first feature. In other words, compared with the first feature, the second feature filters out some of the features corresponding to the background noise. Furthermore, when the character prediction result is obtained based on the second feature, the influence of the features corresponding to the background noise on the character prediction result is also weakened accordingly.

[0091] Regarding S1022, it should be noted that, in one example, the character recognition model may include a feature classification processing module, which is used to classify the first feature to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature; and weight the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature to obtain the second feature.

[0092] In one example, considering the Aggregation Cross-Entropy (ACE) algorithm, it is possible to identify whether a feature corresponds to a character or a background noise. Therefore, the ACE module can be used to classify the first feature to determine the features of the first feature corresponding to the character and the features of the first feature corresponding to the background noise. In other words, the aforementioned feature classification processing module may include an ACE module. For this case, the feature classification processing module may also include a feature weighting module, which is used to weight the features of the first feature corresponding to the character and the features of the first feature corresponding to the background noise to obtain the second feature.

[0093] Regarding the ACE module, it should be noted that the ACE model can classify features in two-dimensional features to determine whether the feature corresponds to a character or a background noise. In other words, the first feature can be a two-dimensional feature. For example, if the size of the training image is h*w, the first feature can be a h1*w1 feature, where h1 / h can be equal to w1 / w. Of course, h1 / h may not be equal to w1 / w. For example, if the height of the image to be processed is downsampled by 4 times and the width is downsampled by 4 times, then w1 / w=0.25, h1 / h=0.25, that is, h1 / h is equal to w1 / w. For another example, if the height of the image to be processed is downsampled by 4 times and the width is downsampled by 8 times, then w1 / w=0.25, h1 / h=0.125, that is, h1 / h is equal to w1 / w.

[0094] Another advantage of using the ACE module to identify the features of the corresponding characters in the first feature and the features of the corresponding background noise is that for training images with relatively complex content, when identifying the features of the corresponding characters in the first feature and the features of the corresponding background noise, the features corresponding to the characters and the features corresponding to the background noise will not be mixed together in the dimension of height, thereby avoiding the two being mixed together in the dimension of height and affecting the accuracy of recognition.

[0095] For the convenience of description, any feature of the first feature corresponding to the background noise is called a "first sub-feature", and the weight of the first sub-feature is a value less than 1 and greater than or equal to 0. It can be understood that the smaller the weight of the first sub-feature is, the more parts of the first sub-feature are filtered out, and the smaller the impact of the first sub-feature on the effect of character recognition is.

[0096] In one example, the weight of the first sub-feature can be determined based on the confidence that the first sub-feature is a feature corresponding to the background noise. For example, the weight of the first sub-feature can be equal to (1-first confidence), where the first confidence is the confidence that the first sub-feature is a feature corresponding to the background noise. When the first feature is classified using the ACE module, the weight of the first sub-feature can be the probability that the first sub-feature output by the ACE module is a feature corresponding to the background noise. Generally, the probability that the first sub-feature output by the ACE module is a feature corresponding to the background noise is very high, almost close to 1, so the second feature, compared with the first feature, almost completely filters out the feature of the background noise corresponding to the first feature.

[0097] Regarding S1024, it should be noted that, in one example, the character recognition model may include a character recognition module, and the character recognition module is used to obtain a character prediction result according to the second feature. In one example, the character recognition module may be a decoder, and the decoder may be a native decoder of a traditional Transformer model, which is not described in detail here.

[0098] The “obtaining the characters included in the image to be processed according to the second feature” mentioned here can be understood as: processing the third feature obtained by further processing the second feature to obtain the character prediction result.

[0099] In one example, in addition to the i encoders for extracting the first feature, the character recognition model may also include another (Ni) encoder. In other words, the character recognition model may include N encoders. And, when (Ni) is greater than 1, the latter (Ni) encoders are connected in series, and the latter (Ni) encoders are used to process the second feature to obtain the third feature. It can be understood that for the (i+1)th encoder among the N encoders, its input is no longer the first feature output by the i-th encoder, but the second feature output by the feature classification processing module. In other words, the i-th encoder and the (i+1)-th encoder are not directly connected in series, and a feature classification processing module is connected in series between the i-th encoder and the (i+1)-th encoder.

[0100] S1025: Based on the character prediction result and the label corresponding to the training image, update the parameters of the character recognition model.

[0101] Since the label corresponding to the training image is used to indicate the characters included in the training image, and the character prediction result is the character in the training image recognized by the character recognition model, the parameters of the character recognition model can be updated based on the character prediction result and the label corresponding to the training image. In the subsequent training process, the character prediction result of the character recognition model after parameter adjustment can be closer to the label corresponding to the training image.

[0102] Regarding the character recognition model, now combined with Figure 2 Provide explanation. Figure 2 A structural diagram of a character recognition model provided in an embodiment of the present application.

[0103] like Figure 2 As shown, the character recognition model includes i encoders 210, a feature classification processing module 220, (Ni) encoders 230 and a decoder 240. Among them:

[0104] The encoder 210 and the encoder 230 have the same structure.

[0105] The output of the i-th encoder is the first feature, which serves as the input of the feature classification processing module 220.

[0106] The feature classification processing module 220 includes an ACE module 221 and a feature weighting module 222. The ACE module 221 is used to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature. The feature weighting module 222 is used to weight the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature to obtain the second feature.

[0107] (Ni) encoders 230 obtain the third feature based on the second feature.

[0108] The decoder 240 obtains a character prediction result according to the third feature.

[0109] In one example, N=7, i=2.

[0110] After the character recognition model is trained, the character recognition model can be used to recognize characters in an image. In one example, an image to be processed including characters can be obtained, and the image to be processed can be obtained by shooting with a shooting device, or obtained from a network resource, or obtained by other means, which is not specifically limited in the embodiments of the present application.

[0111] After obtaining the image to be processed, the image to be processed can be input into the trained character recognition model, and the character recognition model can output the characters included in the image to be processed. Since the features corresponding to the background noise are weakened when training the character recognition model, the character recognition model can also weaken the features corresponding to the background noise when recognizing the characters in the image to be processed, so that the recognized characters are more accurate.

[0112] Next, the image processing method provided by the embodiment of the present application is introduced. Figure 3 , which is a schematic diagram of the process of an image processing method provided in an embodiment of the present application. The method may include, for example, S301-S302.

[0113] S301: Acquire an image to be processed including characters.

[0114] S302: Input the image to be processed into the character recognition model to obtain the characters included in the image to be processed; wherein: the character recognition model is used to: extract a first feature of the image to be processed; classify the first feature to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature; weight the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature to obtain a second feature, wherein the weight of the features of the first feature corresponding to the background noise is less than 1 and greater than or equal to 0; based on the second feature, obtain the characters included in the image to be processed.

[0115] Regarding the image to be processed, please refer to the relevant description section above, which will not be described in detail here.

[0116] The character recognition model mentioned in S302 refers to the use of Figure 1 The character recognition model is trained using the method shown.

[0117] pass Figure 1 The method shown shows that:

[0118] In one example, the character recognition model includes N encoders, a feature classification processing module and a decoder;

[0119] The first i encoders of the N encoders are connected in series, and the first i encoders are used to obtain the first feature according to the image to be processed, and the first feature is the output of the i-th encoder, where i is a positive integer less than N;

[0120] The feature classification processing module is used to classify the first feature to determine the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature; and to weight the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature to obtain the second feature;

[0121] The latter (Ni) encoders are connected in series, and the latter (Ni) encoders are used to process the second feature to obtain the third feature;

[0122] The decoder is used to obtain the character prediction result according to the third feature.

[0123] In one example, the feature classification processing module includes: an ACE module, and the ACE module is used to classify the first feature to determine features of characters corresponding to the first feature and features of background noise corresponding to the first feature.

[0124] In one example, the feature corresponding to the background noise in the first feature includes a first sub-feature, and the weight of the first sub-feature is determined according to the confidence that the first sub-feature is the feature corresponding to the background noise.

[0125] Exemplary Devices

[0126] Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.

[0127] See also Figure 4 , which is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The device 400 may specifically include: an acquisition unit 401 and a processing unit 402.

[0128] An acquisition unit 401 is used to acquire an image to be processed including characters;

[0129] The processing unit 402 is used to input the image to be processed into the character recognition model to obtain the characters included in the image to be processed; wherein:

[0130] The character recognition model is used for:

[0131] Extracting a first feature of the image to be processed;

[0132] Classifying the first features to determine features corresponding to characters in the first features and features corresponding to background noise in the first features;

[0133] Weighting a feature corresponding to the character in the first feature and a feature corresponding to the background noise in the first feature to obtain a second feature, wherein the weight of the feature corresponding to the background noise in the first feature is less than 1 and greater than or equal to 0;

[0134] According to the second feature, characters included in the image to be processed are obtained.

[0135] Optionally, the character recognition model includes N encoders, a feature classification processing module and a decoder;

[0136] The first i encoders of the N encoders are connected in series, and the first i encoders are used to obtain the first feature according to the image to be processed, and the first feature is the output of the i-th encoder, where i is a positive integer less than N;

[0137] The feature classification processing module is used to classify the first feature to determine the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature; and to weight the feature of the character corresponding to the first feature and the feature of the background noise corresponding to the first feature to obtain the second feature;

[0138] The latter (Ni) encoders are connected in series, and the latter (Ni) encoders are used to process the second feature to obtain the third feature;

[0139] The decoder is used to obtain the character prediction result according to the third feature.

[0140] Optionally, the feature classification processing module includes: an aggregate cross entropy ACE module, and the ACE module is used to classify the first feature to determine the features of the corresponding characters in the first feature and the features of the corresponding background noise in the first feature.

[0141] Optionally, the feature corresponding to the background noise in the first feature includes a first sub-feature, and the weight of the first sub-feature is determined according to the confidence that the first sub-feature is the feature corresponding to the background noise.

[0142] Optionally, the character recognition model is trained in the following manner:

[0143] Acquire a training image and a label corresponding to the training image, wherein the label corresponding to the training image is used to indicate a character included in the training image;

[0144] Based on the training image and the label corresponding to the training image, a character recognition model is trained, and the character recognition model is used to recognize characters in the image; wherein:

[0145] The step of training a character recognition model based on the training image and the label corresponding to the training image includes:

[0146] Extracting a first feature of the training image;

[0147] Classifying the first feature of the training image to determine a feature of a character corresponding to the first feature of the training image and a feature of a background noise corresponding to the first feature of the training image;

[0148] Weighting a feature of the first feature of the training image corresponding to the character and a feature of the first feature of the training image corresponding to the background noise to obtain a second feature of the training image, wherein the weight of the feature of the first feature of the training image corresponding to the background noise is less than 1 and greater than or equal to 0;

[0149] Obtaining a character prediction result according to the second feature of the training image;

[0150] Based on the character prediction results and the labels corresponding to the training images, the parameters of the character recognition model are updated.

[0151] Since the device 400 is a device corresponding to the image processing method provided by the above method embodiment, the specific implementation of each unit of the device 400 is based on the same concept as the image processing method described in the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 400, please refer to the relevant description part of the above method embodiment, which will not be repeated here.

[0152] The embodiment of the present application also provides a device, the device comprising a processor and a memory;

[0153] The processor is used to execute the instructions stored in the memory, so that the device executes the image processing method provided by the above method embodiment.

[0154] An embodiment of the present application provides a computer-readable storage medium, including instructions, wherein the instructions instruct a device to execute the image processing method provided by the above method embodiment.

[0155] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the image processing method provided by the above method embodiment.

[0156] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0157] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0158] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed including characters; The image to be processed is input into a character recognition model to obtain characters included in the image to be processed; wherein: The character recognition model is used for: Extracting a first feature of the image to be processed; Classifying the first features to determine features corresponding to characters in the first features and features corresponding to background noise in the first features; Weighting a feature corresponding to the character in the first feature and a feature corresponding to the background noise in the first feature to obtain a second feature, wherein the weight of the feature corresponding to the background noise in the first feature is less than 1 and greater than or equal to 0; According to the second feature, characters included in the image to be processed are obtained.

2. The method according to claim 1, characterized in that The character recognition model includes N encoders, a feature classification processing module and a decoder; The first i encoders of the N encoders are connected in series, and the first i encoders are used to obtain the first feature according to the image to be processed, and the first feature is the output of the i-th encoder, where i is a positive integer less than N; The feature classification processing module is used to classify the first feature to determine the feature of the first feature corresponding to the character and the feature of the first feature corresponding to the background noise; and weight the feature of the first feature corresponding to the character and the feature of the first feature corresponding to the background noise to obtain the second feature; The last (Ni) encoders of the N encoders are connected in series, and the last (Ni) encoders are used to process the second feature to obtain a third feature; The decoder is used to obtain the character prediction result according to the third feature.

3. The method according to claim 2, characterized in that The feature classification processing module includes: an aggregate cross entropy ACE module, and the ACE module is used to classify the first feature to determine the features of the first feature corresponding to the characters and the features of the first feature corresponding to the background noise.

4. The method according to any one of claims 1 to 3, characterized in that: The feature corresponding to the background noise in the first feature includes a first sub-feature, and the weight of the first sub-feature is determined according to the confidence that the first sub-feature is the feature corresponding to the background noise.

5. The method according to claim 1, characterized in that The character recognition model is trained in the following way: Acquire a training image and a label corresponding to the training image, wherein the label corresponding to the training image is used to indicate a character included in the training image; Based on the training image and the label corresponding to the training image, a character recognition model is trained, and the character recognition model is used to recognize characters in the image; wherein: The step of training a character recognition model based on the training image and the label corresponding to the training image includes: Extracting a first feature of the training image; Classifying the first feature of the training image to determine a feature of a character corresponding to the first feature of the training image and a feature of a background noise corresponding to the first feature of the training image; Weighting a feature of the first feature of the training image corresponding to the character and a feature of the first feature of the training image corresponding to the background noise to obtain a second feature of the training image, wherein the weight of the feature of the first feature of the training image corresponding to the background noise is less than 1 and greater than or equal to 0; Obtaining a character prediction result according to the second feature of the training image; Based on the character prediction results and the labels corresponding to the training images, the parameters of the character recognition model are updated.

6. An image processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring an image to be processed including characters; A processing unit is used to input the image to be processed into a character recognition model to obtain the characters included in the image to be processed; wherein: The character recognition model is used for: Extracting a first feature of the image to be processed; Classifying the first features to determine features corresponding to characters in the first features and features corresponding to background noise in the first features; Weighting a feature corresponding to the character in the first feature and a feature corresponding to the background noise in the first feature to obtain a second feature, wherein the weight of the feature corresponding to the background noise in the first feature is less than 1 and greater than or equal to 0; According to the second feature, characters included in the image to be processed are obtained.

7. The device according to claim 6, characterized in that The character recognition model includes N encoders, a feature classification processing module and a decoder; The first i encoders of the N encoders are connected in series, and the first i encoders are used to obtain the first feature according to the image to be processed, and the first feature is the output of the i-th encoder, where i is a positive integer less than N; The feature classification processing module is used to classify the first feature to determine the feature of the first feature corresponding to the character and the feature of the first feature corresponding to the background noise; and weight the feature of the first feature corresponding to the character and the feature of the first feature corresponding to the background noise to obtain the second feature; The last (Ni) encoders of the N encoders are connected in series, and the last (Ni) encoders are used to process the second feature to obtain a third feature; The decoder is used to obtain the character prediction result according to the third feature.

8. The device according to claim 7, characterized in that The feature classification processing module includes: an aggregate cross entropy ACE module, and the ACE module is used to classify the first feature to determine the features of the first feature corresponding to the characters and the features of the first feature corresponding to the background noise.

9. A device, characterized in that: The device comprises a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the device performs the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The method comprises instructions for instructing a device to execute the method according to any one of claims 1 to 5.

Images