Image processing method, device and readable storage medium

Abstract

The present disclosure relates to an image processing method, device and readable storage medium, and relates to the technical field of image processing. The method comprises: correcting a distorted to-be-processed image to obtain a corrected image, the corrected image comprising a segmented first sub-image and a second sub-image; inputting the first sub-image into a local network layer and the second sub-image into a local transition layer, the local network layer being configured to perform a convolution and splicing operation on the first sub-image to obtain a target splicing image, and the local transition layer being configured to perform convolution and splicing on the target splicing image after convolution and the second sub-image to obtain a target image; extracting a relationship between each character in the target image to obtain a sequence feature with a vocabulary sequence; and inputting the sequence feature into a sequence recognition network to obtain a prediction result. The image processing method provided by the present disclosure can improve the recognition accuracy of a distorted text image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to an image processing method, device and readable storage medium. BACKGROUND

[0002] The finger reading learning is an intelligent device based on intelligent hardware such as a learning machine and AI software cooperation, so that the user points the finger in the text area of any book, and the intelligent device can recognize the text in the finger pointing area and assist in displaying the learning resources matched with the text.

[0003] In the finger reading scene, the placement angle of the camera carried by the intelligent device and the lens distortion, and the bending degree of the book itself also interfere with the shooting of the camera, so that the intelligent device often cannot collect the text image with a standard shape, and the text image collected by the intelligent device is severely distorted, and the accuracy of the characters recognized by the intelligent device based on the distorted text image is low. SUMMARY

[0004] To overcome the problems in the related art, the present disclosure provides an image processing method, device and readable storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, and the method comprises:

[0006] correcting the distorted to-be-processed image to obtain a corrected image, wherein the corrected image comprises a first sub-image and a second sub-image after segmentation;

[0007] inputting the first sub-image into a local network layer and the second sub-image into a local transition layer, wherein the local network layer is configured to perform a convolution and splicing operation on the first sub-image to obtain a target spliced image, and the local transition layer is configured to perform a convolution and splicing operation on the target spliced image after convolution and the second sub-image to obtain a target image;

[0008] extracting the relationship between each character in the target image to obtain a sequence feature with a vocabulary sequence;

[0009] inputting the sequence feature into a sequence recognition network to obtain a prediction result.

[0010] Optionally, the local network layer comprises a plurality of network layers, and the target spliced image is obtained through the following steps:

[0011] for each network layer, performing convolution on the first sub-image or the spliced image output by the previous network layer to obtain a convolution image;

[0012] For each of the network layers, the convolutional images in the same network layer are spliced with the first sub-image, or the convolutional images in the same network layer are spliced with the spliced image output by the previous network layer, to obtain the target spliced image.

[0013] Optionally, the sequence recognition network is a network in an image processing model, and the inputting of the sequence feature into the sequence recognition network to obtain the prediction result comprises:

[0014] In the training phase of the image processing model, the sequence feature sample is forward propagated by using a time sequence classification module and an attention mechanism module of the sequence recognition network to obtain a training result;

[0015] In the testing phase of the image processing model, the sequence feature is forward propagated by using the time sequence classification model to obtain the prediction result.

[0016] Optionally, the distorted to-be-processed image is corrected to obtain a corrected image, comprising:

[0017] A set of correction points corresponding to a set of control points of the to-be-processed image is obtained by using the image processing model;

[0018] The corrected image is obtained according to the set of correction points.

[0019] Optionally, the image processing model is obtained by the following steps:

[0020] A first model is obtained by training a to-be-trained model by using a set of scanning image samples;

[0021] The first model is trained according to a first target image sample in the set of scanning image samples and having an image quality less than a first preset condition to obtain a second model;

[0022] The second model is trained according to a second target image sample in the set of scanning image samples and a third target image sample in a set of photographed image samples to obtain a third model;

[0023] The third model is trained according to a fourth target image sample in the set of scanning image samples and having an image quality less than the first preset condition, and a fifth target image sample in the set of photographed image samples and having an image quality less than a second preset condition to obtain the image processing model.

[0024] Optionally, the first model is trained according to the first target image sample in the set of scanning image samples and having an image quality less than the first preset condition to obtain the second model, comprising:

[0025] select, from the scan image sample set, a first target image sample with image quality less than a first preset condition according to a first preset proportion;

[0026] train the first model according to the first target image sample to obtain the second model.

[0027] Optionally, the second model is trained according to a second target image sample in the scan image sample set and a third target image sample in a photograph image sample set to obtain a third model, including:

[0028] select, from the scan image sample set, a second target image sample according to a second preset proportion, and select, from the photograph image sample set, a third target image sample according to a second preset proportion;

[0029] train the second model according to the second target image sample and the third target image sample to obtain the third model.

[0030] Optionally, the third model is trained according to a fourth target image sample in the scan image sample set with image quality less than a first preset condition and a fifth target image sample in the photograph image sample set with image quality less than a second preset condition to obtain the image processing model, including:

[0031] select, from the scan image sample set, a first image sample according to a third preset proportion, and select, from the photograph image sample set, a second image sample according to a third preset proportion;

[0032] select, from a plurality of the first image samples, a fourth target image sample with image quality less than the first preset condition according to a fourth preset proportion, and select, from a plurality of the second image samples, a fifth target image sample with image quality less than a second preset condition according to a fourth preset proportion;

[0033] train the third model according to the fourth target image sample and the fifth target image sample to obtain the image processing model.

[0034] Optionally, the third model is trained according to the fourth target image sample and the fifth target image sample to obtain the image processing model, including:

[0035] obtain a fourth model according to the fourth target image sample and the fifth target image sample;

[0036] select, from the scan image sample set, a third image sample according to a fifth preset proportion, and select, from the photograph image sample set, a fourth image sample according to a fifth preset proportion;

[0037] warping the third image sample and the fourth image sample to obtain warped image samples;

[0038] training the fourth model by using the warped image samples to obtain the image processing model.

[0039] According to a second aspect of the embodiments of the present disclosure, an image processing apparatus is provided, comprising:

[0040] a correction module configured to correct the warped image to be processed to obtain a corrected image, the corrected image comprising a segmented first sub-image and a segmented second sub-image;

[0041] a target image determination module configured to input the first sub-image to a local network layer and input the second sub-image to a local transition layer, the local network layer being configured to perform a convolutional concatenation operation on the first sub-image to obtain a target concatenated image, and the local transition layer being configured to perform a convolutional concatenation operation on the target concatenated image and the second sub-image to obtain a target image;

[0042] a sequence feature determination module configured to extract a relationship between each character in the target image to obtain a sequence feature with a word sequence;

[0043] a prediction result determination module configured to input the sequence feature to a sequence recognition network to obtain a prediction result.

[0044] According to a third aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the steps of the image processing method provided in the first aspect of the present disclosure.

[0045] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects:

[0046] The warped image to be processed can be corrected, so that the intelligent device can accurately recognize characters according to the corrected image, and the accuracy of character recognition is improved. The corrected image is segmented into a first sub-image and a second sub-image, the first sub-image is input to a local network layer, and the second sub-image is input to a local transition layer. Since the data of the first sub-image is less than that of the corrected image, on the one hand, the parameter size of the local network layer can be reduced, and on the other hand, the training difficulty of the local network layer can be reduced, thereby reducing the training difficulty of the image processing model. The dependency relationship between each character in the target image is extracted, and the sequence feature with the dependency relationship is input to a sequence recognition network to extract character information with the dependency relationship from the target image.

[0047] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure, in which, like reference numerals represent similar objects throughout the several views.

[0049] Figure 1 is a flow chart of an image processing method according to an exemplary embodiment.

[0050] Figure 2 is a schematic diagram of a bidirectional BiLSTM according to an exemplary embodiment.

[0051] Figure 3 is a schematic diagram of an image before and after rectification according to an exemplary embodiment.

[0052] Figure 4 is a schematic diagram of a convolutional network according to an exemplary embodiment.

[0053] Figure 5 is a schematic diagram of another convolutional network according to an exemplary embodiment.

[0054] Figure 6 is a schematic diagram of a sequence recognition network according to an exemplary embodiment.

[0055] Figure 7 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0056] Figure 8 is a block diagram of an image processing apparatus according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is merely exemplary and explanatory in nature and is not intended to limit the present disclosure, unless otherwise specified. In other words, the present disclosure is not limited to the exemplary embodiments described herein, but rather, the present disclosure is intended to cover all implementations consistent with the principles of the present disclosure. For example, the first sub-image can be the second sub-image, and the second sub-image can be the first sub-image, which is not limited in the present disclosure.

[0058] It should be noted that all actions of obtaining signals, information or data in this application are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0059] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment, as shown in Figure 1 The image processing method is used in a user equipment, and includes the following steps.

[0060] In step S11, the distorted to-be-processed image is corrected to obtain a corrected image, and the corrected image includes the segmented first sub-image and the second sub-image.

[0061] In some scenarios, there are a large number of texts in common teaching materials such as books, picture books and test papers, and a large number of words in English books. For some users, they may not fully understand these texts and words. Conventional ways of looking up dictionaries, asking parents or searching for these texts and words using search engines often waste a lot of time and increase the learning cost of users.

[0062] In order to enable users to quickly obtain the interpretation related to the text or word, functions of pointing reading learning are derived on intelligent devices such as learning machines, mobile phones and tablets.

[0063] In the pointing reading learning scenario, the user places a book under the intelligent device, the intelligent device automatically detects and recognizes the text content on the page of the book, when the student points to a word or text, the interpretation of the word or text can be seen on the intelligent device, and the intelligent device will match the material library to provide detailed knowledge content of the word or text for the user, such as example sentences of the word or text, current affairs news related to the word or text, etc.

[0064] However, during the process of holding the book or placing the book under the intelligent device, the text or word scanned by the intelligent device is often distorted; or some books are designed to enhance the interest of the text or word, and some texts or words are in the shape of arch or wave, and the text or word is also distorted. When the text content detected by the intelligent device is distorted, the character recognition accuracy of the intelligent device is reduced, and the interpretation of the presented text or word is wrong.

[0065] Therefore, after obtaining the distorted to-be-processed image, the intelligent device in the present disclosure corrects the distorted to-be-processed image to obtain a corrected image, and then identifies accurate characters according to the text in the shape-specified corrected image, to improve the character recognition accuracy.

[0066] The to-be-processed image can be a distorted image, and the corrected image can be a shape-standardized image. The characters in the corrected image are more regular in shape than the characters in the to-be-processed image.

[0067] The intelligent device can obtain a distorted to-be-processed image through the image processing model, correct the distorted to-be-processed image, and obtain a corrected image.

[0068] The first sub-image and the second sub-image are sub-images segmented from the corrected image, and thus the data amount of the first sub-image and the second sub-image is smaller than that of the corrected image.

[0069] In step S12, the first sub-image is input to a local network layer, and the second sub-image is input to a local transition layer. The local network layer is configured to perform a convolution and splicing operation on the first sub-image to obtain a target splicing image. The local transition layer is configured to perform a convolution and splicing operation on the target splicing image after the convolution and the second sub-image to obtain a target image.

[0070] The image processing model has a dense convolution network (DenseNet). The convolution network includes a local network layer (partial dense block) and a local transition layer (partial transition layer). The convolution network is configured to extract visual information from the corrected image. The visual information is extracted from different styles of characters, such as the shape, font, and thickness of the characters. The characters include words, letters, numbers, and the like.

[0071] In the related art, during the training of the image processing model, the entire corrected image sample is directly input to the local network layer of the convolution network. The multiple network layers in the local network layer perform convolution and splicing operations on the corrected image sample. The splicing image sample is output from the local network layer. The splicing image sample is input to the local transition layer. The local transition layer performs convolution on the splicing image sample to obtain visual information of the corrected image sample.

[0072] In this process, the proportion of feature reuse in the convolution network is large, and the parameter size in the convolution network also increases with the convolution and splicing operations. In the process of feature reuse, the previous features are continuously spliced with the subsequent convolution layers, which leads to a large training difficulty of the image processing model.

[0073] In order to reduce the training difficulty of the image processing model, the disclosure divides the corrected image sample into a first sub-image sample and a second sub-image sample, inputs the first sub-image sample into the local network layer, inputs the second sub-image sample into the local transition layer, inputs the target splicing image sample obtained by the convolution splicing operation of the local network layer on the first sub-image sample into the local transition layer after the convolution splicing operation of the local network layer on the first sub-image sample obtains the target splicing image sample; and the local transition layer performs convolution on the target splicing image sample and performs convolution splicing on the second sub-image to obtain the target image sample.

[0074] In this process, since the first sub-image sample input into the local network layer is a sub-image sample divided from the corrected image sample, the data amount is smaller than that of the corrected image sample. Naturally, in the process of inputting the first sub-image sample with relatively less data amount into the local network layer for convolution splicing operation, the parameter scale in the convolution network is relatively small, reducing the training difficulty of the image processing model.

[0075] In the process of training the image processing model, the corrected image sample is divided into a first sub-image sample and a second sub-image sample for training, which can reduce the training difficulty of the image processing model. Naturally, when the image processing model is actually predicted, the corrected image is also divided into a first sub-image and a second sub-image, and the first sub-image is input into the local network layer and the second sub-image is input into the local transition layer, and the actual prediction process is the same as the training process.

[0076] The ratio of dividing the corrected image into the first sub-image and the second sub-image can be a target ratio, which can be 0.5, 0.4, etc., which is not limited in the disclosure.

[0077] In step S13, the relationship between each character in the target image is extracted to obtain a sequence feature with a vocabulary sequence.

[0078] As shown in Figure 2 The image processing model further includes a BiLSTM (Bi-directional Long Short-Term Memory) network layer, which can extract the language logical relationship between each character in the target image to obtain a sequence feature of a vocabulary sequence.

[0079] The vocabulary sequence is a sequence composed of a plurality of words divided from a sentence, and the sequence composed of a plurality of words has a language logical relationship; the sequence feature is a feature corresponding to the vocabulary sequence, which can be recognized by the image processing model.

[0080] The language logical relationship can be understood as a dependency relationship between characters. For example, the character "dirty" in the target image has the character "not good" in the target image. The BiLSTM network layer can determine that "not good" is a modification of the degree of "dirty", and "not good" and "dirty" have a strong language logical relationship.

[0081] The BiLSTM network layer can be a bidirectional BiLSTM network as shown in FIG. 3. Figure 2 The bidirectional BiLSTM network includes a forward LSTM (Long Short-Term Memory), a backward LSTM, and a word2vec layer. The word2vec is represented as word2vec 300d vecter in the bidirectional BiLSTM network. Figure 2 The local network outputs the target image, and the word2vec layer in the BiLSTM network layer converts the multiple characters in the target image into vectors, such as converting "dirty" into a vector of "1, 0, 0, 0". The vectors are then input into the forward LSTM and the backward LSTM, which model the context information between the vectors. In this way, sequence features with dependency relationships are obtained.

[0082] In step S14, the sequence features are input into a sequence recognition network to obtain a prediction result.

[0083] The input of the sequence recognition network is the output of the convolutional network in step S12 and the BiLSTM network layer in step S13. The sequence recognition network mainly recognizes the text content in the corrected image and converts the corrected image into character information.

[0084] For example, referring to FIG. 4, the image at the top is an unprocessed image, and the image at the bottom is a corrected image. The sequence recognition network can extract and convert the character information "a responsible friend" from the corrected image. Figure 3

[0085] ​By the technical solution, the distorted to-be-processed image can be corrected, so that the intelligent device can perform accurate character recognition according to the corrected image, and the accuracy of character recognition is improved. Then, the corrected image is divided into a first sub-image and a second sub-image, the first sub-image is input into the local network layer, and the second sub-image is input into the local transition layer. Since the data of the first sub-image is less than that of the corrected image, on the one hand, the parameter size of the local network layer can be reduced, and on the other hand, the training difficulty of the local network layer can be reduced, and then the training difficulty of the image processing model is reduced. Then, the dependency relationship between each character in the target image is extracted, and the sequence feature with the dependency relationship is input into the sequence recognition network to extract the character information with the dependency relationship from the target image.

[0086] In a possible implementation, the control point set corresponding to the to-be-processed image can be extracted by the image processing model, and the corrected image is obtained according to the control point set.

[0087] The image processing model includes TPS (Thin Plate Spline, thin plate spline function) thin template correction. The TPS thin template correction corrects the control point set predicted by the neural network from an arbitrary shape text line image to obtain a corrected point set, and the corrected point set forms a relatively standard text line image. The control point set is a plurality of position coordinates in the to-be-processed image, and the corrected point set is a plurality of position coordinates in the corrected image.

[0088] For example, in the entire interpolation process, TPS needs to define two terms, one is a fitting term, and the other is a distortion term. The two terms are obtained by the following expression:

[0089]

[0090] In formula (1), ε is a total loss function; is a fitting term; and ε d is a distortion term, used to measure the distortion of the to-be-processed image, is an energy function of surface distortion; λ is a weight coefficient, and the greater the weight coefficient, the closer the deformation of the to-be-processed image to the affine transformation.

[0091] wherein, The expression of is as follows:

[0092]

[0093] In formula (2), is a fitting term, is the distance between the control point after the deformation function and the corrected point; N is the number of control points; p i is a corrected point; and q i is a control point.

[0094] Where, ε d The expression is as follows:

[0095]

[0096] With the aim of minimizing the total loss function of the above formula (1), the following expression can be derived from formula (3):

[0097]

[0098] In formula (4), This represents the calculation formula for inserting control points on a curved surface (the image to be processed) onto a plane (the corrected image). It is a mapping function between control points and correction points, used to calculate the set of correction points on the corrected image; X is any point in the corrected image, X = [a1, a2]. T ;a T =a1,a2]; C is the parameter to be learned; w T =[w1,w2,w3…w N [ ] represents the parameters to be learned; S(X) = [δ(X-X1), δ(X-X2), δ(X-X3)...δ(XX N )] T , is the radial basis kernel function.

[0099] As can be seen from formula (4), according to the mapping function between control points and correction points proposed in formula (4), after inputting the control points on the image to be processed into formula 4, the correction points of the corrected image can be obtained. Then, multiple correction points are combined to form a plane, which is the corrected image.

[0100] The process of converting the control points into correction points is transformed into a neural network result. A Block structure combining Conv2D (convolutional neural network operation functions) and MaxPool (maximum pooling) is used, stacking Block layers to convert the input image (the default input size can be 32 pixels high and any pixels wide) into a 512-dimensional feature vector. Then, through three fully connected layers (FC layers), the resulting 512-dimensional feature vector is multiplied multiple times using a feature dimension of 512-1028-512-40, ultimately yielding a 40-dimensional vector. This 40-dimensional vector represents 20 correction points, each with x and y values.

[0101] Please see Figure 3 As shown, a distorted image to be processed can be corrected into a regular, corrected image.

[0102] By the technical solution, the distorted to-be-processed image can be corrected into a regular corrected image, so that the image processing model can extract more accurate character information according to the regular corrected image.

[0103] In a possible implementation, the local network layer has a plurality of network layers, and the target stitched image is obtained by the following steps: for each network layer, performing convolution on the first sub-image or the stitched image output by the previous network layer to obtain a convolution image; and for each network layer, stitching the convolution image in the same network layer with the first sub-image or stitching the convolution image in the same network layer with the stitched image output by the previous network layer to obtain the target stitched image.

[0104] In the related art, in the process of training the image processing model, the entire to-be-processed image sample is directly input into the local network layer of the convolution network, so as to train the image processing model. However, the proportion of feature reuse in the convolution network is large, and the parameter scale in the convolution network also increases, and in the process of feature reuse, the previous features are continuously stitched with the subsequent convolution layers, resulting in a large training difficulty of the image processing model.

[0105] For example, referring to FIG. 1, Figure 4 As shown in FIG. 1, in the first network layer, the corrected image sample A is first convolved to obtain a convolution image B1, and then the convolution image B1 is stitched with the corrected image sample A to obtain a stitched image C1; in the second network layer, the stitched image C1 is first convolved to obtain a convolution image B2, and then the convolution image B2 is stitched with the stitched image C1 to obtain a stitched image C2…, and the convolution operation and the stitching operation are repeatedly performed in each layer, and the output of each network layer is taken as the convolution input of the next network layer and also participates in the stitching operation of the next network layer. In this process, because the data amount of the to-be-processed image sample is large, after the convolution operation and the stitching operation of multiple layers, the parameter scale in the convolution network gradually increases, resulting in a large training difficulty of the image processing model.

[0106] In order to reduce the training difficulty of the to-be-processed model, the corrected image sample 1 is divided into a first sub-image sample 11 and a second sub-image sample 12 in the present disclosure, the first sub-image sample 11 is input into the first network layer of the local network layer, the local network layer includes a plurality of network layers, and the second sub-image sample 12 is input into the network layer in the local transition layer.

[0107] For example, referring to FIG. 2, Figure 3As shown, in the first layer network layer, the first sub-image sample 11 is convolved to obtain a convolution image D1, and then the convolution image D1 is spliced with the first sub-image sample 11 to obtain a spliced image E1; in the second layer network, the spliced image E1 is convolved to obtain a convolution image D2, and then the spliced image E1 is spliced with the convolution image D2 to obtain a spliced image E2…, the convolution operation and the splicing operation are repeatedly performed in each layer, so that the target spliced image output by the local network layer is obtained through repeated convolution operation and splicing operation; finally, the target spliced image is input to the local transition layer, and after the target spliced image is convolved in the local transition layer, it is spliced with the second sub-image sample, and then convolved to obtain a target image. In this process, since the data amount of the first sub-image sample itself is relatively small compared with the to-be-processed image sample, after the multi-layer convolution operation and splicing operation, the parameter size in the convolution network is smaller compared with the input of the entire to-be-processed image sample, so the training difficulty of the image processing model can be reduced.

[0108] Through the above technical solution, a part of the image in the to-be-processed image is input into the image processing model, which can reduce the data processing amount of the image processing model, so that the image processing model can output the target image faster; and the first sub-image sample is used to train the image processing model, which can also reduce the network parameters brought by repeated convolution operation and splicing operation, and reduce the training difficulty of the image processing model.

[0109] In a possible implementation, the image processing model further includes a sequence recognition network, and the sequence recognition network includes a connectionist temporal classification (CTC) module and an attention mechanism module. In the process of obtaining the prediction result, the CTC module and the attention mechanism module of the sequence recognition network can be used to forward propagate the sequence feature sample in the training stage of the image processing model to obtain a training result; and the CTC module can be used to forward propagate the sequence feature in the test stage of the image processing model to obtain the prediction result.

[0110] In the related art, please refer to Figure 5As shown, in the forward propagation stage, the sequence features output by the local transition layer are first input to the feature extraction module, and the feature extraction module then inputs the sequence features to the parallel CTC network module and Attention network module. The two modules will calculate the final prediction results and the real labels respectively to calculate the loss function. In the back propagation stage, the gradient information of the CTC network module is back propagated within the CTC network module to update the weight coefficients within the CTC network module, and the gradient information of the Attention network module is back propagated within the Attention network module to update the weight coefficients within the Attention network module. This process can also be understood as the CTC network module and the Attention network module updating the weight coefficients of the feature extraction module together.

[0111] The calculation speed of the CTC network module in the forward propagation process is relatively fast, but the gradient information of the CTC network module in the training stage is poor, so the Attention network module is introduced to perform forward propagation and back propagation in parallel with the CTC network module, so as to make up for the poor gradient information of the CTC network module in the training stage of the image processing model through the Attention network module, so that the gradient information of the feature extraction network in the back propagation is better.

[0112] However, in the process of introducing the forward propagation of the Attention network module, the calculation and inference process of the Attention network module is slow and cannot meet the timeliness requirement in the finger reading scene. Therefore, in order to meet the timeliness requirement in the finger reading scene, the present disclosure adopts the CTC network module and the Attention network module to perform forward propagation on the sequence feature samples in the training stage of the image processing model to calculate the loss function between the prediction result and the real result; in the test stage of the image processing model, the CTC network module is used to perform forward propagation on the sequence feature samples to obtain the prediction result.

[0113] In this way, in the training stage of the image processing model, the Attention network module can be used to assist the CTC network module, and in the back propagation process of the sequence recognition network, better gradient information can be fed back to the feature extraction module to improve the feature expression ability of the sequence recognition network; in the test stage of the image processing model, only the CTC network module can be used for forward propagation to avoid introducing the Attention network module and reduce the time of the calculation and inference process, thereby meeting the timeliness requirement in the finger reading scene.

[0114] By the technical solution, in the training stage of the image processing model, the time sequence classification module and the attention mechanism module are used to jointly perform the reverse propagation of gradient information, so that the gradient information transmitted to the feature extraction module is better; in the test stage of the image processing model, only the time sequence classification module is used to perform the forward propagation on the sequence features, so as to avoid introducing the attention mechanism module in the test stage, thereby avoiding the problem of low reading efficiency.

[0115] In a possible implementation, in the process of training the entire image processing model, the text style in the reading scene is diverse, which makes it difficult for the image processing model to be trained, and the robustness of the image processing model in recognizing text is poor. In order to reduce the training difficulty of the image processing model and improve the robustness of the image recognition model in recognizing text, the present disclosure further includes the following steps.

[0116] In step S21, the scanning image sample set is used to train the to-be-trained model to obtain a first model.

[0117] The scanning image sample set is an image sample set obtained by scanning a scanner, and the scanning image sample data can use D1 cn and D1 en .

[0118] In the training process of the first stage of the image processing model, the scanning image sample set can be used to train the to-be-trained model to obtain the first model. In the preliminary training process, the characters in the scanning image sample set are relatively regular and clear, so the training difficulty of the preliminary training process of the image processing model is relatively small.

[0119] For example, in the process of training the to-be-trained model, the English and Chinese data in the image can be matched according to a preset data ratio, for example, a ratio of 1:1. The to-be-trained model can be iterated for a first preset number of times, and the preset number can be 10 times.

[0120] In step S22, the first model is trained according to the first target image sample in the scanning image sample set whose image quality is less than a first preset condition to obtain a second model.

[0121] In the training process of the second stage of the image processing model, the first target image sample whose image quality is less than the first preset condition is randomly selected from the scanning image sample set at a first preset ratio; and the first model is trained according to the first target image sample to obtain the second model.

[0122] Specifically, in a scanned image sample set, image enhancement determination is performed on the samples in the scanned image sample set: if the image quality of a certain scanned image sample in the scanned image sample set is less than a first preset condition, then it is determined that the scanned image sample needs image enhancement, and this scanned image sample is used as the first target image sample, until the ratio of the number of first target image samples to the number of samples in the scanned image sample set is a first preset ratio; finally, multiple first target image samples are enhanced. Image enhancement methods include, but are not limited to, enhancing image brightness and contrast.

[0123] For example, taking a first preset ratio of 0.3 as an example, image enhancement determination can be performed on each sample in the scanned image sample set. If the image quality of a certain scanned image sample in the scanned image sample set is less than the first preset condition, it is determined that the scanned image sample needs to be enhanced, and the scanned image sample is used as the first target image sample, until the number of the first target image samples obtained accounts for 0.3 of the total number of scanned image samples in the entire scanned image sample set.

[0124] During the training of the first model, it can be iterated a second preset number of times, which can be 5 times. Compared with the first model, the trained second model can detect and identify characters in image samples that have undergone image enhancement.

[0125] It is evident that in the second stage of training the image processing model, image enhancement can be performed on the first target image samples with poor image quality in the scanned image sample set. The enhanced image samples are then used to train the first model to obtain the second model. This makes the entire training process gradual, greatly reduces the training difficulty of the image processing model, and avoids the image processing model from diverging.

[0126] In step S23, the second model is trained based on the second target image sample in the scanned image sample set and the third target image sample in the photographed image sample set to obtain the third model.

[0127] In the third stage of training the image processing model, a second target image sample is randomly selected from the scanned image sample set at a second preset ratio, and a third target image sample is randomly selected from the photographed image sample set; the second model is trained using the second target image sample and the third target image sample to obtain the third model.

[0128] Specifically, the second target image samples can be randomly selected from the scan image sample set, the third target image samples can be randomly selected from the photographed image sample set until the ratio between the number of the second target image samples and the number of the third target image samples is the second preset ratio, and the second target image samples and the third target image samples are input into the second model to train the second model to obtain the third model.

[0129] For example, the scan image sample set is D1 cn , the photographed image sample set is D2 en , and D1 cn and D2 en , D1 cn , D1 en , D2 cn and D2 en , the second target image samples and the third target image samples can be randomly extracted from the scan image sample set and the photographed image sample set according to a second preset ratio of 1:1:2:2.

[0130] In the process of training the second model, the second model can be iterated for a third preset number of times, and the third preset number of times can be 10. The third model obtained by training can accurately identify the text image in the pointing reading scene compared with the second model and the first model.

[0131] In the related art, in the process of training the image processing model, the photographed image samples are collected from natural scenes, such as street view images. The photographed image samples collected from the natural scenes are quite different from the scene in the pointing reading scene. Therefore, the image processing model obtained by training using such photographed image samples has a low accuracy in identifying the image in the pointing reading scene.

[0132] The photographed image sample set proposed by the present disclosure is a collection of text line images in multiple photographed formats, which can simulate and render a text line background with real paper texture and enhance the authenticity of the image by using random lighting, shadow blur, motion blur, and other effects to obtain the photographed image sample set.

[0133] In this way, the photographed image sample set proposed by the present disclosure is obtained by processing to be closer to the image sample in the pointing reading scene. The image processing model obtained by inputting such photographed image sample set into the image processing model for training can more accurately identify the image in the pointing reading scene.

[0134] It is evident that in the third stage of training the image processing model, a set of photographed image samples was introduced. The second model was trained by combining the photographed image sample set with the scanned image sample set. Based on the training process in the second stage, the training difficulty of the image processing model was further increased, making the training process of the image processing model gradual and enabling the image processing model to accurately recognize character information in the finger reading scenario.

[0135] In step S24, the third model is trained based on the fourth target image sample in the scanned image sample set whose image quality is lower than the first preset condition, and the fifth target image sample in the photographed image sample set whose image quality is lower than the second preset condition, to obtain the fourth model.

[0136] In the fourth stage of training the image processing model, a first image sample is randomly selected from the scanned image sample set at a third preset ratio, and a second image sample is randomly selected from the photographed image sample set; a fourth target image sample with image quality lower than the first preset condition is randomly selected from multiple first image samples at a fourth preset ratio, and a fifth target image sample with image quality lower than the second preset condition is randomly selected from multiple second image samples; the third model is trained based on the fourth target image sample and the fifth target image sample to obtain the image processing model.

[0137] Specifically, a first image sample can be randomly selected from the scanned image sample set, and a second image sample can be randomly selected from the photographed image sample set until the ratio of the number of first image samples to the number of second image samples is a third preset ratio; then, a fourth target image sample with image quality lower than the first preset condition can be randomly selected from the first image sample until the ratio of the number of fourth target image samples to the number of first image samples is a fourth preset ratio; a fifth target image sample with image quality lower than the second preset condition can be randomly selected from the second image sample until the ratio of the number of fifth target image samples to the number of second image samples is a fourth preset ratio; finally, image enhancement is performed on the fourth and fifth target image samples, and the enhanced image samples are used to train the third model to obtain the fourth model.

[0138] For example, let's take the scanned image sample set as D1. cn D1 en The sample set of photographed images is D2. cn and D2 en For example, D1 cn D1 en D2 cn and D2 enThe first image sample and the second image sample can be randomly selected from the scanning image sample set and the photographed image sample set according to a third preset ratio of 2:2:3:3; a fourth target image sample with image quality less than a first preset condition is randomly selected from the first image sample until the number of the fourth target image sample accounts for 0.2 of the number of the first image sample; a fifth target image sample with image quality less than a second preset condition is randomly selected from the second image sample until the number of the fifth target image sample accounts for 0.2 of the number of the second image sample; and finally, the fourth target image sample and the fifth target image sample with poor image quality are subjected to image enhancement, and the enhanced image sample is used to train the third model to obtain a fourth model.

[0139] During the training of the third model, the third model can be iterated for a fourth preset number of times, and the fourth preset number of times can be 5. The fourth model obtained by training can improve the generalization ability of the image processing model to recognize scanning images and photographed images, that is, the adaptability of the image processing model to new samples of scanning images and photographed images.

[0140] It can be seen that in the training process of the fourth stage of the image processing model, the photographed image sample set and the scanning image sample set with image quality enhancement are introduced, and the training difficulty of the image processing model is further increased on the basis of the training process of the third stage, so that the training process of the image processing model is gradual, and the generalization ability of the image processing model is improved.

[0141] In step S25, the fourth model is trained according to the distorted image sample of the third image sample in the scanning image sample set and the distorted image sample of the fourth image sample in the photographed image sample set, and an image processing model is obtained.

[0142] During the training process of the fifth stage of the image processing model, the third image sample is randomly selected from the scanning image sample set and the fourth image sample is randomly selected from the photographed image sample set according to a fifth preset ratio; the third image sample and the fourth image sample are distorted to obtain a distorted image sample; and the fourth model is trained with the distorted image sample to obtain the image processing model.

[0143] Specifically, a third image sample can be randomly selected from the scanned image sample set, and a fourth image sample can be randomly selected from the photographed image sample set, until the ratio of the number of third image samples to the number of fourth image samples is a fifth preset ratio. Then, the third image samples and the fourth image samples are distorted according to the TPS thin template correction method mentioned earlier to obtain distorted image samples. Next, image enhancement is performed on the distorted image samples, and distorted image samples with image quality lower than the first preset condition or the second preset condition are used as target distorted image samples, until the ratio of the number of target distorted image samples to the number of distorted image samples is a sixth preset ratio. Finally, the target distorted image samples are input into the fourth model to train the fourth model and obtain the image processing model.

[0144] For example, let's take the scanned image sample set as D1. cn D1 en The sample set of photographed images is D2. cn and D2 en For example, D1 cn D1 en D2 cn and D2 en Then, the third and fourth image samples can be randomly selected from the scanned image sample set and the photographed image sample set in a 1:1:1:1 ratio. The selected third and fourth image samples are then distorted to obtain distorted image samples. Next, the distorted image samples are randomly evaluated for image enhancement, and the distorted image samples with poor image quality are used as target distorted image samples until the ratio between the target distorted image samples and the distorted image samples is 0.5. Finally, the distorted image samples are used to train the fourth model to obtain the image processing model.

[0145] During the training of the fourth model, it can be iterated a fifth time (preset number, which can be 20 times). Compared to the fourth, third, second, and first models, the trained image processing model can accurately recognize text images in real-world finger-reading scenarios. This allows the image processing model to effectively improve its ability to recognize distorted text lines, making it more closely suited to finger-reading usage scenarios.

[0146] The first, second, third, fourth, fifth, and sixth preset ratios can be the same or different. When the six ratios are different, the recognition accuracy of the image processing model for all text images can be improved.

[0147] It can be seen that in the training process of the fifth stage of the image processing model, the scheme of distorting the scanning image sample and the photographed image sample is introduced, and on the basis of the training process of the fourth stage, the training difficulty of the image processing model is further increased, so that the image processing model improves the accuracy of identifying the image in the finger reading scene in the case of obtaining more image samples in the finger reading scene.

[0148] By the above technical solution, the training difficulty is gradually increased in different training stages to train the image processing model. The first aspect can train the model gradually and reduce the training difficulty of the image processing model. The second aspect enables the image processing model to receive different fresh samples in different training stages and improves the generalization ability of the image processing model to adapt to fresh samples. The third aspect distorts the planar image to train the image processing model using the distorted image sample, which can enable the image processing model to more accurately identify the distorted image in the finger reading scene and improve the identification accuracy of the distorted image.

[0149] Figure 7 is a block diagram of an image processing device according to an exemplary embodiment. Referring to Figure 7 The image processing device 210 includes a correction module 211, a target image determination module 212, a sequence feature determination module 213, and a prediction result determination module 214.

[0150] The correction module 211 is configured to correct the distorted to-be-processed image to obtain a corrected image, the corrected image including a segmented first sub-image and a second sub-image.

[0151] The target image determination module 212 is configured to input the first sub-image to a local network layer and the second sub-image to a local transition layer, the local network layer being used to perform a convolution and splicing operation on the first sub-image to obtain a target splicing image, and the local transition layer being used to perform a convolution and splicing operation on the target splicing image after convolution and the second sub-image to obtain a target image.

[0152] The sequence feature determination module 213 is configured to extract the relationship between each character in the target image to obtain a sequence feature with a word sequence.

[0153] The prediction result determination module 214 is configured to input the sequence feature to a sequence recognition network to obtain a prediction result.

[0154] Optionally, the image processing device 210 includes:

[0155] a convolution image determination submodule configured to, for each of the network layers, perform convolution on the first sub-image or the spliced image output by the previous network layer to obtain a convolution image;

[0156] a target spliced image determination submodule configured to, for each of the network layers, splice the convolution image in the same network layer with the first sub-image or splice the convolution image in the same network layer with the spliced image output by the previous network layer to obtain the target spliced image.

[0157] Optionally, the sequence recognition network is a network in an image processing model, and the prediction result determination module 214 includes:

[0158] a first forward propagation submodule configured to, in a training phase of the image processing model, perform forward propagation on a sequence feature sample by using a time sequence classification module and an attention mechanism module of the sequence recognition network to obtain a training result;

[0159] a second forward propagation submodule configured to, in a test phase of the image processing model, perform forward propagation on the sequence feature by using the time sequence classification model to obtain the prediction result.

[0160] Optionally, the correction module 211 includes:

[0161] a correction point set determination submodule configured to obtain a correction point set corresponding to a control point set of the to-be-processed image by using the image processing model;

[0162] a corrected image determination submodule configured to obtain the corrected image according to the correction point set.

[0163] Optionally, the image processing apparatus 210 includes:

[0164] a first model determination submodule configured to train a to-be-trained model by using a scanning image sample set to obtain a first model;

[0165] a second model determination submodule configured to train the first model according to a first target image sample in the scanning image sample set and having an image quality less than a first preset condition to obtain a second model;

[0166] a third model determination submodule configured to train the second model according to a second target image sample in the scanning image sample set and a third target image sample in a photographing image sample set to obtain a third model;

[0167] The image processing model submodule is configured to train the third model according to fourth target image samples with image quality less than the first preset condition in the scanning image sample set and fifth target image samples with image quality less than the second preset condition in the photographing image sample set, to obtain the image processing model.

[0168] Optionally, the second model determination submodule includes:

[0169] The first target image sample determination submodule is configured to randomly select first target image samples with image quality less than the first preset condition from the scanning image sample set at a first preset proportion.

[0170] The first training submodule is configured to train a first model according to the first target image samples, to obtain the second model.

[0171] Optionally, the third model determination submodule includes:

[0172] The third target image sample determination submodule is configured to randomly select second target image samples from the scanning image sample set and third target image samples from the photographing image sample set at a second preset proportion.

[0173] The second training submodule is configured to train the second model with the second target image samples and the third target image samples, to obtain the third model.

[0174] Optionally, the image processing model submodule includes:

[0175] The second image sample determination submodule is configured to randomly select first image samples from the scanning image sample set and second image samples from the photographing image sample set at a third preset proportion.

[0176] The selection submodule is configured to randomly select fourth target image samples with image quality less than the first preset condition from the plurality of first image samples and fifth target image samples with image quality less than the second preset condition from the plurality of second image samples at a fourth preset proportion.

[0177] The first image processing model submodule is configured to train the third model according to the fourth target image samples and the fifth target image samples, to obtain the image processing model.

[0178] Optionally, the first image processing model submodule includes:

[0179] The fourth model determination submodule is configured to obtain a fourth model according to the fourth target image samples and the fifth target image samples.

[0180] The fourth image sample determining sub-module is configured to randomly select a third image sample from the scanning image sample set and a fourth image sample from the photographing image sample set according to a fifth preset proportion.

[0181] The distortion image sample determining sub-module is configured to distort the third image sample and the fourth image sample to obtain a distortion image sample.

[0182] The second image processing model sub-module is configured to train the fourth model by using the distortion image sample to obtain the image processing model.

[0183] As to the apparatus in the above-described embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0184] The present disclosure also provides a computer-readable storage medium having computer program instructions stored therein, which, when executed by a processor, implement the steps of the image processing method provided by the present disclosure.

[0185] Figure 8 is a block diagram of an apparatus 800 for image processing according to an exemplary embodiment. The apparatus 800 can be a mobile phone, computer, digital broadcast terminal, message communicator, game console, tablet device, medical device, fitness device, personal digital assistant, learning machine, etc.

[0186] Referring to Figure 8 , the apparatus 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0187] The processing component 802 usually controls overall operations of the apparatus 800, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of the steps of the image processing method described above. Further, the processing component 802 can include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0188] The memory 804 is configured to store various types of data to support the operation of the device 800. Examples of these data include instructions for any application or method operating on the device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0189] The power supply component 806 supplies electrical power for the various components of the device 800. The power supply component 806 can include a power supply management system, one or more power supplies, and other components associated with generating, managing, and distributing electrical power for the device 800.

[0190] The multimedia component 808 includes a screen providing an output interface between the device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure associated with the touching or swiping action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 800 is in an operation mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0191] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0192] The input / output interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keyboard, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0193] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and keypad of the device 800, a change in position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration / g-force and a temperature change of the device 800. The sensor component 814 can include proximity sensor(s) configured to detect the presence of objects in a proximity without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0194] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a corresponding communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0195] In an example embodiment, the device 800 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic modules to perform the image processing methods described above.

[0196] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided. The instructions can be executable by the processor 820 of the device 800 to perform the image processing methods described above. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, and the like.

[0197] The apparatus described above can be a part of an independent electronic device, for example, in an embodiment, the apparatus can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be one IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC, System on a Chip or System Level Chip), etc. The integrated circuit or chip described above can be used to execute executable instructions (or code) to implement the image processing method described above. The executable instructions can be stored in the integrated circuit or chip, or obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions can be stored in the memory, and when the executable instructions are executed by the processor, the image processing method described above is implemented; or the integrated circuit or chip can receive executable instructions through the interface and transmit them to the processor for execution, to implement the image processing method described above.

[0198] In another exemplary embodiment, a computer program product is also provided, which contains a computer program capable of being executed by a programmable device, the computer program having a code portion for executing the image processing method described above when executed by the programmable device.

[0199] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom in the art. The specification and examples are to be regarded as illustrative only and the true scope and spirit of the present disclosure is indicated by the following claims.

[0200] It should be understood that the present disclosure is not limited to the precise structures described and shown in the drawings, and that various modifications and changes can be made without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims appended hereto.

Claims

1. An image processing method, characterized by, The method comprises: correcting a distorted to-be-processed image to obtain a corrected image, the corrected image comprising a segmented first sub-image and a segmented second sub-image; inputting the first sub-image into a local network layer and inputting the second sub-image into a local transition layer, the local network layer being configured to perform a convolution and splicing operation on the first sub-image to obtain a target spliced image, and the local transition layer being configured to perform a convolution and splicing operation on the target spliced image after convolution and the second sub-image to obtain a target image; extracting a relationship between each character in the target image to obtain a sequence feature with a vocabulary sequence; inputting the sequence feature into a sequence recognition network to obtain a prediction result; wherein the correcting the distorted to-be-processed image to obtain the corrected image comprises: obtaining a set of correction points corresponding to a set of control points of the to-be-processed image by using an image processing model; and obtaining the corrected image according to the set of correction points; wherein the image processing model is obtained by training through the following steps: training a to-be-trained model using a set of scanned image samples to obtain a first model; training the first model according to first target image samples in the set of scanned image samples with image quality less than a first preset condition to obtain a second model; training the second model according to second target image samples in the set of scanned image samples and third target image samples in a set of photographed image samples to obtain a third model; training the third model according to fourth target image samples in the set of scanned image samples with image quality less than the first preset condition and fifth target image samples in the set of photographed image samples with image quality less than a second preset condition to obtain the image processing model.

2. The method of claim 1, wherein, The local network layer comprises a plurality of network layers, and the target spliced image is obtained by the following steps: for each network layer, performing convolution on the first sub-image or a spliced image output by a previous network layer to obtain a convolution image; for each network layer, splicing the convolution image in the same network layer with the first sub-image or splicing the convolution image in the same network layer with a spliced image output by a previous network layer to obtain the target spliced image.

3. The method of claim 1, wherein, The sequence recognition network is a network in the image processing model, and the inputting the sequence feature into the sequence recognition network to obtain the prediction result comprises: in a training phase of the image processing model, using a time sequence classification module and an attention mechanism module of the sequence recognition network to perform forward propagation on a sequence feature sample to obtain a training result; in a testing phase of the image processing model, using the time sequence classification module to perform forward propagation on the sequence feature to obtain the prediction result.

4. The method of claim 1, wherein, The training the first model according to the first target image samples in the set of scanned image samples with image quality less than the first preset condition to obtain the second model comprises: randomly selecting the first target image samples with image quality less than the first preset condition from the set of scanned image samples at a first preset proportion; According to the first target image sample, a first model is trained to obtain the second model.

5. The method of claim 1, wherein, According to the second target image sample in the scanning image sample set and the third target image sample in the photographed image sample set, the second model is trained to obtain a third model, including: Second target image samples are randomly selected from the scanning image sample set at a second preset ratio, and third target image samples are randomly selected from the photographed image sample set at the second preset ratio; The second target image sample and the third target image sample are used to train the second model to obtain the third model.

6. The method of claim 1, wherein, According to the fourth target image sample in the scanning image sample set and the fifth target image sample in the photographed image sample set, the third model is trained to obtain the image processing model, including: First image samples are randomly selected from the scanning image sample set at a third preset ratio, and second image samples are randomly selected from the photographed image sample set at the third preset ratio; Fourth target image samples with image quality less than the first preset condition are randomly selected from a plurality of the first image samples at a fourth preset ratio, and fifth target image samples with image quality less than the second preset condition are randomly selected from a plurality of the second image samples at the fourth preset ratio; According to the fourth target image sample and the fifth target image sample, the third model is trained to obtain the image processing model.

7. The method of claim 6, wherein, According to the fourth target image sample and the fifth target image sample, the third model is trained to obtain the image processing model, including: According to the fourth target image sample and the fifth target image sample, a fourth model is obtained; Third image samples are randomly selected from the scanning image sample set at a fifth preset ratio, and fourth image samples are randomly selected from the photographed image sample set at the fifth preset ratio; The third image sample and the fourth image sample are distorted to obtain a distorted image sample; The distorted image sample is used to train the fourth model to obtain the image processing model.

8. An image processing apparatus characterized by comprising: Including: The correction module is configured to correct the distorted to-be-processed image to obtain a corrected image, the corrected image including a segmented first sub-image and a second sub-image; The target image determination module is configured to input the first sub-image to a local network layer and input the second sub-image to a local transition layer, the local network layer being used to perform a convolution and splicing operation on the first sub-image to obtain a target splicing image, and the local transition layer being used to perform a convolution and splicing operation on the target splicing image after convolution and the second sub-image to obtain a target image; The sequence feature determination module is configured to extract a relationship between each character in the target image to obtain a sequence feature with a word sequence; The prediction result determination module is configured to input the sequence feature into a sequence recognition network to obtain a prediction result; The correction module is further configured to obtain a set of correction points corresponding to a set of control points of the to-be-processed image through an image processing model, and obtain the corrected image according to the set of correction points. The image processing model is obtained through the following steps: The first model is obtained by training the to-be-trained model using a scanning image sample set; The second model is obtained by training the first model according to first target image samples in the scanning image sample set and less than a first preset condition; The third model is obtained by training the second model according to second target image samples in the scanning image sample set and third target image samples in a photographing image sample set; The image processing model is obtained by training the third model according to fourth target image samples in the scanning image sample set and less than the first preset condition and fifth target image samples in the photographing image sample set and less than a second preset condition.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The program instruction is executed by the processor to implement the steps of the method in any one of claims 1-7.

Images