A method, device and equipment for processing mixed arrangement text information in an image

Abstract

The application provides a kind of image mixed arrangement character information processing method, device and equipment, method includes: obtaining image to be processed, image to be processed contains horizontal character and vertical character;According to preset segmentation granularity, the image to be processed is segmented, and a set of images is obtained;Image set includes: multiple first segmentation images containing horizontal character and multiple second segmentation images containing vertical character;Two-dimensional feature extraction is carried out on multiple first segmentation images and multiple second segmentation images, and a set of first feature data images is obtained;The first feature data image set is converted into three-dimensional feature, and a set of second feature data images is obtained;Three-dimensional feature extraction is carried out on the image in the second feature data image set, and a target format image is obtained;The target format image is input into the character recognition unit for character recognition, and the target character information in the image is obtained.The scheme can accurately identify the mixed arrangement of horizontal character and vertical character in the image, and improve the recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer information processing, in particular to an image mixed arrangement text information processing method, device and equipment. BACKGROUND

[0002] Text recognition is one of the basic functions in the field of optical character recognition (OCR), and there are mixed arrangements of horizontal text and vertical text in some recognition scenarios. In the prior art, the CRNN deep learning model is mainly used, and for the vertical text recognition method, the vertical text picture is rotated by 90 degrees, at which time the horizontal / vertical text input window of the model is fully compatible, so that the model training and deployment process can proceed smoothly. However, because the vertical text picture is rotated by 90 degrees, the image features become inconsistent, resulting in an increase in the pressure of the fully connected layer of the model and a decrease in accuracy. SUMMARY

[0003] The present application provides an image mixed arrangement text information processing method, device and equipment to solve the problem of low recognition accuracy in the prior art for mixed arrangements of horizontal text and vertical text.

[0004] To solve the above technical problems, the technical solutions of the present application are as follows:

[0005] An image mixed arrangement text information processing method, comprising:

[0006] Obtaining a to-be-processed image, the to-be-processed image containing horizontal text and vertical text;

[0007] Segmenting the to-be-processed image according to a preset segmentation granularity to obtain an image set comprising a plurality of segmented images; the image set comprises a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text;

[0008] Performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set;

[0009] Converting the first feature data image set into three-dimensional features to obtain a second feature data image set;

[0010] Performing three-dimensional feature extraction on the images in the second feature data image set to obtain a target format image;

[0011] Inputting the target format image into a text recognition unit of a trained text recognition model for text recognition to obtain target text information in the image, the text recognition model being obtained by training a preset network model according to a preset training set.

[0012] Optionally, the to-be-processed image is segmented according to a preset segmentation granularity to obtain an image set comprising a plurality of segmented images, comprising:

[0013] The to-be-processed image is segmented according to N segmentation granularity of N pixels, to obtain an image set of a plurality of segmented images, N being a positive integer respectively.

[0014] Optionally, the plurality of first segmented images and the plurality of second segmented images are subjected to two-dimensional feature extraction to obtain a first feature data image set, comprising:

[0015] The plurality of first segmented images are subjected to two-dimensional convolution operation to obtain a first two-dimensional convolution operation result;

[0016] The plurality of second segmented images are subjected to two-dimensional convolution operation to obtain a second two-dimensional convolution operation result;

[0017] According to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result, a first feature data image set is obtained.

[0018] Optionally, the first feature data image set is subjected to three-dimensional feature conversion to obtain a second feature data image set, comprising:

[0019] The plurality of feature images in the first feature data image set are arranged in time sequence to obtain an image time sequence data set;

[0020] The image time sequence data set is subjected to three-dimensional convolution operation to obtain a second feature data image set.

[0021] Optionally, the second feature data image set is subjected to three-dimensional feature extraction to obtain a target format image, comprising:

[0022] The plurality of feature images in the second feature data image set are subjected to three-dimensional convolution feature extraction to obtain a three-dimensional feature data set;

[0023] The plurality of feature images in the three-dimensional feature data set are subjected to two-dimensional convolution feature extraction to obtain a target format image arranged in time sequence.

[0024] Optionally, the target format image is input into a trained character recognition model for character recognition to obtain target character information in the image, comprising:

[0025] The target format image is input into an input unit of the trained character recognition model for processing to obtain a first output result;

[0026] The first output result is input into a hidden layer of the character recognition model for processing to obtain a second output result;

[0027] inputting the second output result into an output layer of the character recognition model for processing to obtain a target output result;

[0028] inputting the target output result into a full connection layer of the character recognition model for character feature recognition to obtain target character information in the image.

[0029] Optionally, inputting the target output result into a full connection layer of the character recognition model for character feature recognition to obtain target character information in the image, comprising:

[0030] inputting the target output result into a full connection layer of the character recognition model for feature integration and then outputting an image classification result;

[0031] obtaining target character information in the image according to the image classification result.

[0032] The application further provides an image mixed character information processing device, comprising:

[0033] an acquisition module configured to acquire a to-be-processed image, wherein the to-be-processed image contains horizontal characters and vertical characters;

[0034] a processing module configured to split the to-be-processed image according to a preset splitting granularity to obtain an image set comprising a plurality of split images, wherein the image set comprises a plurality of first split images containing horizontal characters and a plurality of second split images containing vertical characters; perform two-dimensional feature extraction on the plurality of first split images and the plurality of second split images to obtain a first feature data image set; perform three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set; perform three-dimensional feature extraction on images in the second feature data image set to obtain a target format image; and input the target format image into a character recognition unit of a trained character recognition model for character recognition to obtain target character information in the image, wherein the character recognition model is obtained by training a preset network model according to a preset training set.

[0035] The application further provides a computing device comprising a processor and a memory storing a computer program, wherein the computer program is executed by the processor to perform the method described above.

[0036] The application further provides a computer-readable storage medium storing instructions, wherein the instructions are executed on a computer to cause the computer to perform the method described above.

[0037] The above-mentioned scheme of the application has at least the following advantages:

[0038] The scheme of the present application, by obtaining the to-be-processed image, the to-be-processed image contains horizontal text and vertical text; the to-be-processed image is segmented according to a preset segmentation granularity, and an image set including a plurality of segmented images is obtained; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text; two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images, and a first feature data image set is obtained; the first feature data image set is subjected to three-dimensional feature conversion, and a second feature data image set is obtained; three-dimensional feature extraction is performed on the images in the second feature data image set, and a target format image is obtained; the target format image is input into a character recognition unit of a trained character recognition model for character recognition, and target character information in the image is obtained, and the character recognition model is obtained by training a preset network model according to a preset training set. The scheme of the present application can accurately recognize the mixed arrangement of horizontal text and vertical text in the picture, and improves the character recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a step diagram of the image mixed arrangement character information processing method provided by the embodiment of the present application;

[0040] Figure 2 is a flowchart of the image mixed arrangement character information processing method provided by the embodiment of the present application;

[0041] Figure 3 is a module schematic diagram of the image mixed arrangement character information processing device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0042] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be accurately conveyed to those skilled in the art.

[0043] As shown in Figure 1 , the embodiment of the present application proposes an image mixed arrangement character information processing method, comprising:

[0044] Step 11, obtaining a to-be-processed image, the to-be-processed image contains horizontal text and vertical text;

[0045] Step 12, the to-be-processed image is segmented according to a preset segmentation granularity, and an image set including a plurality of segmented images is obtained; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text;

[0046] Step 13: Perform two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set;

[0047] Step 14: Perform three-dimensional feature transformation on the first feature data image set to obtain the second feature data image set;

[0048] Step 15: Perform three-dimensional feature extraction on the images in the second feature data image set to obtain the target format image;

[0049] Step 16: Input the target format image into the text recognition unit of the trained text recognition model to perform text recognition and obtain the target text information in the image. The text recognition model is obtained by training a preset network model based on a preset training set.

[0050] In this embodiment, the horizontal and vertical text images are first divided into n square images with a certain step size. At this point, the relationship between the images becomes separated. After a long text strip is divided into n square images, a 3D convolutional model is used to combine the features of the fragmented images and arrange them in a time sequence, which can be regarded as forming a video stream with n frames. Then, the combined and arranged images are input into a trained text recognition model for text recognition, and the output is classified to obtain the target text information. In this way, horizontal and vertical text images in the image can be input into the model at the same time, and the horizontal and vertical text images will not affect each other when rotated. This solves the problem that the horizontal and vertical text images in the image will have inconsistent features when rotated by 90 degrees, and the sizes of the horizontal and vertical text boxes are different, making it impossible to directly input the two image sizes into the text recognition unit of the text recognition model for processing.

[0051] The text recognition model in this embodiment includes an image processing unit and a text recognition unit. The image processing unit has image processing functions such as two-dimensional feature extraction, three-dimensional feature extraction, and three-dimensional feature transformation. The text recognition unit has the function of recognizing text in the image after it has been processed by the image processing unit.

[0052] The text information processing method in this embodiment can accurately identify mixed horizontal and vertical text in images, thus improving the recognition accuracy.

[0053] like Figure 2 As shown, in an optional embodiment of the present invention, step 12 may include:

[0054] Step 121, process the image to be processed according to N The image set is obtained by segmenting at a granularity of N pixels to obtain multiple segmented images, where N is a positive integer.

[0055] In this embodiment, the original image is divided into multiple square pictures according to N N pixels, which realizes standardization and unification in the shape dimension, and facilitates subsequent uniform feature processing. N represents the height and width pixel numbers of each small image after division, and the two parameters determine the size specification of the small image.

[0056] The number of division in the height direction is: , wherein The positive integer is taken downward.

[0057] The number of division in the width direction is: , wherein The positive integer is taken downward.

[0058] By traversing each pixel position of the initial image, a small block image is cut out according to the size of N1*N2. Specifically, starting from the top left corner of the initial image,

[0059] In the horizontal direction, N1 is taken as the step, and in the vertical direction, N2 is taken as the step, to extract an image region starting from the initial coordinates (i , j ), ending coordinates , width N1, and height N2; wherein, (0 ).

[0060] Optionally, the cutting step value used in this embodiment is 8, and N (N1, N2) is usually 32, and after cutting, there are 57 pictures of 32x32 pixels. The specific step value is 8, that is, the picture is moved forward by 8 pixels each time, and since the picture size is 32x32 pixels, the picture will have a serious overlap. At this time, the overlap is used to generate stable information in the time t dimension, which is similar to a video playback. Subsequently, the 3D convolution is used to capture the features in the time t dimension to support image classification.

[0061] In an optional embodiment of the present application, step 13 can include:

[0062] Step 131, performing a two-dimensional convolution operation on the plurality of first divided images to obtain a first two-dimensional convolution operation result;

[0063] Step 132, performing a two-dimensional convolution operation on the plurality of second divided images to obtain a second two-dimensional convolution operation result;

[0064] Step 133, obtaining a first feature data image set according to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result.

[0065] In this embodiment, after obtaining the unified square picture image set, 2D convolution operation is used to extract the features of each square picture, such as image edges, textures, local shapes and the like. After each square picture is subjected to such 2D convolution operation, a set of feature representations capable of representing the unique information of the picture is obtained, which prepares for further integration of features.

[0066] Specifically, by formula , 2D feature extraction is performed on the square pictures in the image set, including features such as edges, textures, shapes and the like. The extracted feature data is combined into corresponding images to obtain a first feature data image.

[0067] wherein, is a first feature data image set, x and y are coordinates of elements in the first feature data image set, obtained by multiplying and accumulating the input local region after the convolution kernel is flipped; is a pixel value of an input image set (an image set composed of first segmentation images and second segmentation images), the position is determined by discrete coordinates ; is a weight value of a 2D (two-dimensional) convolution kernel, the position is an offset relative to the center; is a horizontal direction index, the range is ; is a vertical direction index, the range is , is an image set composed of a plurality of first segmentation images and second segmentation images, is a weight value.

[0068] In an optional embodiment of the present application, step 14 can include:

[0069] Step 141, arranging a plurality of feature images in the first feature data image set in time sequence to obtain an image time sequence data set;

[0070] Step 142, performing 3D convolution operation on the image time sequence data set to obtain a second feature data image set.

[0071] In this embodiment, 3D convolution is used to integrate the 2D features of the n square pictures obtained before. The difference between 3D convolution and 2D convolution is that 3D convolution considers not only the length and width dimensions of the image, but also the depth (which can be understood as the dimension corresponding to the n square pictures).

[0072] Specifically, first, a plurality of feature images in the first feature data image set are arranged in time sequence to obtain an image time sequence data set .

[0073] Further, by formula:

[0074] ;

[0075] ;

[0076] A three-dimensional feature conversion is performed to obtain a second feature data image set.

[0077] wherein, is obtained by multiplying and accumulating the input local area after the convolution kernel is flipped, and x, y, and z are coordinates of elements in the second feature data image set; is an image time series data set, the position is determined by discrete coordinates ; is a weight value of a 3D (three-dimensional) convolution kernel, D, H, and W are respectively the size of the corresponding convolution kernel in the channel, height, and width direction, D is the picture depth, H is the picture height, W is the picture width, i is the depth direction index, j is the height direction index, and k is the width direction index.

[0078] After the entire picture is divided into 57 32x32 small pictures at the beginning, there is a time sequence relationship between these small pictures. 2D convolution cannot well capture the information on the time sequence, while in 3D convolution, the 3D filter can move in all three directions (height, width, and channel of the image). By sliding and operating the 3D convolution kernel in this three-dimensional data space, the features of multiple square pictures in time sequence can be reflected.

[0079] Here, using 3D convolution can not only extract 2D features, but also integrate the relationship in time sequence. It is noted that the two stages of 2D feature extraction and 3D feature extraction are not strictly related or in sequence, and 2D feature extraction can be performed after 3D feature extraction, or 2D convolution and 3D convolution can be combined. In the actual processing process, when 2D features need to be extracted, the D of the 3D convolution is set to 1, at this time, it degenerates into 2D convolution, and when the time sequence feature needs to be extracted, the D of the 3D convolution is set to be greater than 1.

[0080] In an optional embodiment of the present application, step 15 can include:

[0081] Step 151, performing three-dimensional convolution feature extraction on a plurality of feature images in the second feature data image set to obtain a three-dimensional feature data set;

[0082] Step 152, performing two-dimensional convolution feature extraction on a plurality of feature images in the three-dimensional feature data set to obtain a target format image arranged in time sequence.

[0083] In the embodiment, after the first feature data image is extracted by 3D feature extraction, the second feature data image is in a specific 3D state, but in order to be connected to the character recognition model for subsequent processing, it is necessary to perform dimension transformation again to transform the current feature state back to the sequence state, so as to make the feature extracted from the image enter the character recognition model for the next feature extraction work.

[0084] The second feature data images are all generated from original images at different time points, and there is a time sequence between them, so the second feature data images are spliced into time sequence state data along the time dimension.

[0085] In an optional embodiment of the present application, step 16 can include:

[0086] Step 161, input the target format image into the input unit of the trained character recognition model for processing to obtain a first output result;

[0087] Step 162, input the first output result into the hidden layer of the character recognition model for processing to obtain a second output result;

[0088] Step 163, input the second output result into the output layer of the character recognition model for processing to obtain a target output result;

[0089] Step 164, input the target output result into the full connection layer of the character recognition model for character feature recognition to obtain target character information in the image.

[0090] In the embodiment, the character recognition model is first trained, and the recurrent neural network model is used in the embodiment. The recurrent neural network model is first trained using training set, validation set and test set data, the loss function uses mean square error loss function, and the training set includes the cut direction picture sequence data.

[0091] The training data is divided into several batches, for example, the batch size is set to 32, that is, 32 samples are used for training each time.

[0092] The input data of a batch is transmitted into the recurrent neural network model, the output of each layer is calculated in turn according to the network structure, and finally the output value of the model is obtained. The loss value of the current batch data is calculated using the loss function, and the gradient of the loss function with respect to the model parameters is calculated through the back propagation algorithm.

[0093] In the recurrent neural network model, due to the existence of the recurrent connection, the back propagation is unfolded along the time steps to calculate the gradient of the parameters at each time step. The optimizer updates the parameters (such as weights and biases) of the model according to the calculated gradient to reduce the value of the loss function; then a fully connected layer (FC) is connected behind the output layer of the trained recurrent neural network model to obtain the character recognition model.

[0094] The target format image is input into the input unit of the trained character recognition model (i.e., the trained recurrent neural network model) for processing. The input unit performs preliminary preprocessing on the input image data to meet the processing requirements of the subsequent layers.

[0095] Then, the preprocessed image data is input into the hidden layer of the character recognition model. The hidden layer receives the first output result from the input unit. The hidden layer includes multiple neurons (or nodes) to extract and abstract features from the input data.

[0096] Specifically, a neuron in the hidden layer performs a weighted summation operation on the input data, i.e., multiplies each dimension of the input data by the corresponding connection weight, adds the results, processes them with an activation function, and then passes them to the next neuron. The data is gradually passed and processed between these neurons, constantly extracting higher-level and more abstract features to obtain a second output result.

[0097] The output layer receives the second output result, performs a weighted summation operation on the second output result, maps the abstract features to the final output space, generates an output representation related to the target character information based on the features extracted by the hidden layer, and obtains a target output result. The target output result is input into the fully connected layer to integrate and map the output result, finally converts the probability distribution vector output by the model into specific characters, and completes the character recognition function. In an optional embodiment of the present application, step 164 can include:

[0098] Step 1641: inputting the target output result into the fully connected layer of the character recognition model to integrate the features and output an image classification result.

[0099] Step 1642: obtaining the target character information in the image based on the image classification result.

[0100] In this embodiment, the features obtained through the character recognition model are connected to the fully connected layer. The fully connected layer integrates and maps the extracted features, and finally outputs a result for classification. By analyzing and judging these integrated features, it is determined which category the image belongs to, and the task goal of the entire image classification is achieved.

[0101] During training, a character table is established in the full connection layer, the length of the character table can be set to 5000-7000, each position corresponds to a Chinese character, and a corresponding relationship between the characters and the position numbers of the character table is established. For example, "1" represents "space", "2" represents "a", and "100" represents "country". After the character recognition model processes the input image, a vector is output. The vector contains the prediction information of the model for the characters in the input image, and each dimension of the vector may correspond to the possibility or related feature information of a character in the character table. For example, if a certain dimension value in the vector is high, it may indicate that the model predicts that the character in the input image is more likely to correspond to the character associated with the dimension in the character table.

[0102] Using the previously set corresponding relationship between characters and position numbers, the vector output by the model can be converted into specific Chinese characters (or other characters). Specifically, by analyzing the values of each dimension in the vector, the dimension with the maximum value is found, and the number corresponding to the dimension is the position number of a character in the character table. For example, if the dimension corresponding to the number 100 has the maximum value in the vector, according to the previously set corresponding relationship "100 represents country", it can be determined that the character recognized by the model is "country". In this way, the conversion from the model output vector to the specific recognizable character is completed, thereby realizing the character recognition function.

[0103] The image mixed arrangement character information processing method described in the above embodiments of the present application solves the problem of reduced recognition accuracy of conventional model solutions for the case of horizontally arranged and vertically mixed arrangement characters in a picture, can accurately recognize the horizontally arranged and vertically mixed arrangement characters in a picture, and improves the recognition accuracy.

[0104] As shown in Figure 3 The embodiment of the present application also provides an image mixed arrangement character information processing device 30, which comprises:

[0105] An acquisition module 31 is configured to acquire a to-be-processed image, wherein the to-be-processed image contains horizontally arranged characters and vertically arranged characters.

[0106] The processing module 32 is configured to segment the image to be processed according to a preset segmentation granularity to obtain an image set including a plurality of segmented images; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text; two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set; three-dimensional feature conversion is performed on the first feature data image set to obtain a second feature data image set; three-dimensional feature extraction is performed on images in the second feature data image set to obtain a target format image; the target format image is input into a character recognition unit of a trained character recognition model for character recognition to obtain target character information in the image, and the character recognition model is obtained by training a preset network model according to a preset training set.

[0107] Optionally, the image to be processed is segmented according to a preset segmentation granularity to obtain an image set including a plurality of segmented images, including:

[0108] The image to be processed is segmented according to N N-pixel segmentation granularity to obtain an image set of a plurality of segmented images, and N is a positive integer respectively.

[0109] Optionally, two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set, including:

[0110] Two-dimensional convolution operation is performed on the plurality of first segmented images to obtain a first two-dimensional convolution operation result;

[0111] Two-dimensional convolution operation is performed on the plurality of second segmented images to obtain a second two-dimensional convolution operation result;

[0112] According to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result, a first feature data image set is obtained.

[0113] Optionally, three-dimensional feature conversion is performed on the first feature data image set to obtain a second feature data image set, including:

[0114] The plurality of feature images in the first feature data image set are arranged in time sequence to obtain an image time sequence data set;

[0115] Three-dimensional convolution operation is performed on the image time sequence data set to obtain a second feature data image set.

[0116] Optionally, three-dimensional feature extraction is performed on the second feature data image set to obtain a target format image, including:

[0117] The plurality of feature images in the second feature data image set are subjected to three-dimensional convolution feature extraction to obtain a three-dimensional feature data set;

[0118] The plurality of feature images in the three-dimensional feature data set are subjected to two-dimensional convolution feature extraction to obtain target format images arranged in time sequence.

[0119] Optionally, the target format images are input into a trained character recognition model for character recognition to obtain target character information in the images, including:

[0120] The target format images are input into an input unit of the trained character recognition model for processing to obtain a first output result;

[0121] The first output result is input into a hidden layer of the character recognition model for processing to obtain a second output result;

[0122] The second output result is input into an output layer of the character recognition model for processing to obtain a target output result;

[0123] The target output result is input into a full connection layer of the character recognition model for character feature recognition to obtain target character information in the images.

[0124] Optionally, the target output result is input into a full connection layer of the character recognition model for character feature recognition to obtain target character information in the images, including:

[0125] After the target output result is input into the full connection layer of the character recognition model for feature integration, an image classification result is output;

[0126] According to the image classification result, target character information in the images is obtained.

[0127] It should be noted that the device corresponds to the above method, and all implementation manners in the above method embodiments are applicable to the device embodiments and can achieve the same technical effects.

[0128] In an embodiment of the present application, a computing device is provided, including a processor and a memory storing a computer program, when the computer program is run by the processor, the method as described in the above embodiments is executed. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0129] In an embodiment of the present application, a computer readable storage medium is provided, storing instructions, when the instructions are run on a computer, the computer executes the method as described in the above embodiments. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0130] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0132] In the embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of the units is merely a logical function division, and another division manner can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0133] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0134] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0135] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various program code storage media.

[0136] In addition, it should be noted that in the device and method of the present application, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombination should be considered as equivalent solutions of the present application. Moreover, the steps of performing the above series of processes can naturally be executed in time sequence according to the order of description, but do not necessarily have to be executed in time sequence. Some steps can be executed in parallel or independently of each other. It can be understood by those skilled in the art that all or any steps or components of the method and device of the present application can be realized in hardware, firmware, software or a combination thereof in any computing device (including a processor, a storage medium, etc.) or a network of computing devices, which can be realized by those skilled in the art using their basic programming skills after reading the description of the present application.

[0137] Therefore, the object of the present application can also be realized by running a program or a set of programs on any computing device. The computing device can be a commonly known general-purpose device. Therefore, the object of the present application can also be realized only by providing a program product containing program code for realizing the method or device. That is, such a program product also constitutes the present application, and a storage medium storing such a program product also constitutes the present application. Obviously, the storage medium can be any commonly known storage medium or any storage medium developed in the future. It should be noted that in the device and method of the present application, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombination should be considered as equivalent solutions of the present application. Moreover, the steps of performing the above series of processes can naturally be executed in time sequence according to the order of description, but do not necessarily have to be executed in time sequence. Some steps can be executed in parallel or independently of each other.

[0138] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A method for processing mixed text information in an image, characterized in that, include: Obtain an image to be processed, which contains horizontal and vertical text; The image to be processed is segmented according to a preset segmentation granularity to obtain an image set including multiple segmented images; the image set includes multiple first segmented images containing horizontal text and multiple second segmented images containing vertical text; Two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set; The first feature data image set is subjected to three-dimensional feature transformation to obtain the second feature data image set; Three-dimensional feature extraction is performed on the images in the second feature data image set to obtain the target format image; The target format image is input into the text recognition unit of the trained text recognition model for text recognition to obtain the target text information in the image. The text recognition model is obtained by training a preset network model based on a preset training set. The image to be processed is segmented according to a preset segmentation granularity to obtain an image set including multiple segmented images, including: The image to be processed is processed according to N The image set is divided into multiple segmented images with a granularity of N pixels, where N is a positive integer; N represents the height and width of each small image after segmentation. Specifically, two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set, including: Perform a two-dimensional convolution operation on the plurality of first segmented images to obtain the first two-dimensional convolution operation result; Perform a two-dimensional convolution operation on the plurality of second segmented images to obtain the second two-dimensional convolution operation result; Based on the results of the first two-dimensional convolution operation and the second two-dimensional convolution operation, a first feature data image set is obtained; The first feature data image set is subjected to three-dimensional feature transformation to obtain the second feature data image set, including: Arrange multiple feature images from the first feature data image set in chronological order to obtain an image time series dataset; Perform a three-dimensional convolution operation on the image time series dataset to obtain a second feature data image set; Specifically, three-dimensional feature extraction is performed on the second feature data image set to obtain a target format image, including: Three-dimensional convolutional feature extraction is performed on multiple feature images in the second feature data image set to obtain a three-dimensional feature dataset; Two-dimensional convolutional feature extraction is performed on multiple feature images in the three-dimensional feature dataset to obtain target format images arranged in chronological order.

2. The method for processing mixed text information in an image according to claim 1, characterized in that, The target format image is input into a trained text recognition model for text recognition to obtain the target text information in the image, including: The target format image is input into the input unit of the trained text recognition model for processing to obtain the first output result; The first output result is input into the hidden layer of the character recognition model for processing to obtain the second output result; The second output result is input into the output layer of the character recognition model for processing to obtain the target output result; The target output is input into the fully connected layer of the text recognition model for text feature recognition to obtain the target text information in the image.

3. The method for processing mixed text information in an image according to claim 2, characterized in that, The target output is input into the fully connected layer of the text recognition model for text feature recognition to obtain the target text information in the image, including: The target output result is input into the fully connected layer of the text recognition model for feature integration, and then the image classification result is output. Based on the image classification results, the target text information in the image is obtained.

4. A device for processing mixed text information in an image, characterized in that, include: The acquisition module is used to acquire the image to be processed, which contains horizontal and vertical text. The processing module is used to segment the image to be processed according to a preset segmentation granularity to obtain an image set including multiple segmented images; the image set includes multiple first segmented images containing horizontal text and multiple second segmented images containing vertical text; Two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set; the first feature data image set is then subjected to three-dimensional feature transformation to obtain a second feature data image set. Three-dimensional feature extraction is performed on the images in the second feature data image set to obtain the target format image; the target format image is input into the text recognition unit of the trained text recognition model for text recognition to obtain the target text information in the image; the text recognition model is trained on a preset network model according to a preset training set. The image to be processed is segmented according to a preset segmentation granularity to obtain an image set including multiple segmented images, including: The image to be processed is processed according to N The image set is divided into multiple segmented images with a granularity of N pixels, where N is a positive integer; N represents the height and width of each small image after segmentation. Specifically, two-dimensional feature extraction is performed on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set, including: Perform a two-dimensional convolution operation on the plurality of first segmented images to obtain the first two-dimensional convolution operation result; Perform a two-dimensional convolution operation on the plurality of second segmented images to obtain the second two-dimensional convolution operation result; Based on the results of the first two-dimensional convolution operation and the second two-dimensional convolution operation, a first feature data image set is obtained; The first feature data image set is subjected to three-dimensional feature transformation to obtain the second feature data image set, including: Arrange multiple feature images from the first feature data image set in chronological order to obtain an image time series dataset; Perform a three-dimensional convolution operation on the image time series dataset to obtain a second feature data image set; Specifically, three-dimensional feature extraction is performed on the second feature data image set to obtain a target format image, including: Three-dimensional convolutional feature extraction is performed on multiple feature images in the second feature data image set to obtain a three-dimensional feature dataset; Two-dimensional convolutional feature extraction is performed on multiple feature images in the three-dimensional feature dataset to obtain target format images arranged in chronological order.

5. A computing device, characterized in that, include: A processor, a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The system stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 3.

Images