Method, device and equipment for processing mixed character information in image

Abstract

The invention provides a method, device and equipment for processing mixed character information in an image, and the method comprises the steps: obtaining a to-be-processed image which comprises horizontal characters and vertical characters; segmenting the to-be-processed image according to a preset segmentation granularity to obtain an image set; the image set comprises a plurality of first segmented images containing horizontal characters and a plurality of second segmented images containing vertical characters; performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set; performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set; performing three-dimensional feature extraction on the images in the second feature data image set to obtain a target format image; and inputting the target format image into a character recognition unit for character recognition to obtain target character information in the image. According to the scheme, the mixed arrangement condition of the horizontal-row characters and the vertical-row characters in the image can be accurately recognized, and the recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer information processing, and in particular to a method, device and equipment for processing mixed text information in an image. Background Art

[0002] Text recognition is one of the basic functions in the field of optical character recognition (OCR). In some recognition scenarios, horizontal and vertical text are mixed. The existing technology is mainly based on the CRNN deep learning model. For the vertical text recognition method, the vertical text image is rotated 90 degrees. At this time, the horizontal / vertical text input windows of the model are fully compatible, so that the model training and deployment process can proceed smoothly. However, because the vertical text image is rotated 90 degrees, the image features become inconsistent, resulting in increased pressure on the fully connected layer of the model and reduced accuracy. Summary of the invention

[0003] The present invention provides a method, device and equipment for processing mixed text information in an image, so as to solve the problem of low recognition accuracy of mixed horizontal and vertical text in the prior art.

[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0005] A method for processing mixed text information in an image, comprising:

[0006] Acquire an image to be processed, wherein the image to be processed includes horizontal text and vertical text;

[0007] Segmenting the image to be processed according to a preset segmentation granularity to obtain an image set including a plurality of segmented images; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text;

[0008] Performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set;

[0009] Performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set;

[0010] Performing three-dimensional feature extraction on the images in the second feature data image set to obtain a target format image;

[0011] The target format image is input into the text recognition unit of a trained text recognition model for text recognition to obtain target text information in the image. The text recognition model is obtained by training a preset network model according to a preset training set.

[0012] Optionally, the image to be processed is segmented according to a preset segmentation granularity to obtain an image set including a plurality of segmented images, including:

[0013] The image to be processed is processed according to N The image is segmented with a segmentation granularity of N pixels to obtain an image set of multiple segmented images, where N is a positive integer.

[0014] Optionally, performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set includes:

[0015] Performing a two-dimensional convolution operation on the plurality of first segmented images to obtain a first two-dimensional convolution operation result;

[0016] Performing a two-dimensional convolution operation on the plurality of second segmented images to obtain a second two-dimensional convolution operation result;

[0017] A first feature data image set is obtained according to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result.

[0018] Optionally, performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set includes:

[0019] Arranging the plurality of feature images in the first feature data image set in chronological order to obtain an image time series data set;

[0020] A three-dimensional convolution operation is performed on the image time series data set to obtain a second feature data image set.

[0021] Optionally, performing three-dimensional feature extraction on the second feature data image set to obtain a target format image includes:

[0022] Perform three-dimensional convolution feature extraction on a plurality of feature images in the second feature data image set to obtain a three-dimensional feature data set;

[0023] Perform two-dimensional convolution feature extraction on multiple feature images in the three-dimensional feature data set to obtain target format images arranged in chronological order.

[0024] Optionally, the target format image is input into a trained text recognition model to perform text recognition to obtain target text information in the image, including:

[0025] Inputting the target format image into an input unit of a trained text recognition model for processing to obtain a first output result;

[0026] Inputting the first output result into the hidden layer of the text recognition model for processing to obtain a second output result;

[0027] Inputting the second output result into the output layer of the text recognition model for processing to obtain a target output result;

[0028] The target output result is input into the fully connected layer of the text recognition model to perform text feature recognition to obtain the target text information in the image.

[0029] Optionally, inputting the target output result into the fully connected layer of the text recognition model to perform text feature recognition to obtain target text information in the image, including:

[0030] After inputting the target output result into the fully connected layer of the text recognition model for feature integration, the image classification result is output;

[0031] According to the image classification result, the target text information in the image is obtained.

[0032] The present invention also provides a device for processing mixed text information in an image, comprising:

[0033] An acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes horizontal text and vertical text;

[0034] A processing module is used to segment the image to be processed according to a preset segmentation granularity to obtain an image set including multiple segmented images; the image set includes multiple first segmented images containing horizontal text and multiple second segmented images containing vertical text; two-dimensional feature extraction is performed on the multiple first segmented images and multiple second segmented images to obtain a first feature data image set; the first feature data image set is converted into a three-dimensional feature to obtain a second feature data image set; three-dimensional feature extraction is performed on the images in the second feature data image set to obtain a target format image; the target format image is input into a text recognition unit of a trained text recognition model for text recognition to obtain target text information in the image, and the text recognition model is obtained by training a preset network model according to a preset training set.

[0035] The present invention also provides a computing device, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed.

[0036] The present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0037] The above solution of the present invention includes at least the following beneficial effects:

[0038] The above scheme of the present invention obtains an image to be processed, wherein the image to be processed contains horizontal text and vertical text; the image to be processed is segmented according to a preset segmentation granularity to obtain an image set including multiple segmented images; the image set includes multiple first segmented images containing horizontal text and multiple second segmented images containing vertical text; two-dimensional feature extraction is performed on the multiple first segmented images and multiple second segmented images to obtain a first feature data image set; the first feature data image set is converted into a three-dimensional feature to obtain a second feature data image set; three-dimensional feature extraction is performed on the images in the second feature data image set to obtain a target format image; the target format image is input into the text recognition unit of a trained text recognition model for text recognition to obtain the target text information in the image, and the text recognition model is obtained by training a preset network model according to a preset training set. The scheme of the present invention can accurately identify the mixed arrangement of horizontal and vertical text in an image, thereby improving the accuracy of text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a step diagram of a method for processing mixed text information in an image provided by an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a method for processing mixed text information in an image provided by an embodiment of the present invention;

[0041] Figure 3 It is a module schematic diagram of a device for processing mixed text information in an image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0043] like Figure 1 As shown, an embodiment of the present invention provides a method for processing mixed text information in an image, comprising:

[0044] Step 11, obtaining an image to be processed, wherein the image to be processed includes horizontal text and vertical text;

[0045] Step 12, segmenting the image to be processed according to a preset segmentation granularity to obtain an image set including a plurality of segmented images; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text;

[0046] Step 13, performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set;

[0047] Step 14, performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set;

[0048] Step 15, performing three-dimensional feature extraction on the images in the second feature data image set to obtain a target format image;

[0049] Step 16, inputting the target format image into the text recognition unit of the trained text recognition model for text recognition to obtain the target text information in the image, wherein the text recognition model is obtained by training a preset network model according to a preset training set.

[0050] In this embodiment, the horizontal and vertical text images are first divided into n square images according to a certain step length. At this time, the relationship between the images becomes separated. After a long text strip is divided into n square images, the chopped images are feature spliced ​​and arranged in time series through a 3D convolution model, which can be regarded as forming a video stream with n frames. The spliced ​​and arranged images are then input into a trained text recognition model for text recognition, and the output is classified to obtain the target text information. In this way, the horizontal text image and the vertical text image in the image can be input into the model at the same time, and when rotated, the horizontal text image and the vertical text image will not affect each other, thereby solving the problem that the horizontal text image and the vertical text image in the image will cause inconsistent features when rotated 90 degrees, and the sizes of the horizontal and vertical text boxes are different, and the two image sizes cannot be directly input into the text recognition unit of the text recognition model for processing.

[0051] The text recognition model of this embodiment includes an image processing unit and a text recognition unit, wherein the image processing unit has image processing functions such as two-dimensional feature extraction, three-dimensional feature extraction, and three-dimensional feature conversion, and the text recognition unit has the function of recognizing text in the image processed by the image processing unit.

[0052] The text information processing method of this embodiment can accurately identify the mixed arrangement of horizontal and vertical text in the picture when there is a mixed arrangement of horizontal and vertical text in the recognition scene, thereby improving the recognition accuracy.

[0053] like Figure 2 As shown, in an optional embodiment of the present invention, step 12 may include:

[0054] Step 121: process the image to be processed according to N The image is segmented with a segmentation granularity of N pixels to obtain an image set of multiple segmented images, where N is a positive integer.

[0055] In this embodiment, the original image is processed according to N N pixels are segmented into multiple square images, which achieves standardization and unification in shape dimension, facilitating the subsequent unified feature processing process. N represents the number of pixels in height and width of each small image after segmentation. These two parameters determine the size specifications of the small image.

[0056] The number of divisible values ​​along the height direction is: ,in Take the positive downward;

[0057] The number of divisible points along the width direction is: ,in Take positive downwards.

[0058] By traversing each pixel position of the initial image, a small image is cut out according to the size of N1*N2. Specifically, starting from the upper left corner of the initial image,

[0059] In the horizontal direction, N1 is used as the step length, and in the vertical direction, N2 is used as the step length. , j ) The starting and ending coordinates are , the image area with a width of N1 and a height of N2; where (0 ).

[0060] Optionally, the segmentation step value used in this embodiment is 8, and N (N1, N2) is usually 32, and the segmentation results in 57 32x32 pixel images. The specific step value is 8, that is, each time the image is slid forward by 8 pixels, and since the image size is 32x32 pixels, there will be serious overlap in the image. At this time, this overlap is used to generate stable information in the time t dimension, similar to a video playback, and then 3D convolution is used to capture the features in the time t dimension to support image classification.

[0061] In an optional embodiment of the present invention, step 13 may include:

[0062] Step 131, performing a two-dimensional convolution operation on the plurality of first segmented images to obtain a first two-dimensional convolution operation result;

[0063] Step 132, performing a two-dimensional convolution operation on the plurality of second segmented images to obtain a second two-dimensional convolution operation result;

[0064] Step 133: Obtain a first feature data image set according to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result.

[0065] In this embodiment, after obtaining a uniform set of square images, a 2D convolution operation is used to extract features of each square image, such as image edges, textures, local shapes, etc. After each square image undergoes such a 2D convolution operation, a set of feature representations that can represent the unique information of the image can be obtained, preparing for further feature integration in the future.

[0066] Specifically, through the formula , extract two-dimensional features of the square images in the image set, including features such as edges, textures, shapes, etc. The extracted feature data are combined into a corresponding image to obtain a first feature data image.

[0067] in, is the first feature data image set, x and y are the coordinates of the elements in the first feature data image set, which are obtained by flipping the convolution kernel and multiplying and adding it with the input local area; is the pixel value of the input image set (the image set consisting of the first segmented image and the second segmented image), and the position is represented by discrete coordinates Sure; is the weight value of the 2D (two-dimensional) convolution kernel, and the position is the offset relative to the center ; is the horizontal index, range ; is the vertical index, range , is an image set consisting of a plurality of first segmented images and second segmented images, is the weight value.

[0068] In an optional embodiment of the present invention, step 14 may include:

[0069] Step 141, arranging a plurality of feature images in the first feature data image set in chronological order to obtain an image time series data set;

[0070] Step 142, performing a three-dimensional convolution operation on the image time series data set to obtain a second feature data image set.

[0071] In this embodiment, 3D convolution is used to integrate the 2D features of the n square images obtained previously. The difference between 3D convolution and 2D convolution is that in addition to considering the length and width dimensions of the image, it also adds consideration of depth (here the depth can be understood as the dimension corresponding to the n square images).

[0072] Specifically, firstly, a plurality of feature images in the first feature data image set are arranged in time sequence to obtain an image time series data set. .

[0073] Furthermore, through the formula:

[0074] ;

[0075] ;

[0076] Perform three-dimensional feature conversion to obtain a second feature data image set.

[0077] in, is the second feature data image set, which is obtained by flipping the convolution kernel and multiplying and adding it with the input local area, and x, y, z are the coordinates of the elements in the second feature data image set; For image time series datasets, the positions are represented by discrete coordinates Sure; is the weight value of the 3D (three-dimensional) convolution kernel, D, H, and W are the dimensions of the corresponding convolution kernel in the channel, height, and width directions, respectively. D is the image depth, H is the image height, W is the image width, i is the depth index, j is the height index, and k is the width index.

[0078] After the entire image is divided into 57 32x32 small images at the beginning, there is a time series relationship between these small images. 2D convolution cannot capture the time series information well, but in 3D convolution, the 3D filter can move in all three directions (height, width, and channel of the image). By sliding and calculating the 3D convolution kernel in this three-dimensional data space, the time series characteristics of multiple square images can be reflected.

[0079] Here, 3D convolution can not only extract 2D features, but also integrate the temporal relationship. Note that the two stages of 2D feature extraction and 3D feature extraction are not separated or strictly sequential. You can also extract 3D features first and then 2D features, or cross-combine 2D convolution and 3D convolution. In the actual processing process, when you need to extract 2D features, set the D of the 3D convolution to 1, and it degenerates to 2D convolution. When you need to extract temporal features at the same time, set the D of the 3D convolution to > 1.

[0080] In an optional embodiment of the present invention, step 15 may include:

[0081] Step 151, performing three-dimensional convolution feature extraction on a plurality of feature images in the second feature data image set to obtain a three-dimensional feature data set;

[0082] Step 152, performing two-dimensional convolution feature extraction on the multiple feature images in the three-dimensional feature data set to obtain target format images arranged in chronological order.

[0083] In this embodiment, after the first feature data image undergoes 3D feature extraction, the second feature data image is in a specific 3D state. However, in order to be able to connect it to the text recognition model for subsequent processing, it is necessary to perform dimensional transformation again to transform the current feature state back to a sequence state to form a target format image in the sequence form that meets the input requirements of the text recognition model, so that the features extracted from the image can smoothly enter the text recognition model for the next step of feature extraction.

[0084] The second feature data images are all generated from original images at different time points, and there is a time sequence between them. Therefore, the second feature data images are spliced ​​into time series state data along the time dimension.

[0085] In an optional embodiment of the present invention, step 16 may include:

[0086] Step 161, inputting the target format image into an input unit of a trained text recognition model for processing to obtain a first output result;

[0087] Step 162, inputting the first output result into the hidden layer of the text recognition model for processing to obtain a second output result;

[0088] Step 163, inputting the second output result into the output layer of the text recognition model for processing to obtain a target output result;

[0089] Step 164: input the target output result into the fully connected layer of the text recognition model to perform text feature recognition to obtain the target text information in the image.

[0090] In this embodiment, the text recognition model is first trained. This embodiment adopts a recurrent neural network model. The recurrent neural network model is first trained using training set, validation set and test set data. The loss function uses a mean square error loss function. The training set includes cut directional image sequence data.

[0091] The training data is divided into several batches. For example, the batch size is set to 32, that is, 32 samples are used for training each time.

[0092] A batch of input data is passed into the recurrent neural network model, and the output of each layer is calculated in sequence according to the network structure. Finally, the output value of the model is obtained, and the loss value of the current batch of data is calculated using the loss function. The gradient of the loss function with respect to the model parameters is calculated through the back propagation algorithm.

[0093] In the recurrent neural network model, due to the existence of recurrent connections, back propagation will be carried out along the time step to calculate the gradient of the parameters at each time step. The optimizer updates the parameters of the model (such as weights and biases) based on the calculated gradient to reduce the value of the loss function; then the fully connected layer (FC) is connected to the output layer of the trained recurrent neural network model to obtain the text recognition model.

[0094] The target format image is input into the input unit of the trained text recognition model (i.e. the trained recurrent neural network model mentioned above) for processing. The input unit performs preliminary preprocessing on the input image data to meet the processing requirements of subsequent layers.

[0095] Then, the preprocessed image data is input into the hidden layer of the text recognition model. The hidden layer receives the first output result from the input unit. The hidden layer includes multiple neurons (or nodes) to extract and abstract the features of the input data.

[0096] Specifically, a neuron in the hidden layer will perform a weighted sum operation on the input data, that is, multiply each dimension of the input data by the corresponding connection weight, add them together, process them with the activation function, and then pass them to the next neuron. The data will be gradually transmitted and processed between these neurons, continuously extracting more advanced and abstract features to obtain the second output result;

[0097] The output layer receives the second output result, performs a weighted sum operation on the second output result, maps the abstract features to the final output space, generates an output representation related to the target text information based on the features extracted by the hidden layer, and obtains the target output result; the target output result is input to the fully connected layer, the output result is integrated and mapped, and finally the probability distribution vector output by the model is converted into specific text to complete the text recognition function. In an optional embodiment of the present invention, step 164 may include:

[0098] Step 1641, inputting the target output result into the fully connected layer of the text recognition model for feature integration, and then outputting the image classification result;

[0099] Step 1642: Obtain target text information in the image based on the image classification result

[0100] In this embodiment, the features obtained by processing the text recognition model are connected to the fully connected layer. The fully connected layer integrates and maps the previously extracted features and finally outputs the results for classification. By analyzing and judging these integrated features, it is determined which category the image belongs to, thereby achieving the task goal of the entire image classification.

[0101] During training, a character table is established in the fully connected layer. The length of the character table can be set to 5000-7000. Each position corresponds to a Chinese character, and a correspondence is established between the character and the position number of the character table. For example, "1" represents "space", "2" represents "a", and "100" represents "country". After processing the input image, the text recognition model outputs a vector. This vector contains the model's prediction information for the characters in the input image. Each dimension of the vector may correspond to the possibility of a character appearing in the character table or related feature information. For example, a high value of a dimension in the vector may mean that the model predicts that the character in the input image is more likely to correspond to the character associated with that dimension in the character table.

[0102] By using the previously set correspondence between characters and position numbers, the vector output by the model can be converted into specific Chinese characters (or other characters). Specifically, by analyzing the values ​​of each dimension in the vector, find the dimension with the largest value. The number corresponding to this dimension is the position number of a character in the character table. For example, if the dimension value corresponding to number 100 is the largest in the vector, according to the previously set correspondence relationship "100 represents country", it can be determined that the character recognized by the model is "country 0". In this way, the conversion from the model output vector to specific recognizable text is completed, thereby realizing the text recognition function.

[0103] The method for processing mixed text information in an image described in the above embodiment of the present invention solves the problem of reduced recognition accuracy of conventional model solutions for the situation of mixed horizontal and vertical text in an image. It can accurately identify the mixed situation of horizontal and vertical text in an image, thereby improving the recognition accuracy.

[0104] like Figure 3 As shown, the embodiment of the present invention further provides a device 30 for processing mixed text information in an image, comprising:

[0105] An acquisition module 31 is used to acquire an image to be processed, wherein the image to be processed includes horizontal text and vertical text;

[0106] The processing module 32 is used to segment the image to be processed according to a preset segmentation granularity to obtain an image set including multiple segmented images; the image set includes multiple first segmented images containing horizontal text and multiple second segmented images containing vertical text; two-dimensional feature extraction is performed on the multiple first segmented images and the multiple second segmented images to obtain a first feature data image set; the first feature data image set is converted into a three-dimensional feature to obtain a second feature data image set; three-dimensional feature extraction is performed on the images in the second feature data image set to obtain a target format image; the target format image is input into the text recognition unit of a trained text recognition model for text recognition to obtain target text information in the image, and the text recognition model is obtained by training a preset network model according to a preset training set.

[0107] Optionally, the image to be processed is segmented according to a preset segmentation granularity to obtain an image set including a plurality of segmented images, including:

[0108] The image to be processed is processed according to N The image is segmented with a segmentation granularity of N pixels to obtain an image set of multiple segmented images, where N is a positive integer.

[0109] Optionally, performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set includes:

[0110] Performing a two-dimensional convolution operation on the plurality of first segmented images to obtain a first two-dimensional convolution operation result;

[0111] Performing a two-dimensional convolution operation on the plurality of second segmented images to obtain a second two-dimensional convolution operation result;

[0112] A first feature data image set is obtained according to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result.

[0113] Optionally, performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set includes:

[0114] Arranging the plurality of feature images in the first feature data image set in chronological order to obtain an image time series data set;

[0115] A three-dimensional convolution operation is performed on the image time series data set to obtain a second feature data image set.

[0116] Optionally, performing three-dimensional feature extraction on the second feature data image set to obtain a target format image includes:

[0117] Perform three-dimensional convolution feature extraction on a plurality of feature images in the second feature data image set to obtain a three-dimensional feature data set;

[0118] Perform two-dimensional convolution feature extraction on multiple feature images in the three-dimensional feature data set to obtain target format images arranged in chronological order.

[0119] Optionally, inputting the target format image into a trained text recognition model to perform text recognition to obtain target text information in the image includes:

[0120] Inputting the target format image into an input unit of a trained text recognition model for processing to obtain a first output result;

[0121] Inputting the first output result into the hidden layer of the text recognition model for processing to obtain a second output result;

[0122] Inputting the second output result into the output layer of the text recognition model for processing to obtain a target output result;

[0123] The target output result is input into the fully connected layer of the text recognition model to perform text feature recognition to obtain the target text information in the image.

[0124] Optionally, inputting the target output result into the fully connected layer of the text recognition model to perform text feature recognition to obtain target text information in the image, including:

[0125] After inputting the target output result into the fully connected layer of the text recognition model for feature integration, the image classification result is output;

[0126] According to the image classification result, the target text information in the image is obtained.

[0127] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0128] The embodiment of the present invention further provides a computing device, including: a processor, a memory storing a computer program, and when the computer program is executed by the processor, the method described in the above embodiment is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0129] In an embodiment of the present invention, a computer-readable storage medium is further provided, which stores instructions, and when the instructions are executed on a computer, the computer executes the method described in the above embodiment. All implementations in the above method embodiment are applicable to this embodiment, and can also achieve the same technical effect.

[0130] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0136] In addition, it should be pointed out that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understandable that all or any steps or components of the method and apparatus of the present invention can be implemented in hardware, firmware, software or a combination thereof in any computing device (including processors, storage media, etc.) or a network of computing devices, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0137] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0138] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing mixed text information in an image, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes horizontal text and vertical text; Segmenting the image to be processed according to a preset segmentation granularity to obtain an image set including a plurality of segmented images; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text; Performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set; Performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set; Performing three-dimensional feature extraction on the images in the second feature data image set to obtain a target format image; The target format image is input into the text recognition unit of a trained text recognition model for text recognition to obtain target text information in the image. The text recognition model is obtained by training a preset network model according to a preset training set.

2. The method for processing mixed text information in an image according to claim 1, characterized in that: The image to be processed is segmented according to a preset segmentation granularity to obtain an image set including a plurality of segmented images, including: The image to be processed is processed according to N The image is segmented with a segmentation granularity of N pixels to obtain an image set of multiple segmented images, where N is a positive integer.

3. The method for processing mixed text information in an image according to claim 1, characterized in that: Performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set includes: Performing a two-dimensional convolution operation on the multiple first segmented images to obtain a first two-dimensional convolution operation result; Performing a two-dimensional convolution operation on the plurality of second segmented images to obtain a second two-dimensional convolution operation result; A first feature data image set is obtained according to the first two-dimensional convolution operation result and the second two-dimensional convolution operation result.

4. The method for processing mixed text information in an image according to claim 1, characterized in that: The first feature data image set is subjected to three-dimensional feature conversion to obtain a second feature data image set, comprising: Arranging the plurality of feature images in the first feature data image set in chronological order to obtain an image time series data set; A three-dimensional convolution operation is performed on the image time series data set to obtain a second feature data image set.

5. The method for processing mixed text information in an image according to claim 1, characterized in that: Extracting three-dimensional features from the second feature data image set to obtain a target format image includes: Performing three-dimensional convolution feature extraction on a plurality of feature images in the second feature data image set to obtain a three-dimensional feature data set; Perform two-dimensional convolution feature extraction on multiple feature images in the three-dimensional feature data set to obtain target format images arranged in chronological order.

6. The method for processing mixed text information in an image according to claim 1, characterized in that: Input the target format image into the trained text recognition model to perform text recognition to obtain the target text information in the image, including: Inputting the target format image into an input unit of a trained text recognition model for processing to obtain a first output result; Inputting the first output result into the hidden layer of the text recognition model for processing to obtain a second output result; Inputting the second output result into the output layer of the text recognition model for processing to obtain a target output result; The target output result is input into the fully connected layer of the text recognition model to perform text feature recognition to obtain the target text information in the image.

7. The method for processing mixed text information in an image according to claim 6, characterized in that: Inputting the target output result into the fully connected layer of the text recognition model to perform text feature recognition to obtain target text information in the image, including: After inputting the target output result into the fully connected layer of the text recognition model for feature integration, the image classification result is output; According to the image classification result, the target text information in the image is obtained.

8. A device for processing mixed text information in an image, characterized in that: include: An acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes horizontal text and vertical text; A processing module, configured to segment the image to be processed according to a preset segmentation granularity to obtain an image set including a plurality of segmented images; the image set includes a plurality of first segmented images containing horizontal text and a plurality of second segmented images containing vertical text; Performing two-dimensional feature extraction on the plurality of first segmented images and the plurality of second segmented images to obtain a first feature data image set; performing three-dimensional feature conversion on the first feature data image set to obtain a second feature data image set; Perform three-dimensional feature extraction on the images in the second feature data image set to obtain a target format image; input the target format image into the text recognition unit of a trained text recognition model to perform text recognition to obtain target text information in the image, and the text recognition model is obtained by training a preset network model according to a preset training set.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images