Text extraction method, device, equipment and storage medium

Abstract

The application provides a character extraction method, device and equipment and a storage medium, and belongs to the technical field of data processing and big data. The method comprises the following steps: in response to the start of a character extraction process, obtaining image information to be recognized, at least one piece of extraction task information, at least one text box and text box attributes corresponding to each text box, wherein the text box attributes comprise text to be extracted; generating an input tensor according to the text box attributes and the image information to be recognized; inputting the input tensor into a data extraction model corresponding to the character extraction process to obtain an output tensor output by the data extraction model; determining a target element sequence corresponding to target extraction task information in the output tensor according to the types of elements in the output tensor and the target extraction task information; and extracting target characters from each text to be extracted according to the target element sequence corresponding to the target extraction task information. The method solves the problem of low information extraction accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing and big data, and particularly relates to a character extraction method and device, equipment and a storage medium. BACKGROUND

[0002] Current image recognition has been applied in a wide range of fields, and character recognition is an important branch of image recognition. Through character recognition, specific types of information can be extracted from images.

[0003] At present, in the prior art, specific types of information can be obtained by manually framing a recognition position and then performing line-by-line recognition.

[0004] However, the inventors have found that the current information extraction method has at least the following technical problem: the accuracy of the current information extraction method is low. SUMMARY

[0005] The present application provides a character extraction method, device, equipment and storage medium to solve the problem of low accuracy of information extraction.

[0006] In a first aspect, the present application provides a character extraction method, comprising: in response to the start of a character extraction process, obtaining image information to be recognized, at least one extraction task information, at least one text box and text box attributes corresponding to each text box, wherein the text box attributes include text to be extracted. According to the text box attributes and the image information to be recognized, an input tensor is generated. The input tensor is input into a data extraction model corresponding to the character extraction process to obtain an output tensor output by the data extraction model, wherein the output tensor includes at least two elements. According to the type of elements in the output tensor and the target extraction task information, a target element sequence corresponding to the target extraction task information in the output tensor is determined, wherein the target extraction task information is any extraction task information. According to the target element sequence corresponding to the target extraction task information, the target character is extracted from each text to be extracted.

[0007] In a possible implementation, the text box attributes further include the text to be extracted, the text box coordinates, and the text box sequence number. Accordingly, according to the text box attributes and the image information to be recognized, the input tensor is generated, comprising: inputting the text length of the target text box corresponding to the text to be extracted into a preset conversion model to obtain a target sequence, wherein the target text box is any text box. According to the text box coordinates corresponding to the target text box and the image information to be recognized, the normalized target text box coordinates are calculated. The text box sequence number corresponding to the target text box is divided by the maximum text box sequence number to obtain a target position ratio. The target sequence, the normalized target text box coordinates and the target position ratio are spliced to obtain a target vector corresponding to the target text box. The target vectors corresponding to each text box are spliced to obtain the input tensor.

[0008] In a possible implementation, the target element sequence corresponding to the target extraction task information in the output tensor is determined according to the category of the element in the output tensor and the target extraction task information, including: determining, as a first reference element, an element in the output tensor that contains the target extraction task information and has a first element category. Determining, as a second reference element, an element in the output tensor that contains the target extraction task information and has a second element category. Extracting a first vector corresponding to the first reference element and a second vector corresponding to the second reference element in the output tensor. Removing the second reference element in the second vector to obtain a second vector after removing the mark. Concatenating the first vector and the second vector after removing the mark to obtain the target element sequence corresponding to the target extraction task information.

[0009] In a possible implementation, the target element sequence corresponding to the target extraction task information in the output tensor is determined according to the category of the element in the output tensor and the target extraction task information, including: determining, as a target reference element corresponding to the target extraction task information, an element in the output tensor that contains the target extraction task information and has a target element category. Determining, as the target element sequence corresponding to the target extraction task information, an element in the output tensor between the target reference element and another reference element. Or, determining, as the target element sequence corresponding to the target extraction task information, an element in the output tensor between the target reference element and a last element in the output tensor.

[0010] In a possible implementation, the target character is extracted from each text to be extracted according to the target element sequence corresponding to the target extraction task information, including: determining, as a target element, an element in the target element sequence corresponding to the target extraction task information that has a preset element value. Determining, as a target position, a position corresponding to the target element. Determining, as a target character, a character in the text to be extracted corresponding to the target position. Reading the target character in the text to be extracted.

[0011] In a possible implementation, before the character extraction process starts, the method further includes: obtaining training image information corresponding to each training image, a training text box, a training text box attribute corresponding to each training text box, and training task information. According to the training text box attribute and the training task information, the coincidence degree corresponding to each training task information is calculated. According to the coincidence degree and the training text box attribute, the coincidence degree mark corresponding to each training text box is generated. According to the coincidence degree mark corresponding to each training text box, the training text box attribute, and the preset standard character, the character coincidence degree mark of each training character corresponding to each training text box is generated. According to the coincidence degree mark corresponding to each training text box and the character coincidence degree mark, a standard output tensor is generated. According to the training text box attribute and the training image information, a standard input tensor is generated. The training model to be trained is trained by using the standard input tensor and the standard output tensor, and a data extraction model is obtained.

[0012] In a second aspect, the present application provides a text extraction device, comprising: a data acquisition module, configured to acquire image information to be recognized, at least one piece of extraction task information, at least one text box, and text box attributes corresponding to each text box in response to the start of a text extraction process, wherein the text box attributes comprise text to be extracted; a tensor generation module, configured to generate an input tensor according to the text box attributes and the image information to be recognized; a tensor input module, configured to input the input tensor into a data extraction model corresponding to the text extraction process to obtain an output tensor output by the data extraction model, wherein the output tensor comprises at least two elements; a sequence determination module, configured to determine a target element sequence corresponding to target extraction task information in the output tensor according to the types of elements in the output tensor and the target extraction task information, wherein the target extraction task information is any extraction task information; and a text extraction module, configured to extract target text from each text to be extracted according to the target element sequence corresponding to the target extraction task information.

[0013] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory connected to the processor in communication; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so that the processor executes the text extraction method described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and the computer execution instructions are used to implement the text extraction method described in the first aspect when executed by a processor.

[0015] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the text extraction method described in the first aspect.

[0016] The text extraction method, device, equipment and storage medium provided by the present application can increase the accuracy of text extraction by acquiring image information to be recognized, at least one piece of extraction task information, at least one text box, and text box attributes corresponding to each text box, obtaining an input tensor from the text box attributes and the image information to be recognized, inputting the input vector into a data extraction model to obtain an output vector, obtaining a target element sequence required from the element types of each element in the output vector and the extraction task information, and extracting text in the text to be extracted from the target element sequence. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0018] Figure 1 The application scenario of the text extraction method provided by the present application is shown in the following figure.

[0019] Figure 2 A to-be-identified image information schematic diagram provided for an embodiment of the present application is shown in the following figure:

[0020] Figure 3 A text extraction method flowchart provided for an embodiment of the present application is shown in the following figure:

[0021] Figure 4 A text extraction device structure schematic diagram provided for an embodiment of the present application is shown in the following figure:

[0022] Figure 5 A structure schematic diagram of an electronic device provided for an embodiment of the present application is shown in the following figure.

[0023] The above figures have shown the specific embodiments of the present application, which will be described in more detail hereinafter. These figures and the written description are not intended to limit the scope of the present application concept in any way, but to illustrate the present application concept to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0024] The exemplary embodiments will be described in detail herein with reference to the attached drawings. Unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The following exemplary embodiments are described in order to provide a thorough understanding of the present application concept. However, the present application concept can be practiced by other than the exemplary embodiments as described. In other words, the following description does not indicate that the present application concept can be implemented only in the following exemplary embodiments.

[0025] Current image recognition technology has been widely applied in various fields. As an important branch of image recognition, text recognition aims to extract specific types of information from complex images. These information may include text content, text layout, etc. Through the application of text recognition technology, people can more conveniently obtain information in images, and thus better utilize the information.

[0026] Currently, text recognition technology mainly adopts the way of manually framing the recognition position, and then recognizing the text content line by line. This way is relatively cumbersome and prone to errors, with low accuracy. Therefore, it is necessary to develop a more efficient and accurate text recognition technology to improve this situation.

[0027] In view of the above technical problems, the inventors propose the following technical concept: by obtaining the to-be-identified image information corresponding to the to-be-identified image, the text box and the text box attribute corresponding to each text box, generating the input tensor corresponding to each text box attribute and the to-be-identified image information, inputting the input tensor into the data extraction model to obtain the output tensor, and determining the position of the text to be extracted in all texts in the image according to the content of the output tensor, the corresponding text is extracted.

[0028] Figure 1 The application scenario of the text extraction method provided by the embodiment of the present application is shown in the figure. As shown in the figure, Figure 1 In this scenario, it includes a server 101, an image acquisition device 102, and a control terminal 103.

[0029] The server 101 can include one server or a cluster of multiple servers with more powerful processing capability and higher security to implement, and in possible cases, a computer, a notebook computer, etc. with stronger computing capability can also be used for replacement.

[0030] The image acquisition device 102 can be an image acquisition device such as a camera, a mobile phone, etc., or a device for receiving images from the outside world such as a computer, a tablet computer, a database, etc.

[0031] The control terminal 103 can include a computer, a server, a tablet, a mobile phone, a Personal Digital Assistant (PDA), a notebook, etc., which can input data and send instructions.

[0032] In the specific implementation process, the server 101 is configured to acquire a to-be-recognized image and related information of the to-be-recognized image from the image acquisition device 102, and obtain the text in the to-be-recognized image from the data. The server 101 can also receive the to-be-recognized image related information sent by the control terminal 103, so as to perform text extraction.

[0033] The control terminal 103 is configured to send a control instruction to the server 101, so that the server 101 starts the text extraction process.

[0034] The connection between the server 101, the image acquisition device 102, and the control terminal 103 can be a communication connection.

[0035] It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the text extraction method. In other possible embodiments of the present application, the above architecture can include more or fewer components than the figure, or combine certain components, or split certain components, or different component arrangement, which can be determined according to the actual application scenario, and is not limited herein. Figure 1 The shown scenario can be realized by hardware, software, or a combination of software and hardware.

[0036] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail in the following specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in detail in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0037] Figure 2 The image information to be identified provided for the embodiment of the present application is shown in the figure. As shown in the figure, the image to be identified can be a table, but can also be other pictures. Figure 2 Figure 2 The text surrounded by the dashed line in the figure is the content to be extracted (extraction task). The extraction task in the current figure includes the product name, the specification model, and the specific data thereof.

[0038] Figure 3 The flowchart of the text extraction method provided for the embodiment of the present application is shown in the figure. The execution subject of the embodiment of the present application can be a server in the figure, or can be a computer and / or a mobile phone, and the embodiment is not particularly limited thereto. As shown in the figure, the method comprises the following steps. Figure 1 Figure 3

[0039] S301: In response to the start of the text extraction flow, obtain the image information to be identified, at least one piece of extraction task information, at least one text box, and the text box attribute corresponding to each text box, wherein the text box attribute comprises the text to be extracted.

[0040] In this step, the text extraction flow can be started to execute after receiving the instruction of the control terminal or receiving the new image to be identified, and can also be generated according to the extraction task information. The image information to be identified can include the length value of the image. The extraction task information can be included in the text extraction flow, or can be sent by the control terminal. The extraction task information can also be replaced by an identifier or a code. The text box can be a text box obtained by recognizing the text in the image, and can be obtained by pre-recognizing the characters using a character recognition model. The text box attribute corresponding to the text box can also be output by the character recognition model, and can include at least one of the serial number, the position of the text box, the text to be extracted, and the confidence. One picture can have multiple text boxes.

[0041] Referring again to the figure, the text box attribute corresponding to the text box is, for example: Figure 2

[0042] “1. (100, 100, 200, 200); item number; 0.95;

[0043] 2. (200, 100, 200, 300); product number; 0.8;

[0044] 3. ……”

[0045] In the example, the first item is the serial number, the second item is the position of the text box, the third item is the text to be extracted, and the fourth item is the confidence.

[0046] S302: Generate an input tensor according to the text box attribute and the image information to be identified.

[0047] ​​​​In this step, a vector corresponding to the text box attribute can be generated according to any text box attribute and image information to be recognized, vectors corresponding to various text boxes are spliced, and an input tensor is obtained.

[0048] The vector generation manner can include dividing an image size in the image information to be recognized by a preset standard image size to obtain a size ratio, multiplying the size ratio by a position of the text box in the text box attribute to obtain a transformed text box position, and converting the text to be extracted into a number corresponding to the text by using a preset text conversion model. The sequence number, the transformed text box position, and the number corresponding to the text are spliced to obtain the vector.

[0049] S303: The input tensor is input into a data extraction model corresponding to the text extraction process to obtain an output tensor output by the data extraction model, wherein the output tensor includes at least two elements.

[0050] In this step, the data extraction model can be run and the input tensor can be read, or the input tensor can be copied into the data extraction model. Different text extraction processes can correspond to different data extraction models, so that the model is targeted.

[0051] The types of elements in the output tensor can be determined according to the values of the elements. If there are two different values, there are two elements, and if there are three different values, there are two elements.

[0052] S304: According to the types of elements in the output tensor and target extraction task information, a target element sequence corresponding to the target extraction task information in the output tensor is determined, wherein the target extraction task information is any extraction task information.

[0053] In this step, elements containing extraction task information in the output tensor can be found, these elements are taken as marker elements, and all elements between the first marker element and the last marker element corresponding to the same extraction task information are taken as a target element sequence corresponding to the extraction task information.

[0054] In one possible implementation, if the first marker element and the last marker element corresponding to the same extraction task information contain multiple marker elements, the intermediate marker elements are removed, and elements with a preset value can also be removed, and finally the target element sequence is obtained.

[0055] S305: According to the target element sequence corresponding to the target extraction task information, target text is extracted from each text to be extracted.

[0056] In this step, elements in the target element sequence with element values equal to the target preset value can be determined as target elements, and the position of each target element among all elements is recorded. This value is used as the order value of the target element, and the text with the same order value in the text to be extracted is retrieved to obtain the target text.

[0057] As can be seen from the description of the above embodiments, in the embodiments of the present application, by obtaining the information of the image to be recognized, at least one extraction task information, at least one text box, and the text box attributes corresponding to each text box, an input tensor is obtained from the text box attributes and the information of the image to be recognized. The input vector is input into the data extraction model to obtain an output vector. The target element sequence needed is obtained from the element types of each element in the output vector and the extraction task information, and the text in the text to be extracted is retrieved from the target element sequence, thereby improving the accuracy of text extraction.

[0058] In a possible implementation, the text box attributes further include the text to be extracted, the text box coordinates, and the text box serial number.

[0059] Among them, the text box coordinates can be calibrated using the horizontal and vertical coordinates of the upper left corner and the horizontal and vertical coordinates of the lower right corner of the text box. The text to be extracted and the text box serial number are similar to those described in the above embodiments and will not be elaborated here.

[0060] Correspondingly, in the above step S302, generating an input tensor according to each text box attribute and the information of the image to be recognized includes:

[0061] S3021: Input the text length of the text to be extracted corresponding to the target text box into a preset conversion model to obtain a target sequence, where the target text box is any text box.

[0062] In this step, the text length of the text to be extracted can be the number of characters in the text to be extracted. The preset conversion model can use any pre-trained language model based on the transformers architecture, such as the BERT model, the ALBERT model, etc. Assuming that bert is used as our pre-trained language model, the input of the BERT model is a set of sequences, and the output is also a set of sequences. For example: Input: "[Product Name, Specification Model]". Then the output will be: " <cls>s0, s1, s2, s3, s4, s5, s6, s7, s8, <sep>]”. Each of the vectors in the output sequence is a 768-dimensional vector, where <cls>The sentence vector represents the meaning of the sentence and can be used in a text classification task; s0 to s8 represent the word vectors corresponding to each word and can be used in a recognition task. <sep>The symbol represents a sub-sentence. For each text to be extracted, if the text to be extracted has N words, N+1 vectors of 768 dimensions are extracted, where +1 is the vector of CLS (classification). If the number of words in each text to be extracted is different, a preset program or algorithm can be used to fill the values of the obtained vectors with lower dimensions to obtain vectors with the same dimensions.

[0063] S3022: Calculate the regularized target text box coordinates according to the text box coordinates corresponding to the target text box and the image information to be recognized.

[0064] In this step, the text box coordinates can include two sets of horizontal coordinates and vertical coordinates of the diagonals, and the image information to be recognized includes the length of the horizontal axis and the width of the vertical axis of the image to be recognized. Correspondingly, the regularized target text box coordinates can be obtained by dividing the horizontal coordinates of the text box by the length of the corresponding image to be recognized and dividing the vertical coordinates of the text box by the width of the corresponding image to be recognized.

[0065] 3023: Divide the text box serial number corresponding to the target text box by the maximum text box serial number to obtain the target position ratio.

[0066] In this step, each text box has a corresponding serial number, which is the serial number of the text box when the text box is initially obtained according to the position of the text box from top left to bottom right.

[0067] For example, the text box serial number corresponding to the target text box is 30, and the maximum text box serial number is 120, so the obtained target position ratio is 0.25. For another example, the text box serial number corresponding to the target text box is 20, and the maximum text box serial number is 200, so the obtained target position ratio is 0.1.

[0068] S3024: Concatenate the target sequence, the regularized target text box coordinates and the target position ratio to obtain the target vector corresponding to the target text box.

[0069] In this step, the concatenation can be performed by concatenating the target sequence, the regularized target text box coordinates and the target position ratio in a preset order. If the text to be extracted has N words, there are (N+1) x 768 dimensions of text features, 4 dimensions of position features and 1 dimension of number features, and the target vector obtained after concatenation is (N+1) x 773 dimensions.

[0070] Correspondingly, the dimension of the vector in the output tensor is (N+1) x M, where M represents the total number of possible element categories corresponding to the extraction task information, for example, if there are two extraction task information, namely commodity name and commodity specification model, the element values include "B-commodity name, I-commodity name, B-commodity specification model, I-commodity specification model, O, B, I", which are 7 categories, where B represents the first preset character in the following text, I represents the second preset character, and O represents the third preset character.

[0071] S3025: Splicing the target vectors corresponding to each text box to obtain an input tensor.

[0072] In this step, each target text box can be spliced in the order of the sequence number corresponding to each text box to obtain an input tensor.

[0073] From the description of the above embodiments, it can be known that, by inputting the text length of the text to be extracted into a preset conversion model, the embodiments of the present application obtain a target sequence corresponding to a target text box, regularize the text box coordinates corresponding to the target text box according to the image information to be recognized, divide the text box sequence number corresponding to the target text box by the maximum text box sequence number to obtain a target position ratio, and splice the three values obtained to obtain an input vector, so as to realize the comprehensive data of three dimensions, so that the recognition result will not be offset due to the different sizes of the pictures, and the subsequent text extraction is more accurate.

[0074] In a possible implementation, in the step S304, according to the element categories in the output tensor and the target extraction task information, a target element sequence corresponding to the target extraction task information in the output tensor is determined, including:

[0075] S3041A: The element containing the target extraction task information and having a first element category in the output tensor is determined as a first reference element, where the target extraction task information is any extraction task information.

[0076] This step is explained together in the following step S3042A.

[0077] S3042A: The element containing the target extraction task information and having a second element category in the output tensor is determined as a second reference element.

[0078] In this step, the first element category and the second element category are preset by the staff, which can be determined before model training, or the element containing the first preset character can be determined as the first element category, and the element containing the second preset character can be determined as the second element category, and the first preset character and the second preset character can also be preset by the staff before model training. There can also be a third preset character for identifying a third element category, and the third element category represents an element irrelevant to the extraction task information.

[0079] For example, the current target extraction task information is "quantity", the first preset character is "B", and the current element is "B-quantity", then the element is determined as the first reference element. For another example, the current target extraction task information is "destination", the second preset character is "I", and the current element is "I-destination", then the element is determined as the second reference element. The first preset character and the second preset character can be or can adopt other values preset by the staff.

[0080] S3043A: Extract the first vector corresponding to the first reference element and the second vector corresponding to the second reference element in the output tensor.

[0081] In this step, each layer of the output tensor can be regarded as a vector, and the elements after the first reference element in the vector can be collectively regarded as the first vector. The second vector is the same.

[0082] S3044A: Delete the second reference element in the second vector to obtain the second vector after the mark is removed.

[0083] In this step, for example, the current second vector is "I-goods name, O, I, O, O, I, I, I, I, O", and "I-goods name" is removed to obtain the second vector after the mark is removed "O, I, O, O, I, I, I, I, O". For another example, the current second vector is "Y-goods specifications, I, I, I, I, O, O, I, I, O", and "Y-goods specifications" is removed to obtain the second vector after the mark is removed "I, I, I, I, O, O, I, I, O".

[0084] S3045A: Concatenate the first vector and the second vector after the mark is removed to obtain the target element sequence corresponding to the target extraction task information.

[0085] In this step, for example, the first vector is "B-goods name, O, I, O, O, I, I, I, I, O" and the second vector after the mark is removed is "I, I, I, I, O, O, I, I, O", and the target element sequence obtained by concatenation is "B-goods name, O, I, O, O, I, I, I, I, O, I, I, I, I, O, O, I, I, O". For another example, the first vector is "Y-goods model, I, I, I, I, O, I, I, O" and the second vector after the mark is removed is "O, O, I, I, I, I, O, O, I", and the target element sequence obtained by concatenation is "Y-goods model, I, I, I, I, O, I, I, O, O, O, I, I, I, I, O, O, I"

[0086] As can be known from the description of the above embodiments, the embodiments of the present application determine the reference element in the output tensor by the target extraction task information and the element category, extract the first vector and the second vector in the output tensor from the reference element, splice the first vector after deleting the second reference element in the second vector, and obtain the target element sequence corresponding to the target extraction task information, so as to realize obtaining the element sequence corresponding to each extraction task information from the tensor output by the model, and facilitate subsequent extraction of the required text using the element sequence

[0087] In a possible implementation, in the step S304, the target element sequence corresponding to the target extraction task information in the output tensor is determined according to the element category in the output tensor and the target extraction task information, including:

[0088] S3041B: determining the element in the output tensor as the target reference element corresponding to the target extraction task information, if the element includes the target extraction task information and the element category is the target element category.

[0089] In this step, for example, there are three element categories, A, B and C, if the target element category is A, and if there is an element with category A and including the target extraction task information, the element is determined as the target reference element. For another example, the current target extraction task information is "product name", there are three element categories "O", "B" and "I", there are seven different element values "B-product name", "I-product name", "B-product specification model", "I-product specification model", "O", "B" and "I", and the target element category is "B", then "B-product name" is determined as the target reference element because it belongs to the target element category B and includes the target extraction task information.

[0090] The target extraction task information can be replaced by the identifier of the extraction task.

[0091] In a possible implementation, if there are multiple target reference elements, one of the target reference elements can be retained, and the other target reference elements can be removed.

[0092] S3042B: determining the elements between the target reference element and another reference element in the output tensor as the target element sequence corresponding to the target extraction task information, or determining the elements from the target reference element to the last element in the output tensor as the target element sequence corresponding to the target extraction task information.

[0093] In this step, the other reference element can be the reference element corresponding to other extraction task information except the target extraction task information. In the case where there is no other reference element, all the elements after the target reference element are directly determined as the target element sequence corresponding to the target extraction task information.

[0094] For example, if the current output tensor includes the target reference element "B-specification model" and another reference element "B-quantity", and "B-specification model" and "B-quantity" include "O, I, I, O, I, I, I, O" therebetween, "B-specification model, O, I, I, O, I, I, I, O" is determined as the target element sequence corresponding to the target extraction task information. For another example, if the current output tensor includes the target reference element "B-specification model" and another reference element "B-destination", and "B-specification model" and "B-destination" include "O, I, O, O, I, O, I, I" therebetween, "B-specification model, O, I, O, O, I, O, I, I" is determined as the target element sequence corresponding to the target extraction task information.

[0095] As can be seen from the description of the above embodiments, the embodiments of the present application obtain the reference elements in the output tensor by identifying the extraction task information and the element category in the output tensor, and extract the elements between the two reference elements in the output tensor as the target element sequence, so as to obtain the element sequence corresponding to each extraction task information from the tensor output by the model, which facilitates subsequent extraction of the required text using the element sequence.

[0096] In a possible implementation, in the step S305, the target text is extracted from each text to be extracted according to the target element sequence corresponding to the target extraction task information, including:

[0097] S3051: An element with a preset element value in the target element sequence corresponding to the target extraction task information is determined as a target element.

[0098] In this step, the preset element value can be a value preset by a staff member.

[0099] For example, if the current target element sequence is "B, I, I, B, I, I, O, O, I", and "B" and "I" are preset element values, the first to sixth elements and the ninth element in the target element sequence are target elements. For another example, if the current target element sequence is "X, Y, Y, Y, Y, Z, X, X", and "X" and "Y" are preset element values, the first to fifth elements and the seventh and eighth elements in the target element sequence are target elements.

[0100] S3052: A position corresponding to the target element is determined as a target position.

[0101] In this step, the position corresponding to the target element can be the ordinal position of the target element in the target element sequence. Since the input vector is generated from each text box and the image information to be recognized, each layer vector in the input tensor can correspond to a text box, each layer in the output tensor can also correspond to each text box, and the target position can also be the element position in the vector corresponding to the text box.

[0102] For example, in the first example of step S2051 above, 1 to 6 and 9 are target positions; in the second example, 1 to 5, 7, and 8 are target positions. As another example, if the first element of the current target element sequence is the 20th element of the corresponding vector in the output tensor, and the first to fifth elements of the target element sequence are target elements, then the target positions are 20 to 25. Yet another example, if the first element of the current target element sequence is the 100th element of the corresponding vector in the output tensor, and the first to fifth, seventh, and eighth elements of the target element sequence are target elements, then the target positions are 100 to 105, 107, and 108.

[0103] S3053: The text corresponding to the target position in the text to be extracted is identified as the target text.

[0104] In this step, you can count the order in which each character appears in the text to be extracted, and identify the characters whose appearance order is the same as the target position as the target characters.

[0105] For example, if the text to be extracted is "Product Name, Specifications 008 Bowls 8 Plates Set", and the corresponding target positions are 1 to 9 and 12 to 17, then "Product Name, Specifications 8 Bowls 8 Plates Set" is identified as the target text. As another example, if the text to be extracted is "09 Quantity 18000 Items", and the corresponding target positions are 3 to 9, then "Quantity 18000" is identified as the target text.

[0106] S3054: Read the target text from the text to be extracted.

[0107] In this step, the text corresponding to the target location can be read from the text to be extracted.

[0108] As can be seen from the description of the above embodiments, the embodiments of this application use elements with preset element values ​​in the target element sequence as target elements and determine the position corresponding to the target element as the target position, thereby extracting the target text corresponding to the target position from the text to be extracted, achieving accurate target text from the text to be extracted, and increasing the accuracy of text recognition.

[0109] In one possible implementation, before the text extraction process begins in step S301 above, the following is also included:

[0110] S310: Obtain the training image information, training text boxes, training text box attributes, and training task information corresponding to each training image.

[0111] In this step, the training image information corresponding to each training image, the training text box, the training text box attribute corresponding to each training text box, and the training task information can be read from the database or input by the staff. The training image information can include the size of the training image, and the training text box can be a text box identified by a character recognition model. The training text box attribute can include training text corresponding to the training text box, training text box coordinates, training text box serial number, etc. The training task information can be pre-labeled by the staff.

[0112] S311: According to the training text box attribute and the training task information, the coincidence degree corresponding to each training task information is calculated.

[0113] In this step, the coincidence degree corresponding to each training task information can be calculated according to the coordinates of the training text box and the coordinates in the training task information. The calculated coincidence degree can be written into the training task information, or it can be stored separately as a kind of data. When recording data, it can also be recorded which training text box attribute coincides with which training task information.

[0114] The coordinates of the training text box and the coordinates of the training task information are represented by the upper left corner coordinates and the lower right corner coordinates, respectively, as "[x1, y1, x2, y2]", "[x3, y3, x4, y4]". The calculation formula of the coincidence degree can be as follows:

[0115] IOU=(Intersection Area) / (Union Area)

[0116] Where Intersection Area is the intersection area, and Union Area is the union area.

[0117] The intersection area size calculation method is as follows:

[0118] Intersection Area=(min(x2, x4)-max(x1, x3))×(min(y2, y4)-max(y1, y3))

[0119] Where min represents the minimum value, and max represents the maximum value. For example, "min(x2, x4)" represents the minimum value of x2 and x4.

[0120] The union area size calculation method is as follows:

[0121] Union Area=(x2-x1)(y2-y1)+(x4-x3)(y4-y3)-Intersection Area.

[0122] Where the symbols have the same meaning as in other formulas.

[0123] S312: generating the coincidence degree labels corresponding to each training text box according to the coincidence degrees and the training text box attributes.

[0124] In this step, the training text box sequence number in the training text box attribute can be read, the training text box with the largest coincidence degree greater than the preset coincidence degree threshold and the smallest training text box sequence number is determined as the starting text box, the first label is added to the starting text box, the second label is added to the remaining training text boxes with the coincidence degree greater than the preset coincidence degree threshold, and the third label is added to the training text boxes with the coincidence degree less than or equal to the preset coincidence degree threshold. The first label and the second label correspond to the first element category and the second element category respectively, and the third label corresponds to the irrelevant element. The first label can include the preset character and the training task information with the coincidence degree greater than the preset coincidence degree threshold.

[0125] The to-be-changed label can also be the second element category identifier in the above embodiments.

[0126] S313: generating the character coincidence degree labels of each training character corresponding to each training text box according to the coincidence degree labels corresponding to each training text box, the training text box attributes, and the preset standard character.

[0127] In this step, if the coincidence degree label corresponding to any training text box includes the first label or the second label, the training character in the training text box attribute is read, the regular expression is used to match the training character and the preset standard character, and the first label, the second label, and the third label are used for secondary labeling for each character matched.

[0128] The labeling method of the character is similar to the above step S312. The first matched character is labeled with the first label, the Nth matched character is labeled with the second label, and the character without matching is labeled with the third label, where N is an integer greater than 1.

[0129] For example, the coincidence degree label corresponding to the training text box is "B-specification model" including "B" as the first label, the training character in the training text box is "specification model, clothing set", and the preset standard character is "clothing set". The character coincidence degree label is "O, O, O, O, O, B, I, I, I", where "O" is the third label and "I" is the second label.

[0130] S314: generating the standard output tensor according to the coincidence degree labels corresponding to each training text box and the character coincidence degree labels.

[0131] In this step, the coincidence degree label corresponding to the target training text box can be spliced with each character coincidence degree label to obtain a standard output vector corresponding to the target training text box. The splicing manner can be to write each character coincidence degree label behind the coincidence degree label corresponding to the target training text box. The standard output vectors corresponding to each training text box are spliced according to the sequence numbers of the training text boxes to obtain a standard output tensor.

[0132] For example, the coincidence degree label corresponding to the target training text box is "[B- commodity name]", and the character coincidence degree label corresponding to the target training text box is "O, O, B, I, I, I, I, O". The spliced standard output vector is "[B- commodity name], O, O, B, I, I, I, I, O".

[0133] S315: generating a standard input tensor according to each training text box attribute and training image information.

[0134] This step is similar to the above step S302, which will not be described here.

[0135] S316: training the to-be-trained model using the standard input tensor and the standard output tensor to obtain a data extraction model.

[0136] In this step, the standard input tensor can be input into the to-be-trained model to obtain a to-be-determined tensor output by the to-be-trained model, the error between the to-be-determined tensor and the standard output tensor is calculated, the to-be-trained model is adjusted according to the error, and the step of inputting the standard input tensor into the to-be-trained model to obtain the error is executed again until the error is less than a preset value.

[0137] The to-be-trained model can use an RNN (Recurrent Neural Network) model, an LSTM (Long Short-Term Memory) model, a transformer model, etc. The last layer can use softmax for activation, but retains logits, and the positive and negative of logits are used to determine whether it is a certain classification. When constructing a loss function, the logits of each sub-sequence in the sequence can be used as a weight to calculate the loss of the sequence. <cls>An additional loss is constructed.

[0138] When training the model, the error corresponding to the character overlap label can also be ignored first, and the model part corresponding to the training text box is optimized preferentially, and then the error corresponding to the character overlap label is calculated after the model part corresponding to the training text box converges, thereby increasing the model training speed.

[0139] From the description of the above embodiments, the embodiments of the present application add an overlap label to the training text box by calculating the overlap of the training text box attribute and the training task information, and calculate the overlap of the corresponding training text box attribute and the preset standard character for the training text box with high overlap, obtain the character corresponding overlap label, and superimpose the overlap label corresponding to each training text box and the character overlap label to obtain the standard output tensor. The data extraction model is obtained by training the to-be-trained model using the standard input tensor and the standard output tensor, which facilitates subsequent text extraction using the data extraction model.

[0140] For the above steps S313, S314, additional examples are provided as follows: for example, for training task information 1, there is only one training text box with overlap.

[0141] Preset standard character: "product name: granite products"

[0142] Training text in the training text box attribute: "68029390's granite products"

[0143] The training text in the training text box attribute is directly matched, and the label result is spliced to obtain the standard output vector: "[B-Product Name, B-Product Specification Model], O, O, O, O, O, O, O, O, O, B, I, I, I, I".

[0144] For example, for training task information 2, including the second label "[I-Product Specification Model]", there are two training texts in the training text box attribute with overlap.

[0145] Training task information 2: product specification model: garden engineering | granite | machine cutting fire | 20*20*11cm training text in the training text box attribute:

[0146] Training text 18 in the training text box attribute: "garden engineering | granite | machine cutting fire".

[0147] Training text 19 in the training text box attribute: "burning | 20*20*11cm, maximum surface".

[0148] The vectors are concatenated and matched to obtain the standard output vector corresponding to the training text 18 in the training text box attribute: "[I-product specification model],B,I,I,I,I,I,I,I,I,I,I,I", and the standard output vector corresponding to the training text 19 in the training text box attribute is: "[I-product specification model],I,I,I,I,I,I,I,I,I,I,I,O,O,O,O,O,O,O.O.

[0149] In one possible implementation, the text box properties include the text box number.

[0150] Accordingly, in step S312 above, based on the overlap degree and the attributes of the training text boxes, an overlap degree marker is generated for each training text box, including:

[0151] S3121: Calculate the overlap between each training text box and the training task information in ascending order of text box number.

[0152] S3122: The first corresponding training text box with an overlap greater than a preset overlap threshold is determined as the first target training text box.

[0153] S3123: Add the first overlap marker to the first target text box.

[0154] S3124: The training text boxes outside the target training text box with an overlap degree greater than the preset overlap degree threshold are identified as the second target training text boxes.

[0155] S3125: Add a second overlap marker to the training text box for the second target.

[0156] S3126: The training text boxes whose overlap degree is less than or equal to the preset overlap degree threshold are identified as the third target training text boxes.

[0157] S3127: Add a third overlap marker to the training text box for the third target.

[0158] Figure 4 This is a schematic diagram of the text extraction device provided in an embodiment of this application. Figure 3 As shown, the text extraction device 400 includes: a data acquisition module 401, a tensor generation module 402, a tensor input module 403, a sequence determination module 404, and a text extraction module 405.

[0159] The data acquisition module 401 is used to acquire, in response to the start of the text extraction process, image information to be recognized, at least one extraction task information, at least one text box and text box attributes corresponding to each text box, wherein the text box attributes include the text to be extracted.

[0160] The tensor generation module 402 is configured to generate an input tensor according to the text box attributes and the image information to be recognized.

[0161] The tensor input module 403 is configured to input the input tensor into a data extraction model corresponding to the character extraction process to obtain an output tensor output by the data extraction model, wherein the output tensor includes at least two elements.

[0162] The sequence determination module 404 is configured to determine a target element sequence corresponding to target extraction task information in the output tensor according to the types of elements in the output tensor and the target extraction task information, wherein the target extraction task information is any extraction task information.

[0163] The character extraction module 405 is configured to extract target characters from each text to be extracted according to the target element sequence corresponding to the target extraction task information.

[0164] The apparatus provided in the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again in the embodiment.

[0165] In a possible implementation, the text box attributes further include the text to be extracted, the text box coordinates, and the text box serial number.

[0166] The tensor generation module 402 is specifically configured to input a text length of the text to be extracted corresponding to a target text box into a preset conversion model to obtain a target sequence, wherein the target text box is any text box. Regularize the text box coordinates corresponding to the target text box according to the image information to be recognized. Divide the text box serial number corresponding to the target text box by the maximum text box serial number to obtain a target position ratio. Concatenate the target sequence, the regularized text box coordinates corresponding to the target text box, and the target position ratio to obtain a target vector corresponding to the target text box. Concatenate the target vectors corresponding to all the text boxes to obtain the input tensor.

[0167] The apparatus provided in the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again in the embodiment.

[0168] In a possible implementation, the sequence determination module 404 is specifically configured to determine an element in the output tensor as a first reference element, wherein the element contains the target extraction task information and has a first element type. Determine another element in the output tensor as a second reference element, wherein the element contains the target extraction task information and has a second element type. Extract a first vector corresponding to the first reference element and a second vector corresponding to the second reference element in the output tensor. Delete the second reference element in the second vector to obtain a second vector after the second reference element is removed. Concatenate the first vector and the second vector after the second reference element is removed to obtain a target element sequence corresponding to the target extraction task information.

[0169] The apparatus provided by the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again in the embodiment.

[0170] In a possible implementation, the sequence determining module 404 is specifically configured to determine an element in the output tensor that includes the target extraction task information and has a target element category as a target reference element corresponding to the target extraction task information. Determine an element between the target reference element and another reference element in the output tensor as a target element sequence corresponding to the target extraction task information. Or, determine an element from the target reference element to the last element in the output tensor as the target element sequence corresponding to the target extraction task information.

[0171] The apparatus provided by the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again in the embodiment.

[0172] In a possible implementation, the character extracting module 405 is specifically configured to determine an element in the target element sequence corresponding to the target extraction task information as a target element, where the element has a preset element value. Determine a position corresponding to the target element as a target position. Determine a character in the to-be-extracted text corresponding to the target position as a target character. Read the target character in the to-be-extracted text.

[0173] The apparatus provided by the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again in the embodiment.

[0174] In a possible implementation, the character extracting apparatus 400 further includes a model training module 406.

[0175] The model training module 406 is configured to obtain training image information corresponding to each training image, a training text box, training text box attributes corresponding to each training text box, and training task information. Calculate the coincidence degree corresponding to each training task information according to the training text box attributes and the training task information. Generate a coincidence degree label corresponding to each training text box according to the coincidence degree and the training text box attributes. Generate a character coincidence degree label of each training character corresponding to each training text box according to the coincidence degree label corresponding to each training text box, the training text box attributes, and a preset standard character. Generate a standard output tensor according to the coincidence degree label corresponding to each training text box and the character coincidence degree label. Generate a standard input tensor according to the training text box attributes and the training image information. Train the to-be-trained model by using the standard input tensor and the standard output tensor to obtain the data extraction model.

[0176] The device provided by the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects. Details are not described herein again.

[0177] To implement the above embodiments, the electronic device is also provided in the embodiments of the present application.

[0178] Reference Figure 5 which shows a structural schematic diagram of an electronic device 500 suitable for implementing the embodiments of the present application. The electronic device 500 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers (PAD), portable media players (PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The electronic device shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0179] As shown in Figure 5 The electronic device 500 can include a processor (such as a central processor, a graphics processor, etc.) 501, and a memory 502 connected in communication with the processor. The memory 502 can be a read-only memory (ROM) 501, which can perform various appropriate actions and processes according to the program, computer execution instructions stored in the memory 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503, and implement the text extraction method in any of the above embodiments. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the memory 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0180] In general, the following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 507 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 508 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 509. The communication devices 509 can allow the electronic device 500 to communicate wirelessly or wired with other devices to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it is understood that all of the shown devices are not required to be implemented or present. More or less devices can alternatively be implemented or present.

[0181] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication devices 509, or installed from the storage devices 508, or installed from the memory 502. When the computer program is executed by the processing devices 501, the above-mentioned functions defined in the methods of embodiments of the present application are performed.

[0182] It should be noted that the computer readable storage medium in the above embodiments of the present application can be a computer readable signal medium or a computer storage medium or any combination of the two. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, the computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, the computer readable signal medium can include a computer readable program code propagated in or on a carrier wave, in baseband signal, or using any suitable combination of the above. Such propagated computer readable signal medium can take many forms, including but not limited to, electro-magnetic, optical, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer readable storage medium can be transmitted, propagated, or transported in a variety of ways, including but not limited to, electrical, optical, RF (radio frequency), or any suitable combination of the above.

[0183] The computer readable storage medium described above can be included in the electronic device described above; or can exist separately from the electronic device and be not assembled into the electronic device.

[0184] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiments.

[0185] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0186] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0187] The modules involved in the embodiments of the present application described above can be implemented in software or hardware. Among them, the name of the unit does not constitute a limitation to the module itself in some cases. For example, the tensor generation module can also be described as an "input tensor generation module".

[0188] The functions described in the above description above can be performed at least in part by one or more hardware logic components. For example, non-limiting examples of exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0189] The application further provides a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the technical solution of the character extraction method in any of the above embodiments is realized, the implementation principle and beneficial effects of which are similar to those of the character extraction method, and can be referred to the implementation principle and beneficial effects of the character extraction method, which will not be repeated here.

[0190] In the context of the present application, the machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0191] The application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the technical solution of the character extraction method in any of the above embodiments is realized, the implementation principle and beneficial effects of which are similar to those of the character extraction method, and can be referred to the implementation principle and beneficial effects of the character extraction method, which will not be repeated here.

[0192] The above description is merely the preferred embodiments and the explanation of the technical principles of the application. It should be understood by those skilled in the art that the disclosed scope of the application is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the similar technical features disclosed in the application (but not limited to) with similar functions.

[0193] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application be limited only by the scope of the claims, including any appropriate equivalents.

[0194] It is to be understood that the application is not limited to the precise construction already described above and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.< / cls> < / sep> < / cls> < / sep> < / cls>

Claims

1. A text extraction method, characterized in that, include: In response to the start of the text extraction process, the system acquires the image information to be recognized, at least one extraction task information, at least one text box, and the text box attributes corresponding to each text box, wherein the text box attributes include the text to be extracted. Based on the attributes of each text box and the information of the image to be recognized, an input tensor is generated; The input tensor is input into the data extraction model corresponding to the text extraction process to obtain the output tensor output by the data extraction model, wherein the output tensor includes at least two types of elements. Based on the types of elements in the output tensor and the target extraction task information, determine the target element sequence corresponding to the target extraction task information in the output tensor, wherein the target extraction task information is any extraction task information; Based on the target element sequence corresponding to the target extraction task information, extract the target text from each text to be extracted. Before the text extraction process begins, the process also includes: obtaining training image information, training text boxes, training text box attributes, and training task information corresponding to each training image; Calculate the overlap degree corresponding to each training task information based on the training text box attributes and the training task information; Based on the overlap degree and the attributes of the training text boxes, generate overlap degree markers for each training text box; Based on the overlap markers corresponding to each training text box, the attributes of the training text box, and the preset standard characters, generate text overlap markers for each training text box corresponding to each training text. Generate a standard output tensor based on the overlap markers and text overlap markers corresponding to each training text box; Generate a standard input tensor based on the attributes of each training text box and the training image information; The data extraction model is obtained by training the model to be trained using the standard input tensor and the standard output tensor.

2. The method according to claim 1, characterized in that, The text box attributes also include the text to be extracted, the text box coordinates, and the text box number; Accordingly, generating the input tensor based on the attributes of each text box and the image information to be recognized includes: Input the text length of the text to be extracted corresponding to the target text box into the preset conversion model to obtain the target sequence, wherein the target text box is any text box; Calculate the regularized target text box coordinates based on the text box coordinates corresponding to the target text box and the image information to be recognized. Divide the text box number corresponding to the target text box by the largest text box number to obtain the target rank ratio; The target sequence, the regularized target text box coordinates, and the target position ratio are concatenated to obtain the target vector corresponding to the target text box. The target vectors corresponding to each text box are concatenated to obtain the input tensor.

3. The method according to claim 1, characterized in that, The step of determining the sequence of target elements in the output tensor corresponding to the target extraction task information based on the types of elements in the output tensor and the target extraction task information includes: The elements in the output tensor that contain target extraction task information and whose element type is the first element type are determined as the first reference elements; The elements in the output tensor that contain the target extraction task information and whose element type is the second element type are determined as the second reference elements; Extract the first vector corresponding to the first reference element and the second vector corresponding to the second reference element from the output tensor; Delete the second reference element from the second vector to obtain the second vector after removing the marker; By concatenating the first vector and the second vector after removing the marker, the target element sequence corresponding to the target extraction task information is obtained.

4. The method according to claim 1, characterized in that, The step of determining the sequence of target elements in the output tensor corresponding to the target extraction task information based on the types of elements in the output tensor and the target extraction task information includes: The elements in the output tensor that include target extraction task information and whose element type is the target element type are determined as the target reference elements corresponding to the target extraction task information. The elements between the target reference element and another reference element in the output tensor are determined as the target element sequence corresponding to the target extraction task information; or, the elements from the target reference element to the last element in the output tensor are determined as the target element sequence corresponding to the target extraction task information.

5. The method according to claim 1, characterized in that, The step of extracting target text from each text to be extracted based on the target element sequence corresponding to the target extraction task information includes: In the target element sequence corresponding to the target extraction task information, the element with the preset element value is determined as the target element; Define the position corresponding to the target element as the target position; The text corresponding to the target position in the text to be extracted is identified as the target text; Read the target text from the text to be extracted.

6. A text extraction device, wherein the text extraction device is used to implement the text extraction method according to any one of claims 1-5, characterized in that, include: The data acquisition module is used to acquire, in response to the start of the text extraction process, image information to be recognized, at least one extraction task information, at least one text box and text box attributes corresponding to each text box, wherein the text box attributes include the text to be extracted. The tensor generation module is used to generate input tensors based on the attributes of each text box and the image information to be recognized. Tensor input module is used to input the input tensor into the data extraction model corresponding to the text extraction process to obtain the output tensor output by the data extraction model, wherein the output tensor includes at least two types of elements; The sequence determination module is used to determine the sequence of target elements in the output tensor corresponding to the target extraction task information based on the types of elements in the output tensor and the target extraction task information, wherein the target extraction task information is any extraction task information. The text extraction module is used to extract target text from each text to be extracted based on the target element sequence corresponding to the target extraction task information.

7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the text extraction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the text extraction method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the text extraction method according to any one of claims 1 to 5.

Images