Image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides an image processing method, device, electronic device and storage medium, which relate to the fields of optical character recognition, image recognition and computer technology, so as to at least solve the technical problems of low target area detection accuracy and great difficulty in target information extraction of the target area processing method provided in the prior art. The specific implementation scheme is: obtaining an image to be processed, wherein the display content of the image to be processed includes: a first target area; performing optical character recognition on the image to be processed to obtain a recognition result; obtaining multiple first candidate frames from the first target area based on the recognition result; using multiple first candidate frames to intercept a partial image from the image to be processed, wherein the display content of the partial image includes: a second target area, and the second target area is a partial area of ​​the first target area; detecting the partial image to obtain the target key-value pair information corresponding to the second target area.

Description

Technical Field

[0001] The present disclosure relates to the fields of optical character recognition, image recognition and computer technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] Document intelligence refers to the process of understanding, classifying, extracting and summarizing text information, typesetting formats and other information contained in web pages, pictures, scanned documents or digital documents through artificial intelligence technology. Document intelligence is an important part of the digitalization process. However, in actual application scenarios, the documents to be processed are usually diverse and complex, and document intelligence technology also faces corresponding challenges. How to perform structured analysis and content extraction on complex documents has become one of the important challenges in document intelligence technology.

[0003] In the related art, there are mainly two methods for structural analysis and content extraction of complex documents, and each method has its own advantages and disadvantages.

[0004] The first method is to use deep learning-based document OCR technology to identify the document structure, which can reconstruct the document at the perceptual level and then extract the content. However, the disadvantage of this method is that it cannot determine the target document area, and it is difficult to extract information from complex documents.

[0005] The second method is to use the frame line detection technology based on image morphology (such as document frame line, table frame line, etc.) and post-processing rules to identify the document structure and determine the target area, and then extract the content. However, the defects of this method are: it is only applicable to documents with frame lines, and cannot be used to identify documents without frame lines; the use of post-processing rules leads to low flexibility.

[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0007] The present disclosure provides an image processing method, an apparatus, an electronic device and a storage medium to at least solve the technical problems of low target area detection accuracy and great difficulty in target information extraction in the target area processing method provided in the prior art.

[0008] According to one embodiment of the present disclosure, there is provided an image processing method, comprising: acquiring an image to be processed, wherein the display content of the image to be processed includes: a first target area; performing optical character recognition on the image to be processed to obtain a recognition result; acquiring a plurality of first candidate frames from the first target area based on the recognition result; using the plurality of first candidate frames to intercept a partial image from the image to be processed, wherein the display content of the partial image includes: a second target area, the second target area being a partial area of ​​the first target area; and detecting the partial image to obtain target key-value pair information corresponding to the second target area.

[0009] According to one embodiment of the present disclosure, there is also provided an image processing device, comprising: a first acquisition module, used to acquire an image to be processed, wherein the display content of the image to be processed includes: a first target area; a recognition module, used to perform optical character recognition on the image to be processed to obtain a recognition result; a second acquisition module, used to acquire a plurality of first candidate frames from the first target area based on the recognition result; a capture module, used to capture a partial image from the image to be processed using the plurality of first candidate frames, wherein the display content of the partial image includes: a second target area, which is a partial area of ​​the first target area; and a processing module, used to detect the partial image to obtain target key-value pair information corresponding to the second target area.

[0010] According to one embodiment of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method proposed in the present disclosure.

[0011] According to one embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable a computer to execute the image processing method proposed in the present disclosure.

[0012] According to one embodiment of the present disclosure, a computer program product is also provided, including a computer program, and the computer program executes the image processing method proposed in the present disclosure when a processor executes the computer program.

[0013] In an embodiment of the present disclosure, an image to be processed is obtained, wherein the display content of the image to be processed includes: a first target area, optical character recognition is performed on the image to be processed to obtain a recognition result, and multiple first candidate boxes are obtained from the first target area based on the recognition result, and a partial image is captured from the image to be processed using the multiple first candidate boxes, wherein the display content of the partial image includes: a second target area, and the second target area is a partial area of ​​the first target area, and then the partial image is detected to obtain target key-value pair information corresponding to the second target area, thereby achieving the purpose of deconstructing complex tables, determining target table areas and extracting target table information in a flexible and accurate manner, achieving the technical effect of improving the detection accuracy of target areas and reducing the difficulty of extracting target information, and solving the technical problems of low target area detection accuracy and great difficulty in extracting target information in the target area processing method provided by the related technology.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0016] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present disclosure;

[0017] Figure 2 is a flowchart of an image processing method provided according to an embodiment of the present disclosure;

[0018] Figure 3 is a schematic diagram of an optional processing process of a document image table area according to an embodiment of the present disclosure;

[0019] Figure 4 is a schematic diagram of an optional document image table area according to an embodiment of the present disclosure;

[0020] Figure 5 is a schematic diagram of an optional recognition result of a table area of ​​a document image according to an embodiment of the present disclosure;

[0021] Figure 6 is a schematic diagram of an optional result of dividing a document image table area according to an embodiment of the present disclosure;

[0022] Figure 7 It is a structural block diagram of an image processing device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0025] According to an embodiment of the present disclosure, an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] The method embodiments provided by the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar electronic device. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein. Figure 1 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present disclosure.

[0027] like Figure 1As shown, the computer terminal 100 includes a computing unit 101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 102 or a computer program loaded from a storage unit 108 into a random access memory (RAM) 103. In the RAM 103, various programs and data required for the operation of the computer terminal 100 can also be stored. The computing unit 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0028] A number of components in the computer terminal 100 are connected to the I / O interface 105, including: an input unit 106, such as a keyboard, a mouse, etc.; an output unit 107, such as various types of displays, speakers, etc.; a storage unit 108, such as a disk, an optical disk, etc.; and a communication unit 109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 109 allows the computer terminal 100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0029] The computing unit 101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 101 performs the image processing method described herein. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 108. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer terminal 100 via the ROM 102 and / or the communication unit 109. When the computer program is loaded into the RAM 103 and executed by the computing unit 101, one or more steps of the method for locating a faulty hard disk described herein may be performed. Alternatively, in other embodiments, the computing unit 101 may be configured to perform the method for locating a faulty hard disk by any other appropriate means (e.g., by means of firmware).

[0030] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0031] It should be noted that, in some optional embodiments, the above Figure 1 The electronic device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the electronic devices described above.

[0032] Under the above operating environment, the present disclosure provides Figure 2 The image processing method shown in FIG. 1 can be performed by Figure 1 The computer terminal or similar electronic device shown is used for execution. Figure 2 FIG. 1 is a flowchart of an image processing method provided according to an embodiment of the present disclosure. Figure 2 As shown, the method may include the following steps:

[0033] Step S21, obtaining an image to be processed, wherein the display content of the image to be processed includes: a first target area;

[0034] Optionally, the image to be processed may be an image to be identified or information to be extracted, and may be a single page image of a PDF document, a screenshot of a spreadsheet, a screenshot of an electronic document, a photo of a printed spreadsheet, a photo of a printed document, etc.

[0035] Optionally, the display content of the image to be processed may include: the first target area. The first target area may be a complex table area, for example, a table area including merged cells, split cells, fixed cells, and cells filled with complex content.

[0036] Step S22, performing optical character recognition on the image to be processed to obtain a recognition result;

[0037] Optionally, the above optical character recognition (OCR) refers to a process in which an electronic device examines characters printed on paper or displayed in a picture, determines their shapes by detecting dark and light patterns, and then translates their shapes into computer text using a character recognition method.

[0038] Optionally, the above-mentioned recognition result can be obtained by performing OCR on the above-mentioned image to be processed. The recognition result can include a plurality of computer characters corresponding to the image to be processed and their relative positions in the image.

[0039] Step S23, acquiring a plurality of first candidate frames from the first target area based on the recognition result;

[0040] Optionally, based on the OCR recognition result of the image to be processed, the multiple first candidate boxes can be obtained from the first target area. The first target area can be a complex table area in the image to be processed, and the multiple first candidate boxes can be multiple text display areas in the complex table area.

[0041] Step S24, using the plurality of first candidate frames to intercept a partial image from the image to be processed, wherein the display content of the partial image includes: a second target area, which is a partial area of ​​the first target area;

[0042] Optionally, using the above-mentioned multiple first candidate frames, a partial image can be intercepted from the above-mentioned image to be processed, and the display content of the partial image includes: a second target area. The second target area is a partial area of ​​the above-mentioned first target area. The first target area can be a complex table area in the image to be processed, and the second target area can be a simple table sub-area in the complex table area.

[0043] Specifically, the complex table area may include multiple sub-areas. The second target area may be a sub-area of ​​the multiple sub-areas of the complex table area that does not include complex situations (such as merged cells, split cells, fixed cells, and cells filled with complex content).

[0044] Step S25: detect a portion of the image to obtain target key-value pair information corresponding to the second target area.

[0045] Optionally, the target key-value pair information corresponding to the second target area can be obtained by detecting a portion of the image intercepted from the image to be processed. The second target area can be a simple table sub-area in a complex table area, and the target key-value pair information can be the key-value pair information that needs to be extracted from the image to be processed.

[0046] Specifically, other method steps included in the image processing method can also refer to the further introduction of the embodiments of the present disclosure below, which will not be repeated here.

[0047] In an embodiment of the present disclosure, an image to be processed is obtained, wherein the display content of the image to be processed includes: a first target area, optical character recognition is performed on the image to be processed to obtain a recognition result, and multiple first candidate boxes are obtained from the first target area based on the recognition result, and a partial image is captured from the image to be processed using the multiple first candidate boxes, wherein the display content of the partial image includes: a second target area, and the second target area is a partial area of ​​the first target area, and then the partial image is detected to obtain target key-value pair information corresponding to the second target area, thereby achieving the purpose of deconstructing complex tables, determining target table areas and extracting target table information in a flexible and accurate manner, achieving the technical effect of improving the detection accuracy of target areas and reducing the difficulty of extracting target information, and solving the technical problems of low target area detection accuracy and great difficulty in extracting target information in the target area processing method provided by the related technology.

[0048] The above method of this embodiment is further introduced below.

[0049] As an optional implementation, in step S23, multiple first candidate boxes are obtained from the first target area based on the recognition result, and the following method steps are also included:

[0050] Step S231, extracting text features and image features from the recognition results;

[0051] Step S232, using the target neural network model to analyze the text features and the image features, and obtain multiple first candidate boxes from the first target area, wherein the target neural network model is obtained through machine learning training using multiple sets of data, and each set of data in the multiple sets of data includes: a sample image and multiple candidate boxes identified in the sample image.

[0052] In the above optional implementation, the above recognition result may be an OCR result of the image to be processed. The above text features and the above image features may be extracted from the recognition result. The text features may be used to describe the text information contained in the image to be processed. The image features may be used to describe the location information of the text information contained in the image to be processed.

[0053] In the above optional implementation, the target neural network may be a model trained by machine learning using multiple sets of data. The target neural network model may be used to analyze the text features and image features of the image to be processed. Each set of data in the multiple sets of data includes: the sample image and multiple candidate boxes identified in the sample image.

[0054] In particular, in the process of training the above-mentioned target neural network, the sample image of each group of data in the multiple groups of data and the multiple candidate boxes identified in the sample image can be pre-specified by technicians according to needs in actual application scenarios. The sample image of each group of data in the multiple groups of data and the multiple candidate boxes identified in the sample image can also be updated in real time during the training process of the model.

[0055] In the above optional implementation, the target neural network model can be used to analyze the text features and image features extracted from the above recognition results, so as to obtain the above multiple first candidate boxes from the above first target area. The first target area can be a complex table area in the image to be processed, and the multiple first candidate boxes can be multiple text display areas in the complex table area.

[0056] As an optional implementation, in step S231, extracting text features and image features from the recognition result also includes the following method steps:

[0057] Step S2311, data labeling is performed on the recognition result to obtain a target labeling result;

[0058] Step S2312, preprocessing the target annotation result to obtain text features and image features.

[0059] In the above optional implementation, the recognition result may be an OCR result of the image to be processed. Data annotation of the recognition result may be annotating the OCR result with one or more tags. The above target annotation result may be obtained through the data annotation process.

[0060] In the above optional implementation, the above target annotation result is preprocessed to obtain the above text features and the above image features. The text features can be used to describe the text information contained in the image to be processed. The image features can be used to describe the location information of the text information contained in the image to be processed.

[0061] Specifically, data labeling of the recognition results and preprocessing of the target labeling results can also be referred to the further introduction below, which will not be repeated here.

[0062] As an optional implementation, in step S2311, data annotation is performed on the recognition result to obtain a target annotation result, which also includes the following method steps:

[0063] Step S1101, in the recognition result, a plurality of first candidate boxes are marked with a first mark, and a plurality of second candidate boxes are marked with a second mark, to obtain an initial marking result, wherein the first mark and the second mark are used to distinguish the target area where the candidate boxes are located;

[0064] Step S1102: convert the initial annotation result according to a preset data set format to obtain a target annotation result.

[0065] In the above optional implementation, the recognition result may be an OCR result of the above image to be processed. In the recognition result, the above multiple first candidate boxes may be annotated with the above first mark, and the above multiple second candidate boxes may be annotated with the above second mark, thereby obtaining the above initial annotation result.

[0066] Optionally, the first mark and the second mark can be used to distinguish the target area where the candidate boxes are located. The multiple first candidate boxes can be multiple text display areas in a complex table area of ​​the image to be processed, and the multiple second candidate boxes can be multiple text display areas in a simple table area of ​​the image to be processed. Correspondingly, the first mark can be a mark corresponding to a complex table area, and the second mark can be a mark corresponding to a simple table area.

[0067] Optionally, the above initial marking result may include: a recognition result, a plurality of first candidate boxes and their corresponding first marks, a plurality of second candidate boxes and their corresponding second marks.

[0068] In the above optional implementation, the preset data set format may be a data set format preset by a technician according to requirements in an actual application scenario. The target labeling result may be obtained by converting the above initial labeling result according to the preset data set format.

[0069] Figure 3 is a schematic diagram of an optional processing process of a document image table area according to an embodiment of the present disclosure, such as Figure 3 As shown, in the process of processing the table area of ​​the document image, first, the document image Fig01 (equivalent to the above-mentioned image to be processed) is obtained, and the document image includes the table area Sheet01.

[0070] Figure 4 is a schematic diagram of an optional document image table area according to an embodiment of the present disclosure, such as Figure 4 The table area Sheet01 (equivalent to the first target area) included in the document image Fig01 is shown.

[0071] Still like Figure 3 As shown, the processing process of the document image table area also includes the text position information processing process. The document image Fig01 is subjected to OCR recognition through an OCR service (such as a document recognition OCR interface provided in the application software) to obtain a recognition result, which is recorded as Fig01_OCR.

[0072] Optionally, when the image to be processed is a PDF document, the text position information processing process may further include parsing the PDF document using a PDF parser.

[0073] Figure 5 is a schematic diagram of an optional recognition result of a document image table area according to an embodiment of the present disclosure, such as Figure 5 As shown, after the OCR service, the text display area in the table can be identified (as shown in the black box area in the figure, equivalent to the above-mentioned multiple first candidate boxes), and each of the multiple text display areas in the table can be called a bbox.

[0074] Optionally, the recognition result Fig01_OCR may include: text information of each bbox in the table area Sheet01, position coordinates of each bbox in the table area Sheet01, text of character granularity in each bbox in the table area Sheet01 and position coordinates of the text of the character granularity, etc.

[0075] Figure 6 is a schematic diagram of an optional division result of a document image table area according to an embodiment of the present disclosure, such as Figure 6 As shown, the table area Sheet01 contained in the document image Fig01 can be divided into simple table sub-areas Sheet01_E (such as Figure 6 The shaded area shown in the figure is equivalent to the second target area) and the complex table sub-area Sheet01_C (such as Figure 6 Other areas except the shaded areas shown in the figure).

[0076] It is easy to notice that the simple table sub-area Sheet01_E does not contain complex situations such as merged cells and split cells. In actual application scenarios, similar simple table sub-areas are convenient for users to extract target information from them. In the process of recognizing the entire document, simple table sub-areas can be deconstructed from the document to extract simple information to optimize the document recognition process.

[0077] Still like Figure 3 As shown, the processing process of the document image table area also includes an image feature extraction process. In this image extraction process, text features and image features are extracted from the recognition result Fig01_OCR through feature maps and ResNet feature networks.

[0078] Specifically, extracting text features and image features from the recognition result Fig01_OCR can include two stages: data annotation stage and data preprocessing stage.

[0079] Specifically, in the above data annotation stage, first, all bboxes in the recognition result Fig01_OCR are annotated: the simple table sub-area Sheet01_E (such as Figure 6 The shaded area shown in FIG. 1 is equivalent to the second target area) and the bbox in the complex table subarea Sheet01_C (such as Figure 6 The part of the bbox in the area other than the shaded area shown in (a) is marked as O_bbox.

[0080] Specifically, in the above data annotation stage, secondly, all marked bboxes in the entire table area Sheet01 (equivalent to the above initial annotation results) are converted into the format of the XFUN data set (equivalent to the above preset data set format).

[0081] It should be noted that the above XFUN dataset is a public dataset. Converting all marked bboxes in the entire table area Sheet01 into the format of the XFUN dataset can facilitate the use of existing application programming interfaces (APIs) to generate training datasets, which can then be input into neural network models for training; it can also facilitate subsequent calls to the dataset.

[0082] Optionally, in the above data annotation stage, we may only focus on the area described by each bbox, annotate bboxes belonging to different areas with different tags, and ignore the relationship between multiple bboxes.

[0083] As an optional implementation, in step S2312, the target annotation result is preprocessed to obtain text features and image features, and the following method steps are also included:

[0084] Step S1201, converting the target annotation result by using a sequence annotation method to obtain a first processing result;

[0085] Step S1202, connecting the text fields displayed by the plurality of third candidate boxes contained in the target data unit to obtain a second processing result;

[0086] Step S1203, based on the first processing result and the second processing result, encapsulate the attribute information of each of the plurality of third candidate boxes to obtain a third processing result, wherein the attribute information includes: the start index and the end index of the text field displayed by each of the plurality of third candidate boxes in the second processing result, and the label corresponding to each of the plurality of third candidate boxes in the second processing result;

[0087] Step S1204, performing word segmentation on the third processing result to obtain a fourth processing result;

[0088] Step S1205 resizes the image to be processed to obtain a fifth processing result;

[0089] Step S1206, determining text features and image features using the first processing result, the second processing result, the third processing result, the fourth processing result, and the fifth processing result.

[0090] In the above optional implementation, the target labeling result can be transformed by using a sequence labeling method to obtain the first processing result. The input of the sequence labeling method is a sequence, and the output is also a sequence.

[0091] In the above optional implementation, the text fields displayed by the above multiple third candidate boxes contained in the above target data unit are connected to obtain the above second processing result. The multiple third candidate boxes are multiple text display areas contained in the target data unit. The second processing result obtained by connecting the text fields displayed by the multiple third candidate boxes can be the corresponding document content.

[0092] In the above optional implementation, each of the above multiple third candidate boxes contains attribute information. The attribute information may include: the starting index (such as the starting position coordinates) and the ending index (such as the ending position coordinates) of the text field displayed by each of the multiple third candidate boxes in the above second processing result; and the label corresponding to each of the multiple third candidate boxes in the above second processing result.

[0093] In the optional implementation, based on the first processing result and the second processing result, the attribute information of each candidate box in the plurality of third candidate boxes may be encapsulated to obtain the third processing result. Optionally, the encapsulation may be entity name encapsulation.

[0094] In the above optional implementation, the above third processing result is segmented to obtain the above fourth processing result. Optionally, the segmentation process may be to segment text data to obtain a text segmentation result, and convert the text segmentation result into a segmentation mark.

[0095] In the above optional implementation, the image to be processed is resized to obtain the fifth processing result. Optionally, the resizing may be to adjust the size of the image to be processed according to a preset size rule. The resized image to be processed is used as the fifth processing result.

[0096] In the optional implementation, the text feature and the image feature of the image to be processed can be determined by using the first processing result, the second processing result, the third processing result, the fourth processing result and the fifth processing result. The text feature can be used to describe the text information contained in the image to be processed. The image feature can be used to describe the location information of the text information contained in the image to be processed.

[0097] Still like Figure 3 As shown, in the image feature extraction process, extracting text features and image features from the recognition result Fig01_OCR can include two stages: data annotation stage and data preprocessing stage.

[0098] Specifically, the data preprocessing stage may include the following method steps:

[0099] Step S261, obtaining preset model training sample data sample (equivalent to the above target data unit), the model training sample data sample is OCR recognized and data annotated (including T or O mark), and includes multiple text boxes sample_bbox (equivalent to the above multiple third candidate boxes);

[0100] Step S262, using the BIO annotation method (equivalent to the above sequence annotation method) to convert the labels (the above T or O) of multiple text boxes sample_bbox into token labels, and record all bboxes and the converted labels as labels (equivalent to the above first processing result);

[0101] Step S263, obtaining text fields in multiple text boxes sample_bbox in the model training sample data sample and connecting them in a preset order (such as from top to bottom and from left to right), so as to obtain the corresponding training sample document sample_text (equivalent to the above second processing result);

[0102] Step S264, encapsulating the converted labels labels of the multiple text boxes sample_bbox according to the training sample document sample_text, and obtaining encapsulation result entities (equivalent to the third processing result above);

[0103] Step S265, perform word segmentation on the encapsulation result entities, convert the token label in the label labels corresponding to the word segmentation into token_id, and record all bboxes and the converted token_id as input_ids (equivalent to the fourth processing result above);

[0104] Step S266, adjusting the size of the document image Fig01 (equivalent to the above-mentioned image to be processed) to 3×224×224, and recording the adjusted document image as image (equivalent to the above-mentioned fifth processing result);

[0105] Step S267, encapsulate the sample_bbox and the corresponding input_ids, labels, entities, and image to obtain an encapsulation result;

[0106] Step S268, input the above encapsulation result into the preset data loader class for iteration and adding the attention mask attention_mask;

[0107] Step S269, determining text data (equivalent to the above-mentioned text features) by the text field value of bbox, and determining image data (equivalent to the above-mentioned image features) by the adjusted document image image.

[0108] It should be noted that in the above step S262, the BIO marking method refers to marking each bbox with three token labels: "BX", "IX" or "O". Among them, the token label of a bbox is "BX", which means that the segment where the bbox is located is of type X and the bbox is at the beginning of this segment. The token label of a bbox is "IX", which means that the segment where the bbox is located is of type X and the bbox is in the middle of this segment. The token label of a bbox is "O", which means that the bbox does not belong to any type. In particular, the "segment" here can be a text document. In addition,

[0109] It should be noted that in the above step S268, the preset data loader class can be a class in Python language, and the data loader class can encapsulate the training data as an iterator, in which a data set or sample of a preset size can be taken out in turn and input into the model for training. In addition, by adding the attention mask attention_mask, it is possible to avoid paying attention to fillers when using the attention mechanism in the neural network model later.

[0110] As an optional implementation, the target neural network model includes: an embedding layer, a feature fusion extraction layer. In step S232, the target neural network model is used to analyze the text features and the image features, and multiple first candidate boxes are obtained from the first target area. The following method steps are also included:

[0111] Step S2321, inputting text features into the embedding layer to obtain a first output result, and inputting image features into the embedding layer to obtain a second output result;

[0112] Step S2322, in the feature fusion extraction layer, performing feature fusion extraction on the first output result and the second output result to obtain a third output result;

[0113] Step S2323, obtaining a text sequence from the third output result;

[0114] Step S2324: perform semantic entity recognition on the text sequence, and obtain multiple first candidate boxes from the first target area.

[0115] In the above optional implementation, the target neural network model for analyzing the text features and image features of the image to be processed may include at least: an embedding layer and a feature fusion extraction layer.

[0116] In the above optional implementation, the text features of the above image to be processed are input into the embedding layer of the above target neural network model, so as to obtain the above first output result. The image features of the above image to be processed are input into the embedding layer of the above target neural network model, so as to obtain the above second output result.

[0117] In the above optional implementation, in the above target neural network, the first output result and the second output result output by the embedding layer are input into the feature fusion layer. The feature fusion layer can perform feature fusion extraction on the first output result and the second output result to obtain the above third output result.

[0118] In the above optional implementation, the above text sequence can be obtained from the third output result obtained based on the above feature fusion layer. Furthermore, the above multiple first candidate boxes can be obtained from the above first target area by performing semantic entity recognition on the text sequence.

[0119] Still like Figure 3 As shown, the processing of the document image table area also includes using the target neural network (equivalent to Figure 3 The process of analyzing text data and image data (the text and position embedding layer and the feature fusion extraction layer in the image processing module).

[0120] Specifically, in the text and position embedding layer of the target neural network, partial embedding, text embedding, one-dimensional embedding, two-dimensional embedding and visual embedding can be performed based on text data (equivalent to the above-mentioned text features) and image data (equivalent to the above-mentioned image features) to obtain text feature embedding results (equivalent to the above-mentioned first output results) and image feature embedding results (equivalent to the above-mentioned second output results), respectively.

[0121] Specifically, in the feature fusion extraction layer of the target neural network, feature fusion extraction may be performed on the above text feature embedding result and the above image feature embedding result to obtain a feature fusion extraction result (equivalent to the above third output result).

[0122] Still like Figure 3 As shown, according to the output result of the feature fusion extraction layer of the target neural network, the text sequence (Sequence) and the image Image can be obtained.

[0123] Still like Figure 3 As shown, conditional random field layers and fully connected layers can be set to fine-tune the acquired text sequence and image to obtain output results.

[0124] Specifically, the conditional random field layer can be used to perform semantic entity recognition on the text sequence, thereby obtaining multiple bboxes from the table area Sheet01 contained in the document image Fig01. In particular, the entity in the semantic entity recognition can be a bbox in a simple table sub-area.

[0125] It should be noted that in the feature fusion extraction layer, compared with the self-attention mechanism used in the related technology that can capture the relationship between token tags based on absolute positions, the Transformer implemented using the self-attention mechanism based on spatial perception can make full use of the semantic relative position and spatial relative position to calculate the attention weights, and can also capture the local invariance in the document layout, which has obvious beneficial effects.

[0126] As an optional implementation, in step S24, using multiple first candidate frames to intercept a portion of the image from the image to be processed also includes the following method steps:

[0127] Step S241, determining a second target area using coordinate information of a plurality of first candidate frames;

[0128] Step S242: intercepting a portion of the image from the image to be processed according to the second target area.

[0129] In the above optional implementation, the plurality of first candidate boxes may be a plurality of text display areas in a complex table area of ​​the image to be processed. Each of the plurality of first candidate boxes may include coordinate information of the candidate box. The second target area (which may also include coordinate information of the second target area) may be determined using the coordinate information of the plurality of first candidate boxes.

[0130] In the above optional implementation, the second target area may be a simple table sub-area in a complex table area of ​​the image to be processed. According to the second target area (or according to the coordinate information of the second target area), a partial image may be intercepted from the image to be processed. The image to be processed includes a complex table area, and the intercepted partial image is a simple table sub-area image in the complex table area of ​​the image to be processed.

[0131] Still taking the table area Sheet01 in the document image Fig01 as an example, the token labels corresponding to multiple bboxes in the table area Sheet01 can be obtained from the target neural network model, and the partial bboxes corresponding to the two token labels "BT" and "IT" can be merged into the T_bbox set belonging to the T label. Each bbox of the partial bbox in the T_bbox set can contain the position coordinates of the bbox (such as x coordinates and y coordinates).

[0132] Still taking the processing of the table area Sheet01 in the document image Fig01 as an example, the simple table sub-area Sheet01_E in the table area Sheet01 can be determined by using the position coordinates of each bbox in the above T_bbox set. Specifically, the method for determining the simple table sub-area Sheet01_E according to the position coordinates can be: taking the minimum value of the x coordinate and the y coordinate in the position coordinates of each bbox in the T_bbox set as the upper left corner coordinate of the simple table sub-area Sheet01_E; taking the maximum value of the x coordinate and the y coordinate in the position coordinates of each bbox in the T_bbox set as the lower right corner coordinate of the simple table sub-area Sheet01_E; and determining the simple table sub-area Sheet01_E according to the upper left corner coordinate and the lower right corner coordinate.

[0133] Still taking the processing of the table area Sheet01 in the document image Fig01 as an example, according to the simple table sub-area Sheet01_E, a partial image Fig01_E corresponding to the simple table sub-area Sheet01_E can be intercepted from the document image Fig01 (e.g., Figure 6 All areas shown in the figure are shaded images).

[0134] Still taking the processing of the table area Sheet01 in the document image Fig01 as an example, the partial image Fig01_E can be detected to obtain the target key-value pair information corresponding to the simple table sub-area Sheet01_E.

[0135] It should be noted that for the framed table in the document image, the frame detection technology based on image morphology and the post-processing rule method can be used to obtain the target key-value pair information. For the frameless table in the document image, the complex table OCR technology based on deep learning provided by the embodiment of the present disclosure can be used for processing to obtain the target key-value pair information.

[0136] It is easy to notice that according to the method provided by the embodiment of the present disclosure, a document image (or PDF document) containing a complex table can be deconstructed to detect a simple table area. Therefore, one of the beneficial effects of the embodiment of the present disclosure may be to facilitate the subsequent deep learning-based table OCR technology or image morphology-based frame line detection technology to extract target information in a simple table area.

[0137] It is easy to notice that the above method of the embodiment of the present disclosure can be an information extraction method for the target area of ​​a complex table PDF scan based on the LayoutXLM pre-trained model. Different from the image semantic segmentation or target area detection technology based on computer vision technology, this method not only utilizes the image semantics of the position description of the table cells in the document, but also utilizes the text semantics of the text information of the table cells. It is a multimodal solution for monitoring the target area. Therefore, one of the beneficial effects of the embodiment of the present disclosure can be: due to the adoption of a multimodal solution, the accuracy of target area monitoring and the flexibility and stability of the solution are improved.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0139] In the present disclosure, an image processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0140] Figure 7 is a structural block diagram of an image processing device provided according to an embodiment of the present disclosure, such as Figure 7 As shown, the image processing device 700 includes: a first acquisition module 701, used to acquire the image to be processed, wherein the display content of the image to be processed includes: a first target area; a recognition module 702, used to perform optical character recognition on the image to be processed to obtain a recognition result; a second acquisition module 703, used to acquire multiple first candidate frames from the first target area based on the recognition result; a capture module 704, used to capture a partial image from the image to be processed using the multiple first candidate frames, wherein the display content of the partial image includes: a second target area, and the second target area is a partial area of ​​the first target area; a processing module 705, used to detect the partial image and obtain target key-value pair information corresponding to the second target area.

[0141] Optionally, the second acquisition module 703 is also used to: extract text features and image features from the recognition results; use the target neural network model to analyze the text features and image features, and obtain multiple first candidate boxes from the first target area, wherein the target neural network model is obtained through machine learning training using multiple sets of data, and each set of data in the multiple sets of data includes: a sample image and multiple candidate boxes identified in the sample image.

[0142] Optionally, the second acquisition module 703 is further used to: perform data annotation on the recognition result to obtain a target annotation result; and perform preprocessing on the target annotation result to obtain text features and image features.

[0143] Optionally, the second acquisition module 703 is also used to: in the recognition result, use the first mark to mark multiple first candidate boxes, and use the second mark to mark multiple second candidate boxes to obtain an initial marking result, wherein the first mark and the second mark are used to distinguish the target area where each candidate box is located; convert the initial marking result according to a preset data set format to obtain a target marking result.

[0144] Optionally, the second acquisition module 703 is also used to: transform the target annotation result by using a sequence annotation method to obtain a first processing result; connect the text fields displayed by multiple third candidate boxes contained in the target data unit to obtain a second processing result; based on the first processing result and the second processing result, encapsulate the attribute information of each candidate box in the multiple third candidate boxes to obtain a third processing result, wherein the attribute information includes: the starting index and the ending index of the text field displayed by each candidate box in the multiple third candidate boxes in the second processing result, and the label corresponding to each candidate box in the multiple third candidate boxes in the second processing result; segment the third processing result to obtain a fourth processing result; resize the image to be processed to obtain a fifth processing result; and determine text features and image features using the first processing result, the second processing result, the third processing result, the fourth processing result and the fifth processing result.

[0145] Optionally, the target neural network model includes: an embedding layer and a feature fusion extraction layer, and the second acquisition module 703 is also used to: input text features into the embedding layer to obtain a first output result, and input image features into the embedding layer to obtain a second output result; in the feature fusion extraction layer, perform feature fusion extraction on the first output result and the second output result to obtain a third output result; obtain a text sequence from the third output result; perform semantic entity recognition on the text sequence, and obtain multiple first candidate boxes from the first target area.

[0146] Optionally, the capture module 704 is further configured to: determine a second target area using coordinate information of the plurality of first candidate frames; and capture a portion of the image from the image to be processed according to the second target area.

[0147] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0148] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, including a memory and at least one processor, wherein the memory stores computer instructions, and the processor is configured to run the computer instructions to execute the steps in any one of the above method embodiments.

[0149] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0150] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0151] Step S1, obtaining an image to be processed, wherein the display content of the image to be processed includes: a first target area;

[0152] Step S2, performing optical character recognition on the image to be processed to obtain a recognition result;

[0153] Step S3, obtaining a plurality of first candidate frames from the first target area based on the recognition result;

[0154] Step S4, using a plurality of first candidate frames to intercept a partial image from the image to be processed, wherein the display content of the partial image includes: a second target area, which is a partial area of ​​the first target area;

[0155] Step S5: detect a portion of the image to obtain target key-value pair information corresponding to the second target area.

[0156] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0157] According to an embodiment of the present disclosure, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are configured to execute the steps of any of the above method embodiments when running.

[0158] Optionally, in this embodiment, the non-volatile storage medium may be configured to store a computer program for performing the following steps:

[0159] Step S1, obtaining an image to be processed, wherein the display content of the image to be processed includes: a first target area;

[0160] Step S2, performing optical character recognition on the image to be processed to obtain a recognition result;

[0161] Step S3, obtaining a plurality of first candidate frames from the first target area based on the recognition result;

[0162] Step S4, using a plurality of first candidate frames to intercept a partial image from the image to be processed, wherein the display content of the partial image includes: a second target area, which is a partial area of ​​the first target area;

[0163] Step S5: detect a portion of the image to obtain target key-value pair information corresponding to the second target area.

[0164] Optionally, in this embodiment, the above-mentioned non-transitory computer-readable storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0165] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product. The program code for implementing the image processing method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or completely on a remote machine or server.

[0166] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0167] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0168] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0169] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0170] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0172] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the present disclosure.

Claims

1. An image processing method, comprising: Acquire an image to be processed, wherein the display content of the image to be processed includes: a first target area, which is a complex table area in the image to be processed, and the complex table area is used to represent a table area for performing a formatting operation on a cell; Performing optical character recognition on the image to be processed to obtain a recognition result; Extracting text features and image features from the recognition results; Analyzing the text features and the image features using a target neural network model, and obtaining a plurality of first candidate boxes from the first target area, wherein the target neural network model is obtained by machine learning training using a plurality of sets of data, each set of the plurality of sets of data comprising: a sample image and a plurality of candidate boxes identified in the sample image, the plurality of first candidate boxes being used to represent a plurality of text display areas in the complex table area; Using the multiple first candidate frames to intercept a partial image from the image to be processed, wherein the display content of the partial image includes: a second target area, which is a partial area of ​​the first target area; The partial image is detected to obtain target key-value pair information corresponding to the second target area.

2. The method according to claim 1, wherein: The extracting text features and image features from the recognition result comprises: Performing data labeling on the recognition result to obtain a target labeling result; The target annotation result is preprocessed to obtain the text feature and the image feature.

3. The method according to claim 2, wherein: The data labeling of the recognition result to obtain the target labeling result includes: In the recognition result, the plurality of first candidate boxes are marked with a first mark, and the plurality of second candidate boxes are marked with a second mark, to obtain an initial marking result, wherein the first mark and the second mark are used to distinguish the target area where the candidate boxes are located; The initial annotation result is converted according to a preset data set format to obtain the target annotation result.

4. The method according to claim 2, wherein: The preprocessing of the target annotation result to obtain text features and image features includes: The target annotation result is transformed by using a sequence annotation method to obtain a first processing result; Connecting the text fields displayed by the plurality of third candidate boxes contained in the target data unit to obtain a second processing result; Based on the first processing result and the second processing result, encapsulating the attribute information of each candidate box in the plurality of third candidate boxes to obtain a third processing result, wherein the attribute information includes: the starting index and the ending index of the text field displayed by each candidate box in the plurality of third candidate boxes in the second processing result, and the label corresponding to each candidate box in the plurality of third candidate boxes in the second processing result; Performing word segmentation on the third processing result to obtain a fourth processing result; resizing the image to be processed to obtain a fifth processing result; The text feature and the image feature are determined using the first processing result, the second processing result, the third processing result, the fourth processing result, and the fifth processing result.

5. The method according to claim 1, wherein: The target neural network model includes: an embedding layer and a feature fusion extraction layer, and the using the target neural network model to analyze the text features and the image features to obtain the multiple first candidate boxes from the first target area includes: Inputting the text feature into the embedding layer to obtain a first output result, and inputting the image feature into the embedding layer to obtain a second output result; In the feature fusion extraction layer, feature fusion extraction is performed on the first output result and the second output result to obtain a third output result; Obtaining a text sequence from the third output result; Perform semantic entity recognition on the text sequence, and obtain the multiple first candidate boxes from the first target area.

6. The method according to claim 1, wherein: The using the plurality of first candidate frames to intercept the partial image from the image to be processed comprises: Determine the second target area using the coordinate information of the multiple first candidate frames; The partial image is intercepted from the image to be processed according to the second target area.

7. An image processing device, comprising: A first acquisition module is used to acquire an image to be processed, wherein the display content of the image to be processed includes: a first target area, which is a complex table area in the image to be processed, and the complex table area is used to represent a table area for performing a formatting operation on a cell; A recognition module, used for performing optical character recognition on the image to be processed to obtain a recognition result; A second acquisition module is used to: extract text features and image features from the recognition result; analyze the text features and the image features using a target neural network model, and obtain multiple first candidate boxes from the first target area, wherein the target neural network model is obtained by machine learning training using multiple sets of data, each set of data in the multiple sets of data includes: a sample image and multiple candidate boxes identified in the sample image, and the multiple first candidate boxes are used to represent multiple text display areas in the complex table area; A capture module, configured to capture a partial image from the image to be processed using the multiple first candidate frames, wherein the display content of the partial image includes: a second target area, which is a partial area of ​​the first target area; A processing module is used to detect the partial image and obtain target key-value pair information corresponding to the second target area.

8. The device according to claim 7, wherein: The second acquisition module is also used for: The recognition result is data labeled to obtain a target labeling result; and the target labeling result is preprocessed to obtain the text feature and the image feature.

9. The device according to claim 8, wherein: The second acquisition module is also used for: In the recognition result, the multiple first candidate boxes are labeled with a first mark, and the multiple second candidate boxes are labeled with a second mark to obtain an initial labeling result, wherein the first mark and the second mark are used to distinguish the target area where each candidate box is located; the initial labeling result is converted according to a preset data set format to obtain the target labeling result.

10. The device according to claim 8, wherein: The second acquisition module is also used for: The target annotation result is transformed by using a sequence annotation method to obtain a first processing result; the text fields displayed by multiple third candidate boxes contained in the target data unit are connected to obtain a second processing result; Based on the first processing result and the second processing result, the attribute information of each candidate box in the multiple third candidate boxes is encapsulated to obtain a third processing result, wherein the attribute information includes: the starting index and the ending index of the text field displayed by each candidate box in the multiple third candidate boxes in the second processing result, and the label corresponding to each candidate box in the multiple third candidate boxes in the second processing result; the third processing result is segmented to obtain a fourth processing result; the image to be processed is resized to obtain a fifth processing result; and the text features and the image features are determined using the first processing result, the second processing result, the third processing result, the fourth processing result and the fifth processing result.

11. The device according to claim 7, wherein: The target neural network model includes: an embedding layer and a feature fusion extraction layer, and the second acquisition module is also used for: The text features are input into the embedding layer to obtain a first output result, and the image features are input into the embedding layer to obtain a second output result; in the feature fusion extraction layer, feature fusion extraction is performed on the first output result and the second output result to obtain a third output result; a text sequence is obtained from the third output result; semantic entity recognition is performed on the text sequence, and the multiple first candidate boxes are obtained from the first target area.

12. The device according to claim 7, wherein: The interception module is also used for: The second target area is determined by using the coordinate information of the multiple first candidate frames; and the partial image is intercepted from the image to be processed according to the second target area.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are for causing a computer to execute a method according to any one of claims 1-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images