AI and RPA-based file annotation method, device, equipment and medium

Abstract

The present disclosure proposes a file annotation method, device, equipment and medium based on AI and RPA, which relates to the field of AI and RPA, wherein the method includes: the RPA system obtains a file annotation request; the RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request; the RPA system draws a target image corresponding to the file to be annotated according to the response result; the RPA system responds to the mouse event and determines the area range of the text annotation in the target image; the RPA system determines the text annotation result within the area range according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text fragment of the first text information. In this way, the extraction of text information in the picture and the selection of discontinuous text in the text can be realized, and at the same time, the text information within the annotated area range and the position information of the text fragments in the text information can be obtained, which can meet the needs of model training.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence (AI) and robotic process automation (RPA), and in particular to a file annotation method, device, equipment and medium based on AI and RPA. Background Art

[0002] RPA uses specific "robot software" to simulate human operations on computers and automatically execute process tasks according to rules.

[0003] AI is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0004] With the popularity of RPA, more and more companies are using RPA to help employees complete repetitive work. However, in the process of model training, a large number of manual annotations of files are still required to obtain training data. For example, training data is obtained by manually annotating PDF files or pictures, and document structure information and visual information are modeled, such as the general document pre-training model LayoutLM, which allows the model to perform multi-modal alignment in the pre-training stage.

[0005] However, the above-mentioned file annotation method cannot select discontinuous text and extract text on images, does not contain the location information of the text in the document, and cannot meet the needs of model training. Summary of the invention

[0006] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0007] To this end, the present disclosure proposes a file annotation method, device, equipment and medium based on AI and RPA, so that the RPA system can extract text information from the image and select discontinuous text in the text by determining the text annotation area range in the target image and the text annotation results within the area range. At the same time, the text information within the annotated area range and the position information of text fragments in the text information can be obtained, which can meet the needs of model training.

[0008] The first aspect of the present disclosure proposes a file annotation method based on AI and RPA, including: an RPA system obtains a file annotation request; wherein the file annotation request is used to annotate a file to be annotated; the RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request; the RPA system draws a target image corresponding to the file to be annotated according to the response result; the RPA system determines the area range of text annotation in the target image in response to a mouse event; the RPA system determines the text annotation result within the area range according to first text information obtained by performing optical character recognition (OCR) on the file to be annotated and position information corresponding to each text fragment of the first text information.

[0009] The second aspect of the present disclosure proposes a file annotation device based on AI and RPA, which is applied to the RPA system, including: an acquisition module, used to obtain a file annotation request; wherein the file annotation request is used to annotate a file to be annotated; a generation module, used to respond to the file annotation request and generate a response result corresponding to the file annotation request; a drawing module, used to draw a target image corresponding to the file to be annotated according to the response result; a first determination module, used to determine the area range of the text annotation in the target image in response to a mouse event; and a second determination module, used to determine the text annotation result within the area range according to first text information obtained by optical character recognition (OCR) of the file to be annotated and position information corresponding to each text fragment of the first text information.

[0010] An embodiment of the third aspect of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in the embodiment of the first aspect of the present disclosure is implemented.

[0011] The fourth aspect embodiment of the present disclosure provides a non-temporary computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect embodiment of the present disclosure is implemented.

[0012] A fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect embodiment of the present disclosure.

[0013] The technical solution provided by the embodiments of the present disclosure has the following beneficial effects:

[0014] Obtain a file annotation request through the RPA system; wherein the file annotation request is used to annotate the file to be annotated; the RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request; the RPA system draws a target image corresponding to the file to be annotated according to the response result; the RPA system responds to the mouse event and determines the area range of the text annotation in the target image; the RPA system determines the text annotation result within the area range according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text fragment of the first text information. Therefore, the RPA system realizes the extraction of text information in the image and the selection of discontinuous text in the text by determining the text annotation area range in the target image and the text annotation result within the area range. At the same time, the text information within the annotated area range and the position information of the text fragments in the text information can be obtained, which can meet the needs of model training.

[0015] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0017] Figure 1 A flowchart of a file annotation method based on AI and RPA provided in an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure;

[0019] Figure 3 A schematic diagram of location information of a target sub-picture corresponding to a sub-file to be annotated to which a region range provided in an embodiment of the present disclosure belongs;

[0020] Figure 4 A flowchart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure;

[0021] Figure 5 A flowchart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure;

[0022] Figure 6 A flowchart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure;

[0023] Figure 7A schematic diagram of the structure of a file annotation device based on AI and RPA provided in an embodiment of the present disclosure;

[0024] Figure 8 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0026] The following describes the AI ​​and RPA-based file annotation method, device, equipment, and medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0027] Figure 1 A flowchart of a file annotation method based on AI and RPA provided in an embodiment of the present disclosure.

[0028] The AI ​​and RPA-based file annotation method provided in the embodiment of the present disclosure can be applied to the AI ​​and RPA-based file annotation device in the embodiment of the present disclosure, and the device can be configured in an electronic device. The electronic device can be a personal computer, a mobile terminal, etc. The mobile terminal is, for example, a mobile phone, a tablet computer, a personal digital assistant, and other hardware devices with various operating systems.

[0029] like Figure 1 As shown, the file annotation method based on AI and RPA may include the following steps:

[0030] Step 101: The RPA system obtains a file annotation request, wherein the file annotation request is used to annotate a file to be annotated.

[0031] In the embodiment of the present disclosure, a user may send a file annotation request to the RPA system through an interactive interface, so that the RPA system annotates the file to be annotated according to the file annotation request. It should be noted that the text annotation request may be used to annotate the file to be annotated.

[0032] Step 102: The RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request.

[0033] Furthermore, the RPA system generates a response result corresponding to the annotation request based on the obtained file annotation request, wherein the response result may include: the file to be annotated corresponding to the file annotation request, the converted image corresponding to the file to be annotated; and the first text information corresponding to the file to be annotated and the position information corresponding to each text segment in the first text information obtained based on optical character recognition (OCR). The position information corresponding to the text segment may include the position information corresponding to each word and character in the text. The position information corresponding to each word and character may be the position of each word and character relative to the page, such as the coordinate information of the four vertices of the word or character relative to the page.

[0034] Step 103: The RPA system draws a target image corresponding to the file to be annotated according to the response result.

[0035] As an example, the file to be labeled in the response result may include one or more sub-files to be labeled. Different methods may be used to draw the target image corresponding to the file to be labeled according to the number of sub-files to be labeled.

[0036] As an example, when the file to be annotated includes multiple sub-files to be annotated, the RPA system can draw the target sub-images corresponding to the sub-files to be annotated based on the text information corresponding to the multiple sub-files to be annotated and the position information corresponding to each text fragment in the text information, splice the target sub-images, and obtain the target image corresponding to the file to be annotated.

[0037] As another example, when the file to be annotated includes a sub-file to be annotated, the RPA system can draw a target image corresponding to the file to be annotated based on the first text information corresponding to the file to be annotated and the position information corresponding to each text segment in the first text information.

[0038] Step 104: The RPA system determines the area range of the text annotation in the target image in response to the mouse event.

[0039] In an embodiment of the present disclosure, the RPA system can determine the area range of the text annotation in the target image according to the mouse event. For example, when the mouse event includes: a mouse click event, a mouse move event, and a mouse lift event in sequence, the area range of the text annotation determined by the mouse event can be determined.

[0040] In step 105, the RPA system determines a text annotation result within a region according to the first text information obtained by performing optical character recognition (OCR) on the document to be annotated and the position information corresponding to each text segment of the first text information.

[0041] In the embodiment of the present disclosure, the sub-file to be annotated to which the area range belongs can be determined based on the coordinate information of the area range, and then, the second text information corresponding to the sub-file to be annotated in the first text information and the respective text fragments of the second text information corresponding to the respective text fragments of the first text information can be obtained. Then, based on the position information of the area range relative to the sub-file to be annotated, the text annotation result within the area range is determined from the second text information and the respective text fragments of the second text information.

[0042] As an application scenario, for example, in tender notices and official documents, the document annotation method of the disclosed embodiment can convert unstructured long text into structured data and assist users in intelligently extracting key information of the document.

[0043] In summary, the file annotation request is obtained through the RPA system; wherein the file annotation request is used to annotate the file to be annotated; the RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request; the RPA system draws the target image corresponding to the file to be annotated according to the response result; the RPA system responds to the mouse event and determines the area range of the text annotation in the target image; the RPA system determines the text annotation result within the area range according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text fragment of the first text information. Therefore, the RPA system realizes the extraction of text information in the image and the selection of discontinuous text in the text by determining the text annotation area range in the target image and the text annotation result within the area range. At the same time, the text information within the annotated area range and the position information of the text fragments in the text information can be obtained, which can meet the needs of model training.

[0044] In order to obtain the text information within the marked area and the location information of the text fragments in the text information, such as Figure 2 As shown, Figure 2 A flow chart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure. In the embodiment of the present disclosure, the sub-file to be annotated to which the regional range belongs can be determined, and based on the position information of the sub-file to be annotated relative to the regional range and the position information of the sub-file to be annotated in the file to be annotated, the text annotation result within the regional range can be determined in the position information corresponding to the first text information and each text fragment of the first text information. Figure 2 The illustrated embodiment may include the following steps:

[0045] In step 201, the RPA system obtains a file annotation request, wherein the file annotation request is used to annotate the file to be annotated.

[0046] Step 202: The RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request.

[0047] Step 203: The RPA system draws a target image corresponding to the file to be annotated according to the response result.

[0048] Step 204: The RPA system determines the area range of the text annotation in the target image in response to the mouse event.

[0049] Step 205 : The RPA system determines the sub-file to be annotated to which the area range belongs based on the vertex coordinate information of the area range and the height information of the sub-file to be annotated in the file to be annotated.

[0050] In the disclosed embodiment, the RPA system may pre-set the height information of each subfile to be annotated, and then the RPA system may determine the height information of the vertices of the region range relative to the origin of the target sub-image corresponding to the subfile to be annotated based on the vertex coordinate information of the region range (e.g., the upper left vertex), and the subfile to be annotated to which the region range belongs may be determined based on the height information and the height information of the target sub-image corresponding to each subfile to be annotated. For example, if the height of the vertex of the region range relative to the origin of the target image corresponding to the subfile to be annotated is greater than the height of the target sub-image corresponding to one subfile to be annotated, and less than the height of the target sub-image corresponding to two subfiles to be annotated, it can be determined that the region range belongs to the second subfile to be annotated.

[0051] In step 206, the RPA system determines the position information of the region range relative to the target sub-image corresponding to the sub-file to be annotated to which the region range belongs.

[0052] For example, Figure 3As shown, taking the pdf file to be annotated as an example, the elements on the page from the outside to the inside are: window object window.document, drawing object canvas for drawing the pdf file, the position of canvas relative to document is left and top, and there is no spacing between page and canvas. The target sub-images (page1, page2, etc.) corresponding to multiple sub-files to be annotated in the pdf file, taking the target sub-image corresponding to the sub-file to be annotated to which the area range belongs as page2 as an example, the coordinates of the upper left corner of the area range are (x, y), and the position information of the area range relative to the target sub-image page2 corresponding to the sub-file to be annotated is relativeLeft=x-left, relativeRight=x-left+width, relativeTop=y-top-pageHeight*(PageNo-1), relativeBottom=relativeTop+height. Among them, width and height are the width and height of the area range respectively, and the width and height of the area range can be calculated based on the end coordinates and start coordinates of the annotation. For example, the starting coordinates of the area range marking are (x, y), the ending coordinates of the area range marking are (x1, y1), the width of the area range can be |x1-x|, and the height of the area range can be |y1-y|.

[0053] Step 207: The RPA system determines, based on the location information, text annotation results within the area range in the first text information and the location information corresponding to each text segment of the first text information.

[0054] Optionally, the RPA system determines the second text information corresponding to the subfile to be annotated belonging to the area scope in the first text information based on the position information of the subfile to be annotated belonging to the area scope in the file to be annotated; the RPA system determines the position information corresponding to each text fragment of the second text information in the position information corresponding to each text fragment of the first text information based on the correspondence between the second text information and the first text information; the RPA system determines the third text information in the second text information within the area scope based on the position information of the target sub-image corresponding to the subfile to be annotated belonging to the area scope; the RPA system determines the position information corresponding to each text fragment of the third text information in the position information corresponding to each text fragment of the second text information based on the correspondence between the third text information and the second text information; the RPA system uses the third text information and the position information corresponding to each text fragment of the third text information as the text annotation result within the area scope.

[0055] That is to say, after determining the subfile to be annotated to which the regional range belongs, the RPA system can determine the second text information corresponding to the subfile to be annotated in the first text information according to the position information of the subfile to be annotated in the file to be annotated. For example, the RPA system determines that the subfile to be annotated to which the regional range belongs is the second page in the file to be annotated, and can determine the second text information corresponding to the subfile to be annotated on the second page in the first text information. Then, the RPA system can determine the position information corresponding to each text segment of the second text information in the position information corresponding to each text segment of the first text information according to the correspondence between the second text information and the first text information. Further, the RPA system determines the third text information corresponding to the regional range in the second text information according to the position information of the target sub-image corresponding to the subfile to be annotated to which the regional range belongs. The RPA system determines the position information corresponding to each text segment of the third text information in the position information corresponding to each text segment of the second text information according to the correspondence between the third text information and the second text information; the RPA system uses the third text information and the position information corresponding to each text segment of the third text information as the text annotation result within the regional range.

[0056] In the disclosed embodiment, the RPA system can label and save the location information of the target sub-image of the region range relative to the sub-file to be labeled to which the region range belongs, as well as the text annotation result of the region range, as training data of the model to meet the needs of model training. For example, it can be used as training data for a general document pre-training model.

[0057] As an application scenario, the general document pre-training model can combine document structure information and visual information for multimodal alignment. The model can be applied to tasks such as form understanding, bill understanding, and document image classification.

[0058] In the embodiment of the present disclosure, steps 201-204 may be implemented in any manner in the embodiments of the present disclosure, and the embodiment of the present disclosure does not limit this and will not be described in detail.

[0059] In summary, the RPA system determines the subfile to be annotated to which the area range belongs based on the vertex coordinate information of the area range and the height information of the subfile to be annotated in the file to be annotated; the RPA system determines the location information of the target sub-image corresponding to the subfile to be annotated to which the area range belongs; the RPA system determines the text annotation result within the area range in the location information corresponding to the first text information and each text segment of the first text information based on the location information. In this way, the text annotation result within the area range can be accurately determined, thereby obtaining the text information within the annotated area range and the location information of the text segments in the text information.

[0060] In order to accurately determine the area range of the text annotation in the target image, extract the text information in the image and select the discontinuous text in the text, such as Figure 4 As shown, Figure 4 A flowchart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure is provided. In the embodiment of the present disclosure, the area range of the text annotation in the target image can be determined through mouse click events, mouse movement events and mouse lift events. Figure 4 The illustrated embodiment may include the following steps:

[0061] In step 401, the RPA system obtains a file annotation request, wherein the file annotation request is used to annotate the file to be annotated.

[0062] Step 402: The RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request.

[0063] Step 403: The RPA system draws a target image corresponding to the file to be annotated according to the response result.

[0064] In step 404, the RPA system monitors the mouse events of the target image; the mouse events include: mouse click events, mouse move events, and mouse lift events.

[0065] In an embodiment of the present disclosure, the RPA system can monitor the mouse events of the target icon through a monitoring function. When the mouse events include a mouse click event (mousedown event), a mouse move event (mousemove event), and a mouse up event (mouseup event) in sequence, the selection of the area range for text annotation in the target image can be determined.

[0066] Step 405: The RPA system determines the first coordinate of the area range according to the mouse click event.

[0067] Furthermore, the RPA system can monitor the mouse click event and use the coordinates of the mouse click event as the starting coordinates of the area range, that is, the first coordinate.

[0068] Step 406: The RPA system determines the second coordinate of the area range according to the mouse move event and the mouse lift event.

[0069] Furthermore, the RPA system can determine the end coordinate of the area range, that is, the second coordinate, by monitoring the mouse movement event and the mouse lift event.

[0070] In step 407, the RPA system determines the height and width of the area range according to the first coordinate and the second coordinate.

[0071] For example, the abscissa of the first coordinate can be subtracted from the abscissa of the second coordinate, and the absolute value of the subtraction result can be used as the width value of the region range. The ordinate of the first coordinate can be subtracted from the ordinate of the second coordinate, and the absolute value of the subtraction result can be used as the height value of the region range.

[0072] In step 408, the RPA system uses the enclosed area of ​​the first coordinate, the second coordinate, and the height and width values ​​of the area range as the area range of the text annotation in the target image.

[0073] In the disclosed embodiment, the RPA system adds the horizontal coordinate of the first coordinate to the width of the area range to obtain the third coordinate, and adds the vertical coordinate of the first coordinate to the height of the area range to obtain the fourth coordinate, and uses the enclosed area of ​​the first coordinate, the second coordinate, the third coordinate and the fourth coordinate as the area range of the text annotation in the target image. It should be noted that in order to more accurately determine the text annotation results within the area range, the number of area ranges for text annotation in the target image at the same time is one, and the area range can be determined by the label. accomplish.

[0074] In step 409, the RPA system determines a text annotation result within the area according to the first text information obtained by performing optical character recognition (OCR) on the document to be annotated and the position information corresponding to each text segment of the first text information.

[0075] In the embodiment of the present disclosure, steps 401-403 and 409 may be implemented in any manner in the embodiments of the present disclosure, respectively. The embodiment of the present disclosure does not limit this and will not be described in detail.

[0076] In summary, the RPA system monitors the mouse events of the target image; wherein the mouse events include: mouse click events, mouse movement events, and mouse lift events in sequence; the RPA system determines the first coordinate of the area range according to the mouse click event; the RPA system determines the second coordinate of the area range according to the mouse movement event and the mouse lift event; the RPA system uses the enclosed area of ​​the first coordinate, the second coordinate, and the height and width values ​​of the area range as the area range of the text annotation in the target image. Therefore, the RPA system can accurately determine the area range of the text annotation in the target image in response to the mouse event, and realize the extraction of text information in the image and the selection of discontinuous text in the text.

[0077] In order to obtain the response result corresponding to the request to be annotated, such as Figure 5 As shown, Figure 5 A flow chart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure. In the embodiment of the present disclosure, when the file to be annotated is not an image, the file to be annotated can be first converted into a converted image, and then, character recognition is performed on the converted image according to optical character recognition to obtain the first text information corresponding to the file to be annotated and the position information corresponding to each text fragment of the first text information. Figure 5 The illustrated embodiment may include the following steps:

[0078] Step 501: The RPA system obtains a file annotation request, wherein the file annotation request is used to annotate the file to be annotated.

[0079] Step 502: The RPA system obtains a file to be annotated corresponding to the file annotation request according to the file annotation request.

[0080] In an embodiment of the present disclosure, the RPA system may obtain the file to be annotated corresponding to the file annotation request according to the identifier of the file to be annotated in the file annotation request. The file annotation request may include the identifier of the file to be annotated.

[0081] In step 503, the RPA system performs image conversion on the file to be annotated, and obtains a converted image corresponding to the file to be annotated.

[0082] In the embodiment of the present disclosure, when the file to be annotated is not a picture, the file to be annotated may be converted into a picture.

[0083] As an example, the document to be annotated can be converted into an image through the document image conversion technology, and the converted image can be used as the conversion image. For example, the pdf.js plug-in can be used to convert the pdf file into an image.

[0084] In step 504, the RPA system performs character recognition on the converted image based on optical character recognition to obtain the first text information corresponding to the file to be annotated and the position information corresponding to each text segment of the first text information.

[0085] Furthermore, the RPA system performs character recognition on the conversion image based on optical character recognition, uses the recognized text information as the first text information corresponding to the file to be annotated, and uses the position information of each character or word in the recognized text information (e.g., the x and y axis coordinates of the four vertices of each character or word in the text information on the page) as the position information corresponding to each text fragment of the first text information. It should be noted that in order to avoid unclear images, the conversion image can be enlarged by a preset multiple, and the conversion image enlarged by the preset multiple is sent to the optical character recognition interface to perform character recognition on the conversion image.

[0086] In step 505, the RPA system takes the file to be annotated, the first text information corresponding to the file to be annotated, and the position information corresponding to each text segment of the first text information as a response result corresponding to the file annotation request.

[0087] In an embodiment of the present disclosure, the RPA system may use the file to be annotated corresponding to the file annotation request, the first text information corresponding to the file to be annotated, and the location information corresponding to each text fragment of the first text information as a response result corresponding to the file annotation request.

[0088] Step 506: The RPA system draws a target image corresponding to the file to be annotated according to the response result.

[0089] Step 507: The RPA system determines the area range of the text annotation in the target image in response to the mouse event.

[0090] In step 508, the RPA system determines a text annotation result within the area according to the first text information obtained by performing optical character recognition (OCR) on the document to be annotated and the position information corresponding to each text segment of the first text information.

[0091] In the embodiment of the present disclosure, steps 501, 506-508 can be implemented in any manner in the embodiments of the present disclosure, respectively. The embodiment of the present disclosure does not limit this and will not be described in detail.

[0092] In summary, the RPA system obtains the file to be annotated corresponding to the file annotation request according to the file annotation request; the RPA system converts the file to be annotated into an image to obtain the converted image corresponding to the file to be annotated; the RPA system performs character recognition on the converted image based on optical character recognition to obtain the first text information corresponding to the file to be annotated and the position information corresponding to each text segment of the first text information; the RPA system uses the file to be annotated, the first text information corresponding to the file to be annotated, and the position information corresponding to each text segment of the first text information as the response result corresponding to the file annotation request. Therefore, the RPA system can accurately obtain the response result corresponding to the request to be annotated according to the file annotation request.

[0093] In order to accurately draw the target image corresponding to the file to be labeled, such as Figure 6 As shown, Figure 6 A flow chart of another file annotation method based on AI and RPA provided in an embodiment of the present disclosure. In the embodiment of the present disclosure, a target sub-image corresponding to the sub-file to be annotated can be determined, and a target image corresponding to the file to be annotated can be determined based on multiple target sub-images. Figure 6 The illustrated embodiment may include the following steps:

[0094] In step 601, the RPA system obtains a file annotation request, wherein the file annotation request is used to annotate the file to be annotated.

[0095] Step 602: The RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request.

[0096] Step 603: The RPA system obtains multiple sub-files to be annotated of the file to be annotated in the response result.

[0097] In an embodiment of the present disclosure, the file to be annotated may include multiple sub-files to be annotated or one sub-file to be annotated. For example, the file to be annotated is a pdf file, and the number of pages of the pdf file may be one page or more. When the number of pages of the pdf file is one page, the file to be annotated includes only one sub-file to be annotated. When the number of pages of the pdf file is multiple pages, the RPA system may use each page of the pdf file as a sub-file to be annotated of the file to be annotated. A multi-page pdf file may include multiple sub-files to be annotated. For each sub-file to be annotated in a multi-page pdf file, the RPA system may identify each sub-file to be annotated, and the identification may be used to identify the position of each sub-file to be annotated in the file to be annotated.

[0098] Step 604: The RPA system creates a drawing object corresponding to each sub-file to be annotated.

[0099] In an embodiment of the present disclosure, for each sub-file to be annotated, a drawing object corresponding to the sub-file to be annotated can be created, and a page object can be created according to the attribute information of the sub-file to be annotated. For example, the drawing object is a canvas object, and the page object is a page object, and the page object includes the height information and width information of the sub-file to be annotated.

[0100] It can be understood that when a drawing object is created, the RPA system sets default width and height values ​​for it. In order to make the drawn target image correspond to the size of the file to be annotated (e.g., the same size), in the embodiment of the present disclosure, the RPA system can adjust the size information of the drawing object according to the attribute information of the sub-file to be annotated in the page object.

[0101] Step 605: The RPA system determines text information corresponding to the sub-file to be annotated in the first text information according to the position information of the sub-file to be annotated in the file to be annotated.

[0102] As an example, for each subfile to be annotated, the RPA system may identify each subfile to be annotated, and the identification may be used to identify the position of each subfile to be annotated in the file to be annotated. The RPA system may determine the text information corresponding to the subfile to be annotated in the first text information based on the position information of the subfile to be annotated in the file to be annotated. For example, if the position of the subfile to be annotated in the file to be annotated is the second page, the text information corresponding to the subfile to be annotated on the second page may be obtained in the first text information.

[0103] Step 606: Determine, based on the correspondence between the text information and the first text information, the position information corresponding to each text segment of the text information in the position information corresponding to each text segment of the first text information.

[0104] Furthermore, after determining the text information corresponding to the subfile to be annotated in the first text information, the position information corresponding to each text segment of the text information can be determined in the position information corresponding to each text segment of the first text information according to the correspondence between the text information and the first text information.

[0105] Step 607 : The RPA system draws the target sub-picture corresponding to the sub-file to be annotated according to the size information of the drawing object, the text information corresponding to the sub-file to be annotated, and the position information corresponding to each text segment of the text information.

[0106] Then, the RPA system draws the target sub-image corresponding to the sub-file to be annotated based on the size information of the drawing object, combined with the text information corresponding to the sub-file to be annotated and the position information corresponding to each text segment of the text information. It should be noted that the target sub-image has the same size as the sub-file to be annotated, and the target sub-image may include the text information of the sub-file to be annotated and the position information corresponding to each text segment of the text information.

[0107] In step 608, the RPA system stitches the target sub-images corresponding to the multiple sub-files to be annotated to obtain a target image.

[0108] Furthermore, the RPA system performs image splicing on the target sub-images corresponding to the multiple sub-files to be annotated, and uses the splicing result of the multiple target sub-images as the target image.

[0109] Step 609: The RPA system determines the area range of the text annotation in the target image in response to the mouse event.

[0110] In step 610, the RPA system determines a text annotation result within a region according to first text information obtained by performing optical character recognition (OCR) on the document to be annotated and position information corresponding to each text segment of the first text information.

[0111] In the embodiment of the present disclosure, steps 601-602, 609-610 may be implemented in any manner in the embodiments of the present disclosure, respectively. The embodiment of the present disclosure does not limit this and will not be described in detail.

[0112] In summary, the RPA system obtains multiple sub-files to be annotated of the file to be annotated in the response result; the RPA system creates a drawing object corresponding to the sub-file to be annotated for each sub-file to be annotated; according to the correspondence between the text information and the first text information, the position information corresponding to each text fragment of the text information is determined in the position information corresponding to each text fragment of the first text information; the RPA system draws the target sub-image corresponding to the sub-file to be annotated according to the size information of the drawing object, the text information corresponding to the sub-file to be annotated, and the position information corresponding to each text fragment of the text information; the RPA system splices the target sub-images corresponding to multiple sub-files to be annotated to obtain the target image. Therefore, the RPA system can accurately draw the target image corresponding to the file to be annotated according to multiple sub-files to be annotated, the text information corresponding to the sub-file to be annotated, and the position information corresponding to each text fragment of the text information.

[0113] The file annotation method based on AI and RPA of the disclosed embodiment obtains a file annotation request through the RPA system; wherein the file annotation request is used to annotate the file to be annotated; the RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request; the RPA system draws a target image corresponding to the file to be annotated according to the response result; the RPA system responds to the mouse event and determines the area range of the text annotation in the target image; the RPA system determines the text annotation result within the area range according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text fragment of the first text information. Thus, the RPA system realizes the extraction of text information in the image and the selection of discontinuous text in the text by determining the text annotation area range in the target image and the text annotation result within the area range, and can obtain the text information within the annotated area range and the position information of the text fragments in the text information, which can meet the needs of model training.

[0114] With the above Figures 1 to 6 Corresponding to the AI ​​and RPA-based file annotation method provided in the embodiment, the present disclosure also provides an AI and RPA-based file annotation device. Figures 1 to 6 The AI ​​and RPA-based file annotation methods provided in the embodiments correspond to each other, so the implementation of the AI ​​and RPA-based file annotation method is also applicable to the AI ​​and RPA-based file annotation device provided in the embodiments of the present disclosure, and will not be described in detail in the embodiments of the present disclosure.

[0115] Figure 7 A schematic diagram of the structure of a file annotation device based on AI and RPA provided in an embodiment of the present disclosure.

[0116] like Figure 7 As shown, the AI ​​and RPA-based file annotation device 700 may include: an acquisition module 710 , a generation module 720 , a drawing module 730 , a first determination module 740 and a second determination module 750 .

[0117] Among them, the acquisition module 710 is used to obtain a file annotation request; wherein the file annotation request is used to annotate the file to be annotated; the generation module 720 is used to respond to the file annotation request and generate a response result corresponding to the file annotation request; the drawing module 730 is used to draw a target image corresponding to the file to be annotated according to the response result; the first determination module 740 is used to determine the area range of the text annotation in the target image in response to the mouse event; the second determination module 750 is used to determine the text annotation result within the area range according to the first text information obtained by optical character recognition (OCR) of the file to be annotated and the position information corresponding to each text fragment of the first text information.

[0118] As a possible implementation method of the embodiment of the present disclosure, the second determination module 750 is used to: determine the sub-file to be annotated to which the area range belongs based on the vertex coordinate information of the area range and the height information of the sub-file to be annotated in the file to be annotated; determine the position information of the target sub-image corresponding to the sub-file to be annotated to which the area range belongs relative to the area range; and determine the text annotation result within the area range in the position information corresponding to the first text information and each text fragment of the first text information based on the position information.

[0119] As a possible implementation of the embodiment of the present disclosure, the second determination module 750 is further used to: determine the second text information corresponding to the subfile to be annotated belonging to the regional range in the first text information according to the position information of the subfile to be annotated belonging to the regional range in the file to be annotated; determine the position information corresponding to each text segment of the second text information in the position information corresponding to each text segment of the first text information according to the correspondence between the second text information and the first text information; determine the third text information within the regional range in the second text information according to the position information of the subfile to be annotated belonging to the regional range; determine the position information corresponding to each text segment of the third text information in the position information corresponding to each text segment of the second text information according to the correspondence between the third text information and the second text information; and use the third text information and the position information corresponding to each text segment of the third text information as the text annotation result within the regional range.

[0120] As a possible implementation method of the embodiment of the present disclosure, the first determination module 740 is used to: monitor mouse events of the target image; wherein the mouse events include: mouse click events, mouse movement events and mouse lift events in sequence; determine the first coordinate of the area range according to the mouse click event; determine the second coordinate of the area range according to the mouse movement event and the mouse lift event; determine the height value and width value of the area range according to the first coordinate and the second coordinate; use the enclosed area of ​​the first coordinate, the second coordinate and the height value and width value of the area range as the area range of the text annotation in the target image.

[0121] As a possible implementation method of the embodiment of the present disclosure, the generation module 720 is used to: obtain the file to be annotated corresponding to the file annotation request according to the file annotation request; perform image conversion on the file to be annotated to obtain a converted image corresponding to the file to be annotated; perform character recognition on the converted image based on optical character recognition (OCR) to obtain the first text information corresponding to the file to be annotated and the position information corresponding to each text fragment of the first text information; use the file to be annotated, the first text information corresponding to the file to be annotated, and the position information corresponding to each text fragment of the first text information as a response result corresponding to the file annotation request.

[0122] As a possible implementation of the embodiment of the present disclosure, the drawing module 730 is used to: obtain multiple sub-files to be annotated of the file to be annotated in the response result; create a drawing object corresponding to the sub-file to be annotated for each sub-file to be annotated in the file to be annotated; determine the text information corresponding to the sub-file to be annotated in the first text information according to the position information of the sub-file to be annotated in the file to be annotated; determine the position information corresponding to each text fragment of the text information in the position information corresponding to each text fragment in the first text information according to the correspondence between the text information and the first text information; draw the target sub-image corresponding to the sub-file to be annotated according to the size information of the drawing object, the text information corresponding to the sub-file to be annotated and the position information corresponding to each text fragment of the text information; and splice the target sub-images corresponding to multiple sub-files to be annotated to obtain the target image.

[0123] As a possible implementation of the embodiment of the present disclosure, the file annotation device 700 based on AI and RPA further includes: a processing module. The processing module is used to label and save the position information of the region range relative to the sub-file to be annotated to which the region range belongs, and the third text information within the region range and the position information corresponding to each text segment of the third text information, so as to serve as training data for the model.

[0124] The file annotation device based on AI and RPA in the embodiment of the present disclosure obtains a file annotation request through the RPA system; wherein the file annotation request is used to annotate the file to be annotated; the RPA system responds to the file annotation request and generates a response result corresponding to the file annotation request; the RPA system draws a target image corresponding to the file to be annotated according to the response result; the RPA system responds to the mouse event and determines the area range of the text annotation in the target image; the RPA system determines the text annotation result within the area range according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text fragment of the first text information. Thus, the RPA system realizes the extraction of text information in the image and the selection of discontinuous text in the text by determining the text annotation area range in the target image and the text annotation result within the area range, and can obtain the text information within the annotated area range and the position information of the text fragments in the text information, which can meet the needs of model training.

[0125] In order to implement the above embodiments, the embodiments of the present disclosure also propose an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the file annotation method based on AI and RPA as described in any of the above method embodiments is implemented.

[0126] In order to implement the above embodiments, the embodiments of the present disclosure also propose a non-temporary computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the file annotation method based on AI and RPA as described in any of the above method embodiments is implemented.

[0127] In order to implement the above embodiments, the embodiments of the present disclosure also propose a computer program product. When the instruction processor in the computer program product is executed, the file annotation method based on AI and RPA as described in any of the above method embodiments is implemented.

[0128] like Figure 8 As shown, Figure 8 It is a block diagram of an electronic device according to the AI ​​and RPA-based file annotation method provided in an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0129] like Figure 8 As shown, the electronic device includes: one or more processors 801, memory 802, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 801 is taken as an example.

[0130] The memory 802 is a non-transient computer-readable storage medium provided by the present disclosure. The memory stores instructions executable by at least one processor to enable the at least one processor to execute the file annotation method based on AI and RPA provided by the present disclosure. The non-transient computer-readable storage medium of the present disclosure stores computer instructions, which are used to enable a computer to execute the file annotation method based on AI and RPA provided by the present disclosure.

[0131] The memory 802 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs, and modules, such as program instructions / modules corresponding to the file annotation method based on AI and RPA in the embodiment of the present disclosure (for example, the attached Figure 7 The processor 801 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 802, that is, the file annotation method based on AI and RPA in the above method embodiment is implemented.

[0132] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the use of an electronic device generated according to a semantic representation model, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 802 may optionally include a memory remotely disposed relative to the processor 801, and these remote memories may be connected to an electronic device for file annotation based on AI and RPA via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The electronic device of the file annotation method based on AI and RPA may further include: an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803 and the output device 804 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0134] The input device 803 can receive input digital or character information, and generate key signal input related to user settings and function control of electronic devices generated based on AI and RPA file annotation, such as touch screen, keypad, mouse, trackpad, touchpad, indicator bar, one or more mouse buttons, trackball, joystick and other input devices. The output device 804 may include a display device, an auxiliary lighting device (e.g., LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0135] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0139] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0140] In addition, the acquisition, storage and application of information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions proposed in this disclosure can be achieved, and this document does not limit this.

[0142] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A file annotation method based on artificial intelligence AI and robotic process automation RPA, characterized in that: include: The RPA system obtains a file annotation request; wherein the file annotation request is used to annotate the file to be annotated; The RPA system generates a response result corresponding to the file annotation request in response to the file annotation request; The RPA system draws a target image corresponding to the file to be annotated according to the response result; The RPA system determines the area range of the text annotation in the target image in response to the mouse event; wherein the mouse event includes: a mouse click event, a mouse move event, and a mouse lift event in sequence; The RPA system determines the text annotation result within the area according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text segment of the first text information; The RPA system determines the text annotation result within the area according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text segment of the first text information, including: The RPA system determines the sub-file to be annotated to which the area range belongs according to the vertex coordinate information of the area range and the height information of the sub-file to be annotated in the file to be annotated; The RPA system determines the position information of the area range relative to the target sub-picture corresponding to the sub-file to be annotated to which the area range belongs; The RPA system determines, according to the position information of the target sub-picture corresponding to the sub-file to be annotated to which the area range belongs, in the first text information and the position information corresponding to each text segment of the first text information, the text annotation result within the area range; The RPA system draws a target image corresponding to the file to be annotated according to the response result, including: The RPA system obtains a plurality of to-be-annotated sub-files of the to-be-annotated file in the response result; The RPA system creates, for each of the sub-files to be annotated, a drawing object corresponding to the sub-file to be annotated; The RPA system determines, according to position information of the sub-file to be annotated in the file to be annotated, text information corresponding to the sub-file to be annotated in the first text information; Determining, according to the correspondence between the text information and the first text information, the position information corresponding to each text segment of the text information in the position information corresponding to each text segment of the first text information; The RPA system draws a target sub-picture corresponding to the sub-file to be annotated according to the size information of the drawing object, the text information corresponding to the sub-file to be annotated, and the position information corresponding to each text segment of the text information; The RPA system performs image splicing on the target sub-images corresponding to the multiple sub-files to be annotated to obtain a target image; The RPA system determines, in response to the mouse event, a region range of the text annotation in the target image, including: The RPA system monitors the mouse events of the target image; The RPA system determines the first coordinate of the area range according to the mouse click event; The RPA system determines a second coordinate of the area range according to the mouse movement event and the mouse lift event; The RPA system determines a height value and a width value of the area range according to the first coordinate and the second coordinate; The RPA system uses the first coordinate, the second coordinate, and an enclosed area of ​​the height value and the width value of the area range as the area range of the text annotation in the target image; The RPA system generates, in response to the file annotation request, a response result corresponding to the file annotation request, including: The RPA system obtains, according to the file annotation request, a file to be annotated corresponding to the file annotation request; The RPA system performs image conversion on the file to be annotated, and obtains a converted image corresponding to the file to be annotated; The RPA system performs character recognition on the converted image based on optical character recognition (OCR) to obtain first text information corresponding to the file to be annotated and position information corresponding to each text segment of the first text information; The RPA system uses the file to be annotated, the first text information corresponding to the file to be annotated, and the position information corresponding to each text segment of the first text information as a response result corresponding to the file annotation request.

2. The method according to claim 1, characterized in that The RPA system determines, according to the position information of the area range relative to the target sub-picture corresponding to the sub-file to be annotated to which the area range belongs, in the first text information and the position information corresponding to each text segment of the first text information, a text annotation result within the area range, including: The RPA system determines, according to position information of the sub-file to be annotated belonging to the area range in the file to be annotated, second text information corresponding to the sub-file to be annotated belonging to the area range in the first text information; The RPA system determines, according to a correspondence between the second text information and the first text information, position information corresponding to each text segment of the second text information from position information corresponding to each text segment of the first text information; The RPA system determines, according to position information of the area range relative to the to-be-annotated subfile to which the area range belongs, third text information within the area range in the second text information; The RPA system determines, according to the correspondence between the third text information and the second text information, the position information corresponding to each text segment of the third text information in the position information corresponding to each text segment of the second text information; The RPA system uses the third text information and the position information corresponding to each text segment of the third text information as the text annotation result within the area.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: The RPA system labels and saves the position information of the target sub-image corresponding to the sub-file to be labeled of the area range, as well as the third text information within the area range and the position information corresponding to each text fragment of the third text information, as training data for the model.

4. A file annotation device based on artificial intelligence AI and robotic process automation RPA, characterized in that: The file annotation device is applied to the RPA system, including: An acquisition module, used to acquire a file marking request; wherein the file marking request is used to mark the file to be marked; A generating module, configured to generate a response result corresponding to the file marking request in response to the file marking request; A drawing module, used for drawing a target image corresponding to the file to be annotated according to the response result; A first determination module is used to determine the area range of the text annotation in the target image in response to a mouse event; wherein the mouse event includes: a mouse click event, a mouse move event, and a mouse lift event in sequence; A second determination module is used to determine the text annotation result within the area according to the first text information obtained by performing optical character recognition (OCR) on the file to be annotated and the position information corresponding to each text segment of the first text information; The second determining module is used to determine the sub-file to be annotated to which the area range belongs according to the vertex coordinate information of the area range and the height information of the sub-file to be annotated in the file to be annotated; Determine the position information of the area range relative to the target sub-picture corresponding to the sub-file to be annotated to which the area range belongs; Determine the text annotation result within the area range in the first text information and the position information corresponding to each text segment of the first text information according to the position information of the target sub-picture corresponding to the sub-file to be annotated to which the area range belongs; The drawing module is used to obtain a plurality of sub-files to be annotated of the file to be annotated in the response result; For each sub-file to be annotated in the file to be annotated, creating a drawing object corresponding to the sub-file to be annotated; Determine, according to the position information of the sub-file to be annotated in the file to be annotated, the text information corresponding to the sub-file to be annotated in the first text information; Determining, according to the correspondence between the text information and the first text information, position information corresponding to each text segment of the text information in the position information corresponding to each text segment in the first text information; Draw a target sub-picture corresponding to the sub-file to be annotated according to the size information of the drawing object, the text information corresponding to the sub-file to be annotated, and the position information corresponding to each text segment of the text information; Splicing the target sub-images corresponding to the multiple sub-files to be labeled to obtain a target image; The first determining module is used to monitor the mouse events of the target image; Determine the first coordinate of the area range according to the mouse click event; Determine the second coordinate of the area range according to the mouse moving event and the mouse lifting event; Determine the height and width of the region according to the first coordinate and the second coordinate; Using the first coordinate, the second coordinate, and an enclosed area of ​​the height value and the width value of the area range as the area range of the text annotation in the target image; The generating module is used to obtain a file to be annotated corresponding to the file annotating request according to the file annotating request; Performing image conversion on the file to be annotated to obtain a converted image corresponding to the file to be annotated; Performing character recognition on the converted image based on optical character recognition (OCR) to obtain first text information corresponding to the file to be annotated and position information corresponding to each text segment of the first text information; The file to be annotated, the first text information corresponding to the file to be annotated, and the position information corresponding to each text segment of the first text information are used as a response result corresponding to the file annotation request.

5. The device according to claim 4, characterized in that The second determining module is further used for: Determining, according to the position information of the to-be-annotated subfile to which the area range belongs in the to-be-annotated file, the second text information corresponding to the to-be-annotated subfile to which the area range belongs in the first text information; Determining, according to the correspondence between the second text information and the first text information, the position information corresponding to each text segment of the second text information in the position information corresponding to each text segment of the first text information; Determining third text information in the second text information within the area range according to position information of the area range relative to the to-be-annotated subfile to which the area range belongs; Determining, according to the correspondence between the third text information and the second text information, position information corresponding to each text segment of the third text information in the position information corresponding to each text segment of the second text information; The third text information and the position information corresponding to each text segment of the third text information are used as the text annotation result within the area.

6. The device according to any one of claims 4 to 5, characterized in that: The device also includes: The processing module is used to label and save the position information of the area range relative to the sub-file to be labeled to which the area range belongs, as well as the third text information within the area range and the position information corresponding to each text segment of the third text information, as training data for the model.

7. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

9. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 3.

Images