Data extraction method and rpa robot making method for data extraction

Abstract

The present application relates to a kind of data extraction method and the RPA robot manufacturing method for data extraction, it is related to computer information processing technical field.Based on first software program, the RPA robot of automatic screenshot is made, to find the software interface screenshot where field is located, then based on second software program, the RPA robot of field corresponding content data extraction based on the screenshot obtained is made, so that the automatic production of RPA robot from entering software interface to field content data extraction can be realized, and splicing is carried out to obtain the RPA robot of user data extraction, and based on the splicing RPA robot, data extraction is carried out.Compared with prior art, the workload of RPA robot manufacturing is reduced, and the cost of manual production is reduced.

Description

Technical Field

[0001] This invention relates to the field of computer information processing technology, specifically to a data extraction method and a method for manufacturing an RPA robot for data extraction. Background Technology

[0002] The large number of stroke patients has placed a heavy burden on families and society. Therefore, after stroke patients are discharged from the hospital, the hospital must promptly upload 357 treatment data indicators from the patient's hospitalization to the designated stroke database. This is necessary for further quality control and data analysis of stroke cases, to optimize treatment measures, improve treatment rates, and reduce the disease burden.

[0003] With the rapid development of information technology and the widespread application of artificial intelligence, it is necessary to build a software robot based on RPA (Robotic Process Automation) technology to automatically extract, review, and report stroke center data, serving thousands of stroke centers nationwide and assisting hospitals in reporting stroke data. However, RPA development requires customized development for different vendors and / or data source software versions. Each company's system has many versions, and a new robot needs to be created for each version, leading to a surge in the workload of manual RPA creation. Therefore, reducing the workload of RPA robot creation has become an urgent problem to be solved. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a data extraction method and an RPA robot manufacturing method for data extraction, which has the characteristics of reducing the workload of RPA robot manufacturing and reducing labor costs.

[0005] In a first aspect, one embodiment provides a data extraction method, which extracts data based on an assembled first RPA robot, a second RPA robot, and a third RPA robot, including:

[0006] The software interface is accessed by logging into the software system using the first RPA robot, which is a manually created RPA robot.

[0007] The screenshot is taken after the second RPA robot performs operations on the software interface at preset coordinate positions and with preset operation types, so as to obtain a screenshot containing preset fields. The second RPA robot is an automatically generated RPA robot.

[0008] The required data at the preset location in the screenshot is extracted based on the third RPA robot. The second and third RPA robots are automatically generated RPA robots.

[0009] Secondly, one embodiment provides a method for creating an RPA robot for data extraction, comprising:

[0010] The first RPA robot is created based on manual operation. The first RPA robot is used to log in to the software system and enter the software interface.

[0011] A second RPA robot is created based on a preset first software program. The second RPA robot is used to operate the interface and obtain non-repeating screenshots, and the screenshots correspond to preset fields.

[0012] A third RPA robot is created based on a preset second software program. The third RPA robot is used to obtain the location of the content data corresponding to the preset field and extract the content data based on the acquired screenshot and the correspondence between the screenshot and the preset field.

[0013] The first RPA robot, the second RPA robot, and the third RPA robot are spliced ​​together to obtain an RPA robot for data extraction.

[0014] The beneficial effects of this invention are:

[0015] Because an RPA robot can be created first using a first software program to automatically capture screenshots of the software interface containing the field, and then a second software program can be used to create an RPA robot that extracts the content data corresponding to the field based on the obtained screenshots, the entire process from entering the software interface to extracting field content data can be automated. These extracted data are then combined to create a user data extraction RPA robot, which is then used for data extraction. This reduces the workload of creating RPA robots and lowers the cost of manual creation. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of an embodiment of the RPA robot manufacturing method for data extraction according to this application;

[0017] Figure 2 This is a schematic diagram of the field names and content data arrangement in the first embodiment of this application;

[0018] Figure 3 This is a schematic diagram of the field names and content data arrangement in the second embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the field names and content data arrangement in the third embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the field names and content data arrangement in the fourth embodiment of this application;

[0021] Figure 6 This is a schematic flowchart of a data extraction method according to an embodiment of this application. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0023] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0024] The serial numbers assigned to components in this article, such as "first" and "second", are used only to distinguish the objects being described and have no sequential or technical meaning.

[0025] To facilitate the explanation of the inventive concept of this application, the following is a brief description of the data extraction and RPA robot manufacturing technology.

[0026] Currently, data extraction at stroke centers requires manual intervention for each version of the data source software. The stroke center needs a total of 357 fields, with data originating from various systems. RPA (Robotic Process Automation) is required to create a robot for each field's data collection. Different hospitals are highly likely to use information systems provided by different companies, and these companies may offer multiple versions of the same system. With over a hundred HIS (Hospital Information System) software companies nationwide, finding the correct software and interface for a specific field is extremely time-consuming. If RPA could automatically locate the software system and interface where a field resides, it would significantly reduce manual costs.

[0027] However, traditional RPA robot creation methods make it difficult for RPA to automatically locate the software system and interface where the field is located. Therefore, it is difficult to automatically create RPA robots based on software programs, and manual operation is required to create RPA robots manually.

[0028] Therefore, this application provides a data extraction method and an RPA robot creation method for data extraction. First, an RPA robot that automatically captures screenshots is created based on a first software program to locate the screenshot of the software interface containing the field. Then, an RPA robot that extracts the content data corresponding to the field based on the obtained screenshot is created based on a second software program. This achieves automatic creation of the RPA robot from entering the software interface to extracting the field content data. The data is then combined to obtain the RPA robot for user data extraction, and data extraction is performed based on this combined RPA robot. This reduces the workload of RPA robot creation and lowers the cost of manual creation.

[0029] To more clearly illustrate the data extraction method of the embodiments of this application, the following first introduces the method for creating an RPA robot for data extraction.

[0030] This application provides a method for creating an RPA robot for data extraction. Please refer to [link / reference]. Figure 1 ,include:

[0031] Step S10: Create the first RPA robot based on manual operation. This first RPA robot is used to log in to the software system and enter the software interface.

[0032] Those skilled in the art will know that manufacturing RPA robots based on manual operation is a conventional manufacturing method in the existing technology, and will not be elaborated here.

[0033] Step S20: Create a second RPA robot based on a preset first software program. This second RPA robot is used to operate the interface and obtain unique screenshots, and the screenshots correspond to preset fields.

[0034] In one embodiment, step S20 may include:

[0035] Step S201: The operation interface is operated automatically according to the preset operation mode. The interface is screenshotted and saved after each operation, and the operation coordinate position corresponding to the screenshot after each operation is recorded.

[0036] After entering the software system's interface, different pages may appear at different operation points. For example, pop-ups may appear after certain operations, while further detailed information may appear after others. These new pages may contain fields that need to be extracted. Therefore, we need to take screenshots of the new pages after each operation to create an RPA robot with automatic screenshot functionality. This requires both automating operations on the pages and automatically taking screenshots of the pages after these operations to automate the RPA robot creation process.

[0037] In one embodiment, the user interface is automatically operated according to a preset operation method, including: performing operations position by position according to a preset coordinate position order. That is, in response to the preset operation sequence and operation method instructions, operations position by position are performed according to the preset coordinate position order. For example, after entering a certain user interface, automatic operations can be performed according to a preset left-to-right and top-to-bottom coordinate position order. Thus, after each coordinate position operation is completed, the operation of the next coordinate position is performed according to the preset coordinate position order, until all preset coordinate position operations are completed or all coordinate position operations of the entire page are completed. Those skilled in the art will understand that the coordinate position order here can also be other coordinate position orders, and those skilled in the art can set them according to actual needs, all of which are within the protection scope of this application.

[0038] In one embodiment, performing the coordinate-by-coordinate position operation according to a preset coordinate position order may further include: moving the coordinate position by 3px each time according to the preset coordinate position order. Those skilled in the art will understand that the size of the movement can be configured according to actual needs.

[0039] The software interface is operated to obtain a new interface. This operation may be a single click, a double click, or other types of operation. To ensure the completeness of the new interface display, in one embodiment, the operation is performed automatically according to a preset operation type, and the operation type corresponding to the screenshot after each operation is recorded. The operation type includes single click and double click operations.

[0040] In one embodiment, if there are two or more preset operation types, during the coordinate-by-coordinate position operation, all preset operation types can be executed at each operation position before proceeding to the next operation position. For example, after performing a single-click and double-click operation at the current operation position, the operation can proceed to the next coordinate position to perform the same single-click and double-click operation. Alternatively, one type of coordinate-by-coordinate position operation can be executed first, followed by the next type. Those skilled in the art will understand that each coordinate position operation requires the execution of all operation types. The specific operation method can be set according to requirements and is within the scope of protection of this application.

[0041] By taking screenshots of the interface after each operation and saving them, and recording the coordinates of each screenshot, we can determine the coordinates of the operation required to obtain the screenshot. If we subsequently create an RPA robot, we can then know the coordinates of the operation that needs to be automatically executed to obtain the target screenshot. In cases where there are multiple operation types, because we record the operation type corresponding to each screenshot, we can determine the type of operation that needs to be automatically executed to obtain the target screenshot. Therefore, we can determine the coordinates and type of the operation that needs to be automatically executed to obtain the target screenshot.

[0042] For example, clicking on the operation coordinates (x, y) of the current software page yields a screenshot M. The disease type field to be extracted can be found in screenshot M. If this screenshot M is used as the target screenshot to create an RPA robot, since the specific operation coordinates and operation type corresponding to the screenshot are known, the RPA robot that obtains the target screenshot can be automatically created based on the operation coordinates and operation type corresponding to the screenshot M, thereby realizing the automatic creation of the RPA robot.

[0043] Step S202: For the saved screenshots, identify the screenshots with the same interface and keep one of them, thus obtaining the screenshots with different interfaces that are kept.

[0044] For a software interface, since there is a screenshot for each preset coordinate position, there will be 1000 screenshots if there are 1000 preset coordinate positions. If there are two operation types for each coordinate position, there will be 2000 screenshots. However, the applicant found in the research that there are a large number of duplicate screenshots among these 2000 screenshots, that is, screenshots with the same interface. For example, it was found that 300 of the 2000 screenshots have the same interface, but we only need to keep one of the screenshots to obtain information. Therefore, in order to improve recognition efficiency and reduce the occupation of computer resources, in one embodiment, step S202 may include: for any screenshot after an operation, compare it with the saved screenshots, and determine whether there is a screenshot in the saved screenshots that is the same as the screenshot after the operation. If so, delete the screenshot after the operation; if not, save the screenshot after the operation, thereby obtaining the screenshots with different interfaces.

[0045] Based on the above embodiments, for each screenshot obtained after an operation, it is compared with the saved screenshots. For example, if the first screenshot is obtained after the first operation and the second screenshot is obtained after the second operation, the second screenshot can be compared with the first screenshot. It is determined whether there is a screenshot in the saved screenshots that is identical to the screenshot obtained after the arbitrary operation. If so, the screenshot obtained after the arbitrary operation is deleted; otherwise, the screenshot obtained after the arbitrary operation is saved. If the screenshot obtained after the m-th operation is identical to any of the previously saved screenshots, the screenshot obtained after the m-th operation is deleted to avoid screenshot duplication. If the screenshot obtained after the m-th operation is different from all the previously saved screenshots, the screenshot obtained after the m-th operation is saved. Because each screenshot is compared with the previously saved screenshots as soon as it is obtained, and deleted if they are identical, the storage space occupied can be reduced.

[0046] In one embodiment, as an alternative to the above embodiment, step S202 may include: saving screenshots after each operation in chronological order; based on all saved screenshots, for any current screenshot, determining whether there is a previous screenshot that is the same as the current screenshot, and if so, deleting the current screenshot, thereby obtaining the remaining screenshots with different interfaces.

[0047] Through any embodiment of step S202 above, duplicate images are cleaned up to ensure that each different interface has only one image.

[0048] In one embodiment, when there is only one operation type at each coordinate position, the screenshot can be saved in the following format:

[0049] NO_x_y.jpg

[0050] Where NO represents the screenshot number, starting from 1 and incrementing by 1; x and y represent the x-axis coordinates and y-axis coordinates of the operation location, respectively. For example, if a screenshot can be saved as 102_15_603.jpg, it means that the screenshot number is 102 and the operation location coordinates are (15, 603).

[0051] In cases where there are multiple operation types at each coordinate location, in one embodiment, the screenshot can be saved in the following format:

[0052] NO_x_y_click.jpg

[0053] In this context, NO represents the screenshot number, starting from 1 and incrementing by 1; x and y represent the x and y coordinates of the operation location, respectively; and click represents the operation type. For example, if a screenshot is saved as 102_15_603_1.jpg, it means the screenshot number is 102, the operation location coordinates are (15, 603), and the operation type is 1. If operation type 1 represents a click operation, then the operation type is a click operation. Therefore, if a screenshot with the format 102_15_603_1.jpg is the target image from which preset fields need to be extracted, the RPA robot that creates the screenshot can be obtained based on the screenshot's format information. That is, the information of the RPA robot can include taking a screenshot after a click operation at coordinates (15, 603) and saving it to the preset location.

[0054] Step S203: Based on the retained screenshots, obtain the preset fields, and obtain the correspondence between the screenshots containing any preset field and the preset fields.

[0055] Taking "department name" as one of the preset fields as an example, the screenshot containing "department name" can be identified by OCR. In this way, the correspondence between the preset fields and the screenshots can be obtained, and the screenshots without the preset fields can be deleted, leaving only the screenshots with the preset fields.

[0056] Step S204: Based on the correspondence and the page where the preset field is located, obtain the operation coordinates of the operation to be performed, and create an RPA robot that takes a screenshot of the page where the preset field is located.

[0057] Knowing the correspondence between screenshots and preset fields, the required screenshot for the preset field "Department Name" can be determined. In the above embodiment, since the operation coordinates corresponding to the screenshot are recorded, an RPA robot that generates the screenshot containing the "Department Name" field can be created based on the relationship between the screenshot and the operation coordinates.

[0058] In one embodiment, when there are multiple operation types, the operation coordinates and operation type of the operation to be performed are obtained based on the correspondence and the page where the preset field is located, and an RPA robot is created to take a screenshot of the page where the preset field is located.

[0059] In one embodiment, the second RPA robot created based on step S20 can be one RPA robot or multiple RPA robots. For example, it can be multiple RPA robots with one-to-one correspondence between page screenshots and RPA robots, or it can be one RPA robot that obtains multiple page screenshots.

[0060] By automatically operating the interface according to a preset operation method, taking a screenshot of the interface after each operation and saving it, and recording the operation coordinates corresponding to each screenshot, it is possible to obtain the screenshot that can be obtained at each operation coordinate position. For the saved screenshots, one screenshot with the same interface is retained, resulting in screenshots with different interfaces. Based on the retained screenshots, preset fields are obtained, and the correspondence between screenshots containing any preset field and the preset field itself is established. This allows the screenshot interface containing the preset field to be extracted to be located. Based on the above correspondence and the operation coordinates of the page containing the preset field, an RPA robot is created to take a screenshot of the page containing the preset field. This allows the coordinates of the operation required to obtain the screenshot interface to be obtained each time. Thus, a second RPA robot can be created to extract the preset field by taking a screenshot based on the coordinates of the operation required to extract the preset field. This achieves automatic creation of the second RPA robot for extracting the screenshot required for the preset field, reducing the workload of RPA robot creation and greatly reducing labor costs.

[0061] Step S30: Create a third RPA robot based on a preset second software program. This third RPA robot is used to obtain the location of the content data corresponding to the preset fields and extract the content data based on the acquired screenshots and the correspondence between the screenshots and preset fields.

[0062] In one embodiment, step S30 includes:

[0063] Step S301: Obtain a screenshot containing preset fields, and identify the preset fields based on the screenshot.

[0064] By obtaining a screenshot containing preset fields, the preset fields in the screenshot can be identified first.

[0065] In one embodiment, when the correspondence between preset fields and screenshots is unclear, to obtain a screenshot containing preset fields, the screenshot can be acquired first, and then, based on the acquired screenshot, the preset fields present in the screenshot can be identified through OCR, thus obtaining a screenshot containing preset fields. For example, it may be known that a certain screenshot contains preset fields, but it may not know which preset fields exist, or it may not even know whether preset fields exist in the screenshot at all. However, OCR recognition can be used to determine whether preset fields exist in the screenshot and which preset fields exist.

[0066] In one embodiment, when the correspondence between preset fields and screenshots is clear, to obtain a screenshot containing a preset field, the screenshot containing the content data corresponding to the preset field to be extracted can be directly obtained. For example, if both preset field A and preset field B correspond to screenshot M, and the content data corresponding to preset field A needs to be extracted, screenshot M can be directly obtained based on this correspondence.

[0067] Step S302: Based on the identified preset fields, identify the location of the content data corresponding to the preset fields.

[0068] Based on the identified preset fields, the location of the content data corresponding to the preset fields can be determined, thus revealing the location of the data content corresponding to the preset fields to be extracted.

[0069] The applicant discovered in their research that for the preset fields to be identified and their corresponding content data, there may or may not be corresponding text boxes. Please refer to [the relevant documentation / reference]. Figures 2 to 4 All of these are ways of displaying basic patient information, where name and gender are preset fields. Zhang San is the content data of the name, and male is the content data of the gender. Figure 2 and Figure 3 This refers to a scenario where there are predefined fields and their corresponding content data, each with its own corresponding text box. Figure 4 There are cases where there are preset fields and their corresponding content data, but no corresponding text boxes exist. Therefore, finding a solution that accommodates both of these situations becomes a technical challenge. In view of this problem, in one embodiment of this application, step S302 includes:

[0070] For any predefined field identified, such as "name", the check for the presence of first text content within it only includes the first text box of that predefined field, where the first text content does not include punctuation marks. If it is as follows... Figure 2 and Figure 3If a first text box exists, then within the first same content area, a search is performed to the right for the adjacent second text box. The first same content area refers to the area within a first preset distance threshold range from the right edge of the first text box, for example, the area within the first preset distance threshold range from the right edge of the first text box containing the name. If... Figure 2 As shown, if the border of the adjacent text box on the right is found when searching to the right within the first content area, it is assumed that there is a second adjacent text box on the right. The position of the second text box is then used as the position of the content data of any of the above preset fields (such as name).

[0071] If as Figure 3 As shown, if no border of the adjacent text box on the right is found in the first same content area, it means that the second text box is not to the right of the first text box, but below the first text box. Then, search downwards for the second text box that is aligned with the left end of the first text box below, and take the position of the second text box below as the position of the content data of any of the above preset fields.

[0072] In one embodiment, the location of the second text box can be represented by diagonal coordinates, or by the coordinates of three or four corners. If it is a diagonal coordinate point, it can be the top-left and bottom-right corner coordinates, or the bottom-left and top-right corner coordinates of the second text box.

[0073] Understandably, those skilled in the art can set the first preset distance threshold according to actual needs. In one embodiment, the first preset distance threshold is 10px.

[0074] If the first text box does not exist, then it may be Figure 4 In the case shown, where there is no first text box, the content data corresponding to the preset field may be to the right of the preset field or below the preset field. Therefore, in one embodiment, if there is no first text box, the first start coordinate point and the first end coordinate point of the preset field are identified. For example, the start coordinate point of the "surname" and the end coordinate point of the "given name" preset field are identified. In one embodiment, taking "name" as an example, the first start coordinate point is the upper left corner of the starting position, which is the upper left corner of the "surname", and the first end coordinate point is the lower right corner of the ending position, which is the lower right corner of the "given name". In another embodiment, the first start coordinate point is the lower left corner of the starting position, which is the lower left corner of the "surname", and the first end coordinate point is the upper right corner of the ending position, which is the upper right corner of the "given name".

[0075] First, determine if corresponding content data exists on the right. That is, based on the first ending coordinate point, search to the right within the second content area for the first adjacent character, and determine if the first adjacent character is a preset separator character. If so, based on the second ending coordinate point of the first character, search to the right within the second content area for the second adjacent character. If a second character is found, use the starting coordinate position of the found second character as the starting coordinate position of the content data for any of the preset fields mentioned above, thus finding the starting coordinate position of the content data corresponding to the preset field. The separator character includes ":" (colon) and / or "—" (dash). The applicant discovered in the research that since a preset field may be followed by ":" or "—", in this case, the identified first character is not the corresponding content data. Therefore, it is necessary to further search to the right in the same way for a second adjacent character. If a second character is found, use the starting coordinate position of the found second character as the starting coordinate position of the content data for any of the preset fields mentioned above. If the first character is not a preset separator character, use the starting coordinate position of the first character as the starting coordinate position of the content data for any of the preset fields mentioned above. The second identical content area refers to a preset field area with the same height as the based coordinate point within a second preset distance threshold range. For example, when searching for the first character, the search is based on the first end coordinate point, so the second identical content area is the preset field area with the same height as the first end coordinate point within a second preset distance threshold range to the right. When searching for the second character, the search is based on the second end coordinate point, so the second identical content area is the preset field area with the same height as the second end coordinate point within a second preset distance threshold range to the right.

[0076] Understandably, those skilled in the art can set the second preset distance threshold according to actual needs. In one embodiment, the second preset distance threshold is 10px.

[0077] In some embodiments, the starting coordinate position of the first character includes the coordinate position of the top left corner and / or the coordinate position of the bottom left corner of the first character; the starting coordinate position of the second character includes the coordinate position of the top left corner and / or the coordinate position of the bottom left corner of the second character.

[0078] If the first character or the second character is not found, it means that the corresponding content data is not on the right side of the preset field but below. Then, use the X-axis coordinate of the first starting coordinate point as the starting abscissa of the next line to search for the adjacent third character below to the right, and use the starting coordinate position of the third character as the starting coordinate position where the content data of any of the above preset fields is located. Since the starting position of the content data and the starting position of the preset field may not be aligned vertically, use the X-axis coordinate of the first starting coordinate point as the starting abscissa of the next line to search for the adjacent third character below to the right.

[0079] In one embodiment, the starting coordinate position of the third character includes the upper left corner coordinate position and / or the lower left corner coordinate position of the third character.

[0080] Step S303, make an RPA robot, including the screenshot ID, the corresponding preset field, and the position where the content data corresponding to the corresponding preset field is located.

[0081] In one embodiment, making an RPA robot includes the screenshot ID, the corresponding preset field, the algorithm for extracting content based on the content of the second text box, and the position of the second text box corresponding to the corresponding preset field. Based on the description of the above step S20, for example, as Figure 2 and Figure 3 shown, based on the obtained screenshot ID, the corresponding preset field "Name", the algorithm for extracting content based on the content of the second text box, that is, the algorithm for extracting all content data in the second text box based on the second text box where "Zhang San" is located, and the position of the second text box corresponding to the corresponding preset field, that is, the position of the second text box where "Zhang San" is located, to make a data extraction RPA corresponding to the content data of the preset field "Name". In this way, the production of the data extraction RPA robot is realized.

[0082] In one embodiment, in the case where there is no first text box, an RPA robot can be made based on the screenshot ID, the corresponding preset field, the starting coordinate position where the content data corresponding to the corresponding preset field is located, the text dynamic extension recognition algorithm, and semantic recognition. Among them, text dynamic extension recognition includes extending and recognizing adjacent characters to the right and down based on the starting coordinate position where the content data is located, and judging whether the adjacent characters belong to the content data to be extracted based on semantic recognition, so as to obtain the content data to be extracted. For example, please refer to Figure 5 , based on the obtained screenshot ID, the corresponding preset field "Department Name", there is a second character on the right, so use the starting coordinate position of the second character "New" as the starting coordinate position where the content data is located to extend and recognize adjacent characters to the right and down. In one embodiment, extending to the right is based on the end coordinate point of the current character to search whether there is an adjacent character on the right in the second same content area until the end. In Figure 5 In the illustrated embodiment, the rightward extension continues until it reaches the end of "three". The downward extension is based on the starting coordinate position of the content data. That is, the X-axis coordinate of the starting coordinate position of "new" is the starting abscissa of the next line, and it searches for the third character adjacent below to the right. By combining the above rightward and downward extensions to identify adjacent characters, and semantic recognition, the content data to be extracted is obtained. In Figure 5 In the embodiment of [], after the RPA robot obtains the starting coordinate position of the content data, it can obtain the content data to be extracted, "Neonatal and Maternal and Child Health Department", based on the text dynamic extension recognition algorithm and semantic recognition plug-in set inside the RPA robot. Those skilled in the art can understand that since the semantic recognition model can adopt existing technology models and can be used as a plug-in to be called during the execution of the RPA robot, the specific semantic recognition algorithm will not be elaborated here.

[0083] In one embodiment, the third RPA robot produced based on step S30 can be one RPA robot or multiple RPA robots. For example, it can be multiple RPA robots corresponding one by one to page screenshots or preset fields, or it can be one RPA robot that obtains multiple preset field data.

[0084] Based on the manufacturing method of the third RPA robot in the above one embodiment, by obtaining a screenshot with a preset field and recognizing the preset field based on the screenshot, the screenshot ID corresponding to the preset field can be obtained, and the preset field can be recognized from the screenshot obtained. Based on the recognized preset field, the position where the content data corresponding to the preset field is located can be recognized, and the position where the data content corresponding to the preset field to be extracted is located can be obtained. In this way, an RPA robot can be produced based on the screenshot ID, the corresponding preset field, and the position where the content data corresponding to the preset field is located, so as to obtain an RPA robot for data extraction based on position positioning, reducing the workload of manufacturing the RPA robot and greatly reducing the labor cost.

[0085] Step S40, splice the first RPA robot, the second RPA robot, and the third RPA robot to obtain an RPA robot for data extraction.

[0086] A data extraction method provided in an embodiment of the present application realizes data extraction based on the assembled first RPA robot, second RPA robot, and third RPA robot. Please refer to Figure 6 , the data extraction method includes:

[0087] Step S100, log in to the software system through the first RPA robot to enter the software interface. The first RPA robot is an RPA robot made manually.

[0088] Step S200: After the second RPA robot performs operations on the software interface at preset coordinate positions and with preset operation types, a screenshot is taken to obtain a screenshot containing preset fields. The first RPA robot is a manually created RPA robot.

[0089] Step S300: Extract the required data from the preset location in the screenshot based on the third RPA robot, which is an automatically generated RPA robot.

[0090] In one embodiment, for the manufactured second and third RPA robots, regardless of whether each includes multiple RPA robots, if the second and third RPA robots are treated as a whole and then spliced ​​together in step S40, then in step S200, based on the second RPA robot according to the preset operation coordinate position order and operation type, a screenshot is taken after each operation, and all screenshots are saved to a preset path in a preset save format. In step S300, the saved screenshots are obtained based on the preset path, and based on the correspondence between the screenshot ID and preset fields, the location of the content data corresponding to the preset fields is identified, and the content data corresponding to the preset fields is extracted.

[0091] For example, for the first screenshot saved in the preset path, based on the RPA robot production method, it can be known that the RPA robot program has a correspondence between the screenshot ID and the preset field, as well as the location of the content data corresponding to the preset field. Based on this, the content data corresponding to the preset field can be extracted.

[0092] In one embodiment, for the created second and third RPA robots, the second RPA robot includes multiple RPA robots, and the third RPA robot also includes multiple RPA robots. For the multiple RPA robots included in the second RPA robot, there is a one-to-one correspondence between page screenshots and RPA robots; for the multiple RPA robots included in the third RPA robot, there is a one-to-one correspondence between data extraction screenshots and RPA robots. Therefore, in step S40, the multiple RPA robots in the second RPA robot and the multiple RPA robots in the third RPA robot can be concatenated in a one-to-one correspondence. For example, if the second RPA robot includes I screenshot RPA robots and the third RPA robot includes J data extraction RPA robots, then I = J, and the i-th screenshot RPA robot is concatenated with the j-th data extraction RPA robot, where i = j, 1 ≤ i ≤ I, and 1 ≤ j ≤ J. Therefore, in one embodiment, step S200, taking a screenshot after the second RPA robot performs operations on the software interface at preset coordinate positions and with preset operation types, includes: taking a screenshot after each operation by the second RPA robot according to the preset operation coordinate position sequence and operation type. In step S300, after each screenshot operation, based on the obtained screenshot and its correspondence with preset fields, the location of the content data corresponding to the preset fields is identified, and the content data corresponding to the preset fields is extracted. That is, for each screenshot obtained by the RPA robot, the content data of the preset fields in the screenshot is extracted by the corresponding data extraction RPA robot. Thus, after data extraction, the screenshot can be deleted without needing to be saved separately, saving storage space.

[0093] In one embodiment, identifying the location of the content data corresponding to a preset field and extracting the content data corresponding to the preset field includes: identifying whether the location of the content data is a single coordinate position or a box containing two or more coordinate positions. If it is a single coordinate position, it indicates that the coordinate position is the starting coordinate. The content data corresponding to the preset field is extracted using this single coordinate position, combined with a dynamic text extension algorithm and semantic recognition. The dynamic extension algorithm can be the one described above and will not be repeated here. If it is a box containing two or more coordinate positions (appearing to be a text box position), the data within the box is directly extracted as the content data corresponding to the preset field.

[0094] Based on the data extraction method and RPA robot creation method for data extraction described in the above embodiments, the RPA robot can be automatically created by first taking screenshots using a first software program to locate the software interface screenshot where the field is located. Then, an RPA robot is created using a second software program to extract the content data corresponding to the field based on the obtained screenshot. This enables the automatic creation of an RPA robot from entering the software interface to extracting the field content data, and the data is then concatenated to obtain an RPA robot for user data extraction. Data extraction is performed based on this concatenated RPA robot. This reduces the workload of RPA robot creation and lowers the cost of manual creation.

[0095] One embodiment of this application provides a computer-readable storage medium storing a program, the stored program including methods that can be loaded by a processor and processed in any of the above embodiments.

[0096] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0097] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A data extraction method, characterized in that, Data extraction is achieved based on the assembled first, second, and third RPA robots, including: The software interface is accessed by logging into the software system using the first RPA robot, which is a manually created RPA robot. The screenshot is taken after the software interface is operated by the second RPA robot at a preset coordinate position and with a preset operation type, so as to obtain a screenshot containing preset fields. The second RPA robot is an automatically generated RPA robot. The screenshot after the software interface is operated by the second RPA robot at a preset coordinate position and with a preset operation type includes: taking a screenshot after each operation based on the second RPA robot according to the preset operation coordinate position sequence and operation type. The required data existing in the preset position in the screenshot is extracted based on the third RPA robot. The second RPA robot and the third RPA robot are automatically generated RPA robots. The extraction of the required data existing in the preset position in the screenshot based on the third RPA robot includes: after each screenshot operation, based on the obtained screenshot and the correspondence between the screenshot and the preset field, identifying the location of the content data corresponding to the preset field, and extracting the content data corresponding to the preset field. The process of identifying the location of content data corresponding to a preset field and extracting the content data corresponding to the preset field includes: The system identifies whether the location of the content data is a single coordinate position or a boxed position containing two or more coordinate positions. If it is a single coordinate position, the system uses that single coordinate position as the starting coordinate position and combines a text dynamic extension algorithm and semantic recognition to extract the content data corresponding to the preset field. If it is a boxed position containing two or more coordinate positions, the system directly extracts the data within the boxed position as the content data corresponding to the preset field. The text dynamic extension algorithm includes identifying adjacent characters by extending to the right and downward based on the starting coordinate position of the content data, and determining whether the adjacent characters belong to the content data to be extracted based on semantic recognition, thereby obtaining the content data to be extracted.

2. A method for creating an RPA robot for data extraction, characterized in that, include: The first RPA robot is created based on manual operation. The first RPA robot is used to log in to the software system and enter the software interface. A second RPA robot is created based on a preset first software program. The second RPA robot is used to operate the interface and obtain non-repeating screenshots, and the screenshots correspond to preset fields. The method for creating a second RPA robot based on a preset first software program includes: automatically operating the interface according to a preset operation method; taking screenshots of the interface after each operation and saving them, and recording the operation coordinates corresponding to each screenshot; for the saved screenshots, identifying screenshots with the same interface and keeping one of them, thus obtaining screenshots with different interfaces; based on the saved screenshots, obtaining preset fields, and obtaining the correspondence between screenshots containing any preset field and the preset fields; based on the correspondence and the page where the preset fields are located, obtaining the operation coordinates of the operation to be performed, and creating an RPA robot that takes screenshots of the page where the preset fields are located; The automatic operation of the interface according to a preset operation method includes: performing automatic operation according to a preset operation type and recording the operation type corresponding to the screenshot after each operation, wherein the operation type includes single-click operation and double-click operation; the creation of an RPA robot that takes a screenshot of the page containing the preset field based on the corresponding relationship and the page containing the preset field includes: obtaining the operation coordinates and operation type of the operation to be performed based on the corresponding relationship and the page containing the preset field, and creating an RPA robot that takes a screenshot of the page containing the preset field; A third RPA robot is created based on a preset second software program. This third RPA robot is used to extract the location of content data corresponding to a preset field based on an acquired screenshot and the correspondence between the screenshot and preset fields. The creation of the third RPA robot based on the preset second software program includes: acquiring a screenshot containing a preset field and identifying the preset field based on the screenshot; identifying the location of the content data corresponding to the preset field based on the identified preset field; and creating an RPA robot, including a screenshot ID, the corresponding preset field, and the location of the content data corresponding to the preset field. The first RPA robot, the second RPA robot, and the third RPA robot are spliced ​​together to obtain an RPA robot for data extraction.

3. The RPA robot manufacturing method as described in claim 2, characterized in that: The method of identifying the location of content data corresponding to a preset field based on the identified preset field includes: For any pre-defined field that is identified, the identification of whether there is first text content in it only includes the first text box of the pre-defined field, and the first text content does not include punctuation marks; If a first text box exists, then within the first same content area, search to the right for an adjacent second text box. If an adjacent second text box exists, the location of that second text box is taken as the location of the content data of any preset field. If no adjacent second text box exists, then search downwards for a second text box aligned with the left edge of the first text box, and take the location of that adjacent second text box as the location of the content data of any preset field. The location of the second text box includes: the coordinates of the upper left corner and the lower right corner of the second text box, or the coordinates of the lower left corner and the upper right corner of the second text box. The first same content area refers to the area within a first preset distance threshold range from the right edge of the first text box. If the first text box does not exist, the first start coordinate point and the first end coordinate point of the preset field are identified; based on the first end coordinate point, the adjacent first character is searched to the right within the second same content area, and it is determined whether the adjacent first character is a preset spacing character. If so, based on the second end coordinate point of the first character, the adjacent second character is searched to the right within the second same content area. If the second character is found, the starting coordinate position of the found second character is used as the starting coordinate position of the content data of any preset field; if the first character is not a preset spacing character, the starting coordinate position of the first character is used as the starting coordinate position of the content data of any preset field; if neither the first nor the second character is found, the adjacent third character below is searched to the right with the X-axis coordinate of the first start coordinate point as the starting horizontal coordinate of the next row, and the starting coordinate position of the third character is used as the starting coordinate position of the content data of any preset field; the second same content area refers to the preset field area with the same height as the coordinate point on which it is located within the second preset distance threshold range, and the spacing character includes ":" and / or "—".

Images