Rpa exception handling method and device, server and readable storage medium

Abstract

The application provides an RPA exception processing method and device, a server and a readable storage medium. A template file to be identified is acquired, multiple frames of images are extracted from the template file to be identified, pixel chromatic aberration values and pixel similarity of the multiple frames of images are calculated, the average similarity of the template file to be identified is determined according to the pixel chromatic aberration values and the pixel similarity of the multiple frames of images, the correlation between the template file to be identified and all target video files in a pre-established feature code library is determined based on the average similarity, when the template file to be identified is correlated with any target video file, an exception response processing scheme corresponding to the correlated target video file is queried from a pre-established exception processing scheme library, and the exception response processing scheme is used for processing the exception of the RPA. The method solves the problem of time-consuming and labor-intensive manual operation, improves the efficiency of exception repair, improves the user experience, and overcomes the problem of time-consuming and labor-intensive manual exception processing and low efficiency.

Description

Technical Field

[0001] This application relates to the field of RPA technology, specifically to an RPA exception handling method, apparatus, server, and readable storage medium. Background Technology

[0002] Robotic Process Automation (RPA) is software that runs on personal computers or servers. Its main function is to simulate human operation of various system software and automatically execute business processes. It is suitable for high-volume, highly repetitive business scenarios. Currently, RPA can be applied to functional scenarios in various industries. Common automated actions include automatic material entry, Office document editing, intelligent approval, information entry, business system operation, report information processing, invoice processing, file upload and download, content verification, data collection, and data processing. It involves business software systems with B / S or C / S architectures, such as browsers, Office software, SAP, and ERP.

[0003] RPA boasts unique advantages: low-code and non-intrusive. Low-code means RPA can be operated without requiring high IT skills; even business personnel without programming knowledge can develop processes. Non-intrusive means RPA can simulate human operations without requiring software systems to expose interfaces. While RPA offers many advantages and is very convenient to use, it can also experience anomalies. Currently, when addressing anomalies in RPA automated processes, the workload and pressure are gradually increasing with the growing number of customer processes. The current approach to handling anomalies relies on customers using RPA products and RPA process maintenance personnel proactively identifying, troubleshooting, and resolving issues. This approach, when generating a large volume of maintenance work, cannot respond quickly and efficiently, and manual anomaly troubleshooting and handling are time-consuming, labor-intensive, and inefficient. Summary of the Invention

[0004] In view of this, this application provides an RPA exception handling method, apparatus, server, and readable storage medium to overcome the problems of time-consuming, labor-intensive, and inefficient exception handling in existing systems.

[0005] In a first aspect, embodiments of this application provide an RPA exception handling method, the method comprising:

[0006] Obtain the template file to be identified, wherein the template file to be identified is determined based on key information and abnormal task information extracted from the screen recording file generated when an exception occurs in the RPA process;

[0007] Extract multiple frames of images from the template file to be identified, and calculate the pixel color difference value and pixel similarity of the multiple frames of images;

[0008] The average similarity of the template file to be identified is determined based on the pixel color difference value and pixel similarity of multiple frames of images;

[0009] The correlation between the template file to be identified and all target video files in the pre-established feature code library is determined based on the average similarity; wherein, the feature code library is established based on historical anomaly identification template files and includes multiple target video files;

[0010] When the template file to be identified is related to any of the target video files, an anomaly response processing scheme corresponding to the relevant target video file is retrieved from a pre-established anomaly handling scheme library; wherein, the anomaly response processing scheme is used to handle RPA anomalies.

[0011] Secondly, embodiments of this application provide an RPA exception handling apparatus, the apparatus comprising:

[0012] The file acquisition module is used to acquire the template file to be identified, wherein the template file to be identified is determined based on the key information and abnormal task information extracted from the screen recording file generated when an exception occurs in the RPA process;

[0013] The calculation module is used to extract multiple frames of images from the template file to be identified, and calculate the pixel color difference value and pixel similarity of the multiple frames of images;

[0014] The average similarity determination module is used to determine the average similarity of the template file to be identified based on the pixel color difference values ​​and pixel similarity of multiple frames of images.

[0015] The correlation determination module is used to determine the correlation between the template file to be identified and all target video files in the pre-established feature code library based on the average similarity; wherein, the feature code library is established based on historical anomaly identification template files and includes multiple target video files;

[0016] The scheme determination module is used to query the anomaly response processing scheme corresponding to the relevant target video file from a pre-established anomaly processing scheme library when the template file to be identified is related to any of the target video files; wherein, the anomaly response processing scheme is used to process RPA anomalies.

[0017] Thirdly, embodiments of this application provide a server, including: a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the RPA exception handling method provided in the first aspect above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be called by a processor to execute the RPA exception handling method provided in the first aspect above.

[0019] The RPA anomaly handling method, apparatus, server, and readable storage medium provided in this application first obtain a template file to be identified, which is determined based on key information and anomaly task information extracted from screen recordings generated when an anomaly occurs in the RPA process. Then, multiple frames of images are extracted from the template file to be identified, and the pixel color difference values ​​and pixel similarity of the multiple frames are calculated. The average similarity of the template file to be identified is determined based on the pixel color difference values ​​and pixel similarity of the multiple frames. Next, the correlation between the template file to be identified and all target video files in a pre-established feature code library is determined based on the average similarity. The feature code library is established based on historical anomaly identification template files and includes multiple target video files. When the template file to be identified is related to any target video file, an anomaly response handling scheme corresponding to the relevant target video file is retrieved from a pre-established anomaly handling scheme library. The anomaly response handling scheme is used to handle RPA anomalies.

[0020] The RPA anomaly handling method provided in this application embodiment can automatically retrieve anomaly response handling solutions related to the current anomaly from a pre-established anomaly handling solution library when an RPA anomaly is detected, and output them to the corresponding RPA automation process for repair. Compared with the manual anomaly repair method in related technologies, the technical solution provided in this application embodiment solves the problem of time-consuming and labor-intensive manual operation, improves the efficiency of anomaly repair, and enhances the user experience, thereby overcoming the problems of time-consuming, labor-intensive, and inefficient manual anomaly handling. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a schematic diagram illustrating an application scenario of the RPA exception handling method provided in the embodiments of this application;

[0023] Figure 2 This is a flowchart illustrating an RPA exception handling method provided in one embodiment of this application;

[0024] Figure 3This is a schematic diagram of the structure of an RPA exception handling device provided in one embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of a server provided in one embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium provided in one embodiment of this application. Detailed Implementation

[0027] The technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0028] To provide a more detailed description of this application, the following description, in conjunction with the accompanying drawings, details an RPA exception handling method, apparatus, server, and computer-readable storage medium provided in this application.

[0029] Please refer to Figure 1 , Figure 1 This diagram illustrates an application scenario of the RPA anomaly handling method provided in this embodiment. The application scenario includes a terminal device 102 and a server 104, with a network connecting the server 104 and the terminal device 102. The network serves as the medium for providing a communication link between the terminal device 102 and the server 104. The terminal device 102 interacts with the server 104 via the network to receive or send messages, etc. The server 104 can be a server providing various services. The terminal device 102 is equipped with an RPA robot that executes RPA processes. When an anomaly occurs in the RPA process, the server 104 can control the running state of the RPA robot on the terminal device 102, record the abnormal RPA process, and prepare a recording file. Then, a template file to be identified is generated based on the recording file. The server 104 processes the template file to be identified to execute the RPA anomaly handling method provided in this embodiment.

[0030] Optionally, terminal device 102 can be various electronic devices with a display screen, including but not limited to smartphones and computer devices, wherein the computer device can be at least one of desktop computers, portable computers, laptop computers, tablet computers, etc. Terminal device 102 can refer to one of a plurality of terminal devices. In addition, terminal device 102 can also be used to execute an RPA exception handling method provided in the embodiments of this application.

[0031] It should be understood that the number of terminal devices, networks, and servers is merely illustrative. Depending on the implementation requirements, there can be any number of terminal devices, networks, and servers. For example, the servers could be a server cluster composed of multiple servers.

[0032] Based on this, this application provides an RPA exception handling method in its embodiments. Please refer to... Figure 2 , Figure 2 This paper illustrates a flowchart of an RPA exception handling method provided in an embodiment of this application, demonstrating how this method can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0033] Step S110: Obtain the template file to be identified.

[0034] The template file to be identified is determined based on key information and abnormal task information extracted from screen recordings generated when an exception occurs in the RPA process.

[0035] In one embodiment, determining the template file to be identified includes: obtaining a screen recording file and abnormal task information when an RPA process encounters an exception; performing video recognition on the screen recording file to extract key information from the video; and forming the template file to be identified based on the key information and abnormal task information.

[0036] Specifically, when an RPA process encounters an anomaly, the abnormal RPA process is recorded to create a screen recording file. Simultaneously, the task information of the RPA process is acquired and recorded as abnormal task information. Then, AI (artificial intelligence) video recognition technology (i.e., video recognition algorithms) is used to examine and analyze the video content of the screen recording file to extract key information. Based on this key information and the abnormal task information, a template file to be recognized is generated.

[0037] The abnormal task information refers to information related to abnormal processes, including customer name, process name, abnormal time, and detailed fault reporting information. The template file to be identified includes at least one of the following: analysis frame images, point data, time point information, and time information.

[0038] Point data or time information refers to the main time data in each frame of an image, including relevant information such as time frame images; for example, when watching videos on common video websites, clicking on the progress bar at the bottom of the video displays keyframe images near that time point.

[0039] Step S120: Extract multiple frames of images from the template file to be identified, and calculate the pixel color difference value and pixel similarity of the multiple frames of images.

[0040] Step S130: Determine the average similarity of the template file to be identified based on the pixel color difference value and pixel similarity of the multi-frame images.

[0041] Since the template file to be identified contains a series of sequential frame images, to reduce computational load, multiple frames can be extracted from the series of sequential frame images. Then, the pixel color difference value and pixel similarity of each frame in the multiple frames can be calculated. After calculating the pixel color difference value and pixel similarity of each frame, the average similarity of the multiple frames is calculated based on the pixel color difference value and pixel similarity of each frame, and this average similarity is used as the average similarity of the template file to be identified.

[0042] Step S140: Determine the correlation between the template file to be identified and all target video files in the pre-established feature code library based on the average similarity.

[0043] The feature code library is built based on historical anomaly identification template files and includes multiple target video files. Specifically, the feature code library is constructed according to image weight coefficients based on previous anomaly identification template files. The feature codes include image features, text features, audio features, etc. of the target videos.

[0044] Step S150: When the template file to be identified is related to any target video file, the anomaly response handling scheme corresponding to the related target video file is retrieved from the pre-established anomaly handling scheme library. The anomaly response handling scheme is used to handle RPA anomalies.

[0045] Specifically, the correlation between the template file to be identified and each target video file in the feature code library is determined based on the average similarity. When the template file to be identified is related to any target video file, the anomaly response handling scheme corresponding to the relevant target video file is found from the anomaly handling scheme library. This anomaly response handling scheme can be sent to the terminal device for maintenance personnel to handle RPA anomalies.

[0046] The exception handling solution library is pre-built and stores historical exception log descriptions and corresponding exception response handling solutions.

[0047] The RPA anomaly handling method provided in this application embodiment can automatically retrieve anomaly response handling solutions related to the current anomaly from a pre-established anomaly handling solution library when an RPA anomaly is detected, and output them to the corresponding RPA automation process for repair. Compared with the manual anomaly repair method in related technologies, the technical solution provided in this application embodiment solves the problem of time-consuming and labor-intensive manual operation, improves the efficiency of anomaly repair, and enhances the user experience, thereby overcoming the problems of time-consuming, labor-intensive, and inefficient manual anomaly handling.

[0048] In one embodiment, step S120, which involves extracting multiple frames of images from the template file to be identified and calculating the pixel color difference values ​​and pixel similarity of the multiple frames, includes:

[0049] Step S1: Extract multiple frames from the frame images of the template file to be identified at equal intervals according to the frame sequence.

[0050] Step S2: Calculate the pixel color difference value between two corresponding pixels in any two columns of pixels in each frame of the image.

[0051] In one embodiment, calculating the pixel color difference value between two pixels at corresponding positions in any two columns of pixels in each frame image includes: calculating the pixel color difference value using the following formula:

[0052]

[0053] Among them, C n R is the pixel color difference between two corresponding pixels in any two columns of the nth row; An G An B An R represents the red, green, and blue values ​​of one column of pixels in any two columns of the nth row; Bn G Bn B Bn These represent the red, green, and blue values ​​of the RGB values ​​of the other column of pixels in any two columns of pixels in the nth row.

[0054] Specifically, each frame of an image contains multiple pixels, which can be divided into rows and columns. The pixel color difference value of each frame can be calculated from the color difference between any two columns of pixels.

[0055] In a preferred embodiment, the pixel color difference value can be calculated by selecting the first and last columns of pixels in each frame. Specifically, two corresponding pixels can be selected from the first and last columns of any row, and the pixel color difference value can be calculated based on the selected two pixels.

[0056] In addition, when determining the pixel color difference value, it is also possible to determine whether the pixel color difference value is less than the color difference threshold. If Cn < 100, it can be confirmed that the colors of the two selected pixels are similar.

[0057] Step S3: Calculate the pixel similarity between two pixels at corresponding positions in any two columns of pixels in each frame of the image.

[0058] In one embodiment, calculating the pixel similarity between two pixels at corresponding positions in any two columns of pixels in each frame of an image includes: scaling each frame of the image; representing any two columns of pixels in the scaled image using vectors; and calculating the pixel similarity between two pixels at corresponding positions based on any two columns of pixels represented by vectors.

[0059] Specifically, pixel similarity can be calculated by selecting the first and last columns of pixels in each frame. More specifically, two corresponding pixels can be selected from the first and last columns of any row, and pixel similarity can be calculated based on these two selected pixels.

[0060] First, each frame of the multi-frame image is scaled to ensure that the multiple frames have the same size. Then, the first and last columns of pixels in the scaled image are represented by vectors. The first column pixel vector can be represented as:

[0061] A n ={{R A0 G A0 B A0}, {R A1 G A1 B A1},......,{R An-1 G An-1 B An-1}}

[0062] The last column pixel vector can be represented as:

[0063] B n ={{R B0 G B0 B B0}, {R B1 G B1 B B1},......,{R Bn-1 G Bn-1 B Bn-1}}

[0064] Then, pixel similarity is calculated based on the vector representation.

[0065] Step S4: Calculate the weight coefficient and final similarity of each frame image based on the pixel similarity and pixel similarity of each frame image.

[0066] Step S5: Calculate the average similarity of multiple frames of images using a weighted method based on the weight coefficients of each frame and the final similarity.

[0067] Specifically, based on the pixel similarity of each frame of the image, the weight coefficient and pixel similarity of each frame of the image are determined, and the average similarity of multiple frames of the image is calculated by weighting the weight coefficient and the final similarity of each frame of the image.

[0068] In one embodiment, the target video file includes multiple frames of images. In step S140, the correlation between the template file to be identified and all target video files in the pre-established feature code library is determined based on the average similarity. This includes: calculating the pixel similarity of each frame of images in all target video files; comparing the pixel similarity of each frame with the average similarity of the template file to be identified; when the pixel similarity of any frame is greater than or equal to the average similarity, determining that the feature code of the corresponding target video file has been hit, and counting the number of times the feature code of each target video file has been hit; when the number of times the feature code of the target video file has been hit is greater than or equal to a preset number, determining that the template file to be identified is related to the target video file corresponding to the number of feature code hits.

[0069] In one embodiment, the method further includes: when the number of times the feature code of the target video file is hit is less than a preset number, determining that the template file to be identified is not related to the target video file corresponding to the number of times the feature code is hit.

[0070] Specifically, it is determined whether the pixel similarity of each frame is greater than or equal to the average similarity; if so, a feature code match is triggered. If not, no match is triggered. If the number of feature codes matched by the template file to be identified in the feature code library reaches a preset number, it is determined that the template file to be identified is related to the target video file. If the number of feature codes matched by the template file to be identified in the feature code library does not reach the preset number, it is determined that the template file to be identified is not related to the target video file.

[0071] To facilitate understanding, a detailed implementation example is provided. Assume the feature code library includes two target video files, denoted as target video file A and target video file B, and a template file to be identified, C. Target video file A contains M frames, and target video file B contains N frames. The pixel similarity of each frame in both target video file A and target video file B is calculated to determine if it is greater than the average similarity of template file C. If so, the number of times the template file to be identified matches the feature codes of target video file A and target video file B, E and F, are recorded. If E is greater than or equal to a preset number of matches, the template file to be identified is determined to be similar to target video file A; if F is less than the preset number of matches, the template file to be identified is determined to be dissimilar to target video file B.

[0072] It should be understood that, although Figure 2The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0073] The embodiments disclosed in this application describe in detail an RPA exception handling method. The above-disclosed method can be implemented using various types of devices. Therefore, this application also discloses an RPA exception handling device corresponding to the above method. Specific embodiments are given below for detailed description.

[0074] Please see Figure 3 This application discloses an RPA exception handling device, which mainly includes:

[0075] The file acquisition module 310 is used to acquire the template file to be identified, wherein the template file to be identified is determined based on the key information and abnormal task information extracted from the screen recording file generated when an exception occurs in the RPA process;

[0076] The calculation module 320 is used to extract multiple frames of images from the template file to be identified and to calculate the pixel similarity and pixel similarity of the multiple frames of images.

[0077] The average similarity determination module 330 is used to determine the average similarity of the template file to be identified based on the pixel similarity of multiple frames of images.

[0078] The correlation determination module 340 is used to determine the correlation between the template file to be identified and all target video files in the pre-established feature code library based on the average similarity; wherein, the feature code library is established based on historical anomaly identification template files and includes multiple target video files;

[0079] The scheme determination module 350 is used to query the anomaly response processing scheme corresponding to the relevant target video file from the pre-established anomaly processing scheme library when the template file to be identified is related to any target video file; wherein, the anomaly response processing scheme is used to process the anomalies of RPA.

[0080] In one embodiment, the file acquisition module 310 is used to acquire screen recording files and abnormal task information when an RPA process encounters an exception; perform video recognition on the screen recording files to extract key information from the video; and form a template file to be recognized based on the key information and abnormal task information.

[0081] In one embodiment, the calculation module 320 is configured to extract multiple frames of images from the frame images of the template file to be identified at equal intervals according to the frame sequence; calculate the pixel similarity between two corresponding pixels in any two columns of pixels in each frame image; calculate the pixel similarity between two corresponding pixels in any two columns of pixels in each frame image; calculate the weight coefficient and final similarity of each frame image based on the pixel similarity and pixel similarity of each frame image; and calculate the average similarity of the multiple frames of images using a weighted method based on the weight coefficient and final similarity of each frame image.

[0082] In one embodiment, the calculation module 320 is used to calculate pixel similarity using the following formula:

[0083]

[0084] Among them, C n R is the pixel similarity between two corresponding pixels in any two columns of the nth row; An G An B An R represents the red, green, and blue values ​​of one column of pixels in any two columns of the nth row; Bn G Bn B Bn These represent the red, green, and blue values ​​of the RGB values ​​of the other column of pixels in any two columns of pixels in the nth row.

[0085] In one embodiment, the calculation module 320 is used to scale each frame of the image; represent any two columns of pixels in the scaled image using vectors; and calculate the pixel similarity between two pixels at corresponding positions based on any two columns of pixels after vector representation.

[0086] In one embodiment, the target video file includes multiple frames of images; the correlation determination module 340 is used to calculate the pixel similarity of each frame of image in all target video files; compare the pixel similarity of each frame of image with the average similarity of the template file to be identified; when the pixel similarity of any frame of image is greater than or equal to the average similarity, it is determined that the feature code of the corresponding target video file is hit, and the number of times the feature code of each target video file is hit is counted; when the number of times the feature code of the target video file is hit is greater than or equal to a preset number, it is determined that the template file to be identified is related to the target video file corresponding to the number of feature codes.

[0087] In one embodiment, the correlation determination module 340 is used to determine that the template file to be identified is not related to the target video file corresponding to the number of feature codes when the number of times the feature code of the target video file is hit is less than a preset number.

[0088] Specific limitations regarding the RPA exception handling device can be found in the method limitations section above, and will not be repeated here. Each module in the above device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the server's processor in hardware form or independent of it, or stored in the server's memory in software form, so that the processor can call and execute the operations corresponding to each module.

[0089] Please refer to Figure 4 , Figure 4 This illustration shows a structural block diagram of a server provided in an embodiment of this application. The server 40 may be a computer device. The server 40 in this application may include one or more of the following components: a processor 42, a memory 44, and one or more application programs, wherein the one or more application programs may be stored in the memory 44 and configured to be executed by the one or more processors 42, and the one or more application programs are configured to perform the methods described above in the embodiments applied to RPA exception handling methods.

[0090] Processor 42 may include one or more processing cores. Processor 42 connects to various parts of the server 40 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 44, and by calling data stored in memory 44. Optionally, processor 42 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 42 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 42 and may be implemented separately using a communication chip.

[0091] The memory 44 may include random access memory (RAM) or read-only memory (ROM). The memory 44 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 44 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the server 40 during use.

[0092] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the server to which the solution of this application is applied. A specific server may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0093] In summary, the server provided in this application embodiment is used to implement the corresponding RPA exception handling method in the foregoing method embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0094] Please see Figure 5 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 50 stores program code, which can be invoked by a processor to execute the methods described in the above-described RPA exception handling method embodiments.

[0095] The computer-readable storage medium 50 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 50 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 50 has storage space for program code 52 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 52 may be compressed, for example, in a suitable form.

[0096] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for handling RPA exceptions, the method comprising: The method includes: Obtain the template file to be identified, wherein the template file to be identified is determined based on key information and abnormal task information extracted from the screen recording file generated when an exception occurs in the RPA process; Extract multiple frames of images from the template file to be identified, and calculate the pixel color difference value and pixel similarity of the multiple frames of images; The average similarity of the template file to be identified is determined based on the pixel color difference value and pixel similarity of multiple frames of images; The correlation between the template file to be identified and all target video files in the pre-established feature code library is determined based on the average similarity; wherein, the feature code library is established based on historical anomaly identification template files and includes multiple target video files; When the template file to be identified is related to any of the target video files, an anomaly response processing scheme corresponding to the related target video file is retrieved from a pre-established anomaly handling scheme library; wherein, the anomaly response processing scheme is used to handle RPA anomalies; The step of extracting multiple frames of images from the template file to be identified and calculating the pixel color difference and pixel similarity of the multiple frames includes: Multiple frames are extracted from the frame images of the template file to be identified at equal intervals according to the frame sequence; Calculate the pixel color difference value between two corresponding pixels in any two columns of pixels in each frame of the image; Calculate the pixel similarity between two corresponding pixels in any two columns of pixels in each frame of the image; The weight coefficient and final similarity of each frame image are calculated based on the pixel color difference value and pixel similarity of each frame image. The average similarity of multiple frames is calculated using a weighted method based on the weight coefficients of each frame and the final similarity.

2. The method according to claim 1, characterized in that, Determining the template file to be identified includes: Obtain screen recordings and information about the abnormal task when an RPA process encounters an exception; The screen recording file is subjected to video recognition to extract key information from the video; A template file to be identified is generated based on the key information and abnormal task information.

3. The method according to claim 1, characterized in that, The calculation of the pixel color difference value between two corresponding pixels in any two columns of pixels in each frame of the image includes: The pixel color difference value is calculated using the following formula: wherein C n is the pixel color difference value of the corresponding two pixel points in any two columns of pixels in the nth row; R An , G An , and B An respectively represent the red, green, and blue values of the RGB value of one column of pixel points in any two columns of pixels in the nth row; R Bn , G Bn , and B Bn respectively represent the red, green, and blue values of the RGB value of the other column of pixel points in any two columns of pixels in the nth row.

4. The method according to claim 1, characterized in that, The calculation of pixel similarity between two corresponding pixels in any two columns of pixels in each frame of the image includes: Each frame of the image is scaled. Represent any two columns of pixels in the scaled image using vectors; The pixel similarity between two pixels at corresponding positions is calculated based on any two columns of pixels in the vector representation.

5. The method according to any one of claims 1-4, characterized in that, The target video file includes multiple frames of images; determining the correlation between the template file to be identified and all target video files in the pre-established feature code library based on the average similarity includes: Calculate the pixel similarity of each frame in all target video files; The pixel similarity of each frame image is compared with the average similarity of the template file to be identified. When the pixel similarity of any frame image is greater than or equal to the average similarity, it is determined that the feature code of the corresponding target video file is hit, and the number of times the feature code of each target video file is hit is counted. When the number of times the feature code of the target video file is hit is greater than or equal to the preset number, it is determined that the template file to be identified is related to the target video file corresponding to the number of times the feature code is hit.

6. The method according to claim 5, characterized in that, The method further includes: When the number of times the feature code of the target video file is hit is less than the preset number, it is determined that the template file to be identified is not related to the target video file corresponding to the number of times the feature code is hit.

7. An RPA anomaly handling device, characterized in that, The device includes: The file acquisition module is used to acquire the template file to be identified, wherein the template file to be identified is determined based on the key information and abnormal task information extracted from the screen recording file generated when an exception occurs in the RPA process; The calculation module is used to extract multiple frames of images from the template file to be identified, and calculate the pixel color difference value and pixel similarity of the multiple frames of images. This includes: extracting multiple frames of images from the template file to be identified at equal intervals according to the frame sequence; calculating the pixel color difference value between two corresponding pixels in any two columns of pixels in each frame of the image; calculating the pixel similarity between two corresponding pixels in any two columns of pixels in each frame of the image; calculating the weight coefficient and final similarity of each frame of the image based on the pixel color difference value and the pixel similarity; and calculating the average similarity of the multiple frames of images using a weighted method based on the weight coefficient and the final similarity of each frame of the image. The average similarity determination module is used to determine the average similarity of the template file to be identified based on the pixel color difference values ​​and pixel similarity of multiple frames of images. The correlation determination module is used to determine the correlation between the template file to be identified and all target video files in the pre-established feature code library based on the average similarity; wherein, the feature code library is established based on historical anomaly identification template files and includes multiple target video files; The scheme determination module is used to query the anomaly response processing scheme corresponding to the relevant target video file from a pre-established anomaly processing scheme library when the template file to be identified is related to any of the target video files; wherein, the anomaly response processing scheme is used to process RPA anomalies.

8. A server, characterized in that, include: Memory; One or more processors are coupled to the memory; One or more applications, wherein the one or more applications are stored in memory and configured to be executed by one or more processors, and the one or more applications are configured to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-6.

Images