Image matching method and related equipment

Abstract

The embodiment of the present application discloses an image matching method, which is applied to the process of robot process automation, including: obtaining a template image; setting a preset resolution based on the template image; obtaining an image to be matched; judging whether the resolution of the image to be matched meets the preset resolution; if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image; matching the preprocessed image with the template image. It can be seen from the above technical scheme that the scheme verifies the resolution of the image to be matched before executing the image matching process to judge whether the image to be matched meets the preset resolution. If not, the image to be matched is subjected to resolution processing so that the resolution of the image to be matched meets the requirements, and a preprocessed image is obtained. The preprocessed image is used for matching processing to complete the image matching process, thereby avoiding the recognition accuracy problem caused by the large resolution difference during the image matching process.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to an image matching method and related equipment. Background Art

[0002] The Robotic Process Automation (RPA) system is an application that provides another way to automate user manual operation processes by imitating the user's manual operation methods on the computer.

[0003] In traditional workflow automation technology tools, programmers will generate action lists for automated tasks and use internal application programming interfaces or dedicated scripting languages ​​as interfaces with backend systems. Robotic process automation monitors the work performed by users in the graphical user interface (GUI) of the application software and automatically repeats these tasks directly on the GUI.

[0004] When designing existing RPA robots, template images are preset for subsequent detection and matching. When the RPA robot is running, it obtains the image displayed by the current device as the image to be matched, matches the image to be matched with the template image to find whether the template image contains the detection image, and returns the detection result. However, the resolution of the image to be matched obtained during the detection process may be different from that of the template image. This difference may cause unnecessary errors in the matching detection process, causing certain inconveniences. Summary of the invention

[0005] A first aspect of an embodiment of the present application provides an image matching method, which is applied to robotic process automation, comprising:

[0006] Execute the RPA business process to obtain the image to be matched;

[0007] Acquire a template image, and acquire a preset resolution of the template image;

[0008] Determining whether the resolution of the image to be matched meets the preset resolution;

[0009] If the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image;

[0010] The pre-processed image is matched with the template image.

[0011] Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image includes:

[0012] If the resolution of the image to be matched is less than the preset resolution, bicubic interpolation processing is performed on the image to be matched to obtain a preprocessed image.

[0013] Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, if the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image includes:

[0014] If the resolution of the image to be matched is greater than the preset resolution, a bitwise averaging process is performed on the image to be matched to obtain a preprocessed image.

[0015] Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, before performing averaging processing on the image to be matched to obtain the preprocessed image, the method further includes:

[0016] Perform Gaussian filtering on the image to be matched.

[0017] Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, the method is applied in the working process of a robotic process automation system.

[0018] Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, matching the pre-processed image with the template image includes:

[0019] Determine whether the preprocessed image includes the template image, and return the determination result.

[0020] Based on the image matching method provided in the first aspect of the embodiment of the present application, optionally, the image to be matched is obtained by taking a screenshot of the current user interface.

[0021] A second aspect of an embodiment of the present application provides an image matching device, which is applied to robot process automation, including:

[0022] A first acquisition unit is used to execute the RPA business process and acquire the image to be matched;

[0023] A second acquisition unit, used to acquire the image to be matched and to acquire a preset resolution of the template image;

[0024] A judging unit, used for judging whether the resolution of the image to be matched meets the preset resolution, and triggering the resolution processing unit if the resolution of the image to be matched does not meet the preset resolution;

[0025] A resolution processing unit, used for performing resolution processing on the image to be matched to obtain a pre-processed image;

[0026] A matching unit is used to match the pre-processed image with the template image.

[0027] A third aspect of an embodiment of the present application provides a computer, including:

[0028] CPU, memory;

[0029] The memory is a short-term storage memory or a persistent storage memory;

[0030] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory on the computer to perform a method as described in any one of the first aspects of the embodiments of the present application.

[0031] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium includes instructions. When the instructions are executed on a computer, the computer executes a method as described in any one of the methods described in the first aspect of the embodiment of the present application.

[0032] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: In the RPA process, the resolution of the image to be matched is identified and verified before the image matching process is executed to determine whether the image to be matched meets the preset resolution. The preset resolution is set by the template image. If it does not meet the requirements, the resolution of the image to be matched is processed to make the resolution of the image to be matched meet the requirements, and a pre-processed image is obtained. The pre-processed image is used for matching processing to complete the image matching process, thereby avoiding the recognition accuracy problem caused by the large resolution difference in the image matching process, and improving the accuracy and availability of RPA execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0034] Figure 1 A schematic diagram of a flow chart of an embodiment of the image matching method of the present application;

[0035] Figure 2Another flowchart of an embodiment of the image matching method of the present application is shown;

[0036] Figure 3 A structural schematic diagram of an embodiment of an image matching device of the present application;

[0037] Figure 4 This is another structural schematic diagram of an image matching device embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiment of the present application provides an image matching method for completing the matching process between an image to be matched and a template image of different resolutions. The solution determines whether the image to be matched meets the preset resolution by performing resolution recognition and verification on the image to be matched. The preset resolution is set by the template image. If it does not meet the requirements, the resolution of the image to be matched is processed so that the resolution of the image to be matched meets the requirements, and a pre-processed image is obtained. The pre-processed image is used to perform the image matching process with the template image. The above method avoids the recognition accuracy problem caused by the large resolution difference during the image matching process.

[0039] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0040] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0042] See also Figure 1 , an embodiment of the image matching method of the present application includes: step 101-step 105. The image matching method of the present application is applied to the working process of Robotic Process Automation (RPA).

[0043] 101. Execute the RPA business process to obtain the image to be matched.

[0044] The RPA robot executes the RPA business process and obtains the image to be matched. The image to be matched is the image that needs to be matched with the template image. Specifically, the image to be analyzed can be obtained by taking a screenshot of the current user interface. The RPA robot can take a screenshot of the current user interface when executing the corresponding business process to obtain the corresponding image to be matched. It is understandable that the image to be matched can also be an image obtained by the RPA robot executing other business processes. The specific details can be determined according to the actual situation and are not limited here. Since the image to be analyzed is obtained based on the user's current usage environment, the obtained image to be matched may differ from the template image in resolution, and the resolution of the image to be matched needs to be processed.

[0045] 102. Obtain a template image, and obtain a preset resolution of the template image.

[0046] Get a template image, and get the preset resolution of the template image. The template image is used for the subsequent matching process. During the implementation of the solution, the template image can be manually selected and preset by the user, such as selecting images such as buttons and input fields. It can also be an image such as a screenshot of the current user node, which can be determined according to the actual situation and is not limited here. The server sets a preset resolution based on the template image. The server can set a preset resolution based on the template image to ensure the efficiency of the image matching process. Specifically, the server can identify the resolution of the template image, obtain the resolution data of the template image and set the value of the preset resolution to be the same as the resolution of the template image. The resolution of the template image can also be adjusted to a certain extent, such as increasing or decreasing a certain value, to obtain the data range corresponding to the preset resolution. When the resolution of the picture to be matched is within the range of the preset resolution, it means that the effect is good and the image matching process can be performed.

[0047] 103. Determine whether the resolution of the image to be matched meets the preset resolution.

[0048] The server determines whether the resolution of the image to be matched meets the preset resolution. The server extracts and identifies the resolution of the image to be matched, obtains the resolution data of the image to be matched, compares the resolution of the image to be matched with the preset resolution, and determines whether it is the same as the preset resolution or meets the preset resolution range requirements. If the resolution of the image to be matched does not meet the preset resolution, such as the resolution of the image to be distinguished is greater than or less than the value or range of the preset resolution, then step 104 is executed to perform resolution processing on the image to be matched to obtain a preprocessed image. If the resolution of the image to be matched is the same as the preset resolution or meets the range requirements of the preset resolution, it means that there is no difference in resolution between the image to be matched and the template image or the difference is small, and the accuracy of using the template image to match the image to be matched is high, then the image matching process between the image to be matched and the template image can be executed or other steps can be executed, which are not limited here.

[0049] 104. Perform resolution processing on the image to be matched to obtain a preprocessed image.

[0050] The server performs resolution processing on the image to be matched to obtain a preprocessed image. The preprocessed image is obtained by resolution processing of the image to be matched. The resolution processing method of the image to be matched can be determined according to the actual situation. Specifically, the comparison result between the resolution of the image to be matched and the preset resolution by the server can be a preprocessing vector used to characterize the resolution of the image to be matched. The corresponding processing process is performed based on the direction of the vector. If the resolution of the image to be matched is less than the preset resolution, the preprocessing vector points to the processing method when the resolution is smaller. Specifically, for the case where the resolution of the image to be matched is less than the preset resolution, the processing method can be: interpolation processing is performed on the image, that is, the grayscale value or three-color value of the known adjacent pixel points in the image to be matched is used to insert the corresponding pixel points in the image. The grayscale value or three-color value of the inserted pixel point is obtained by the corresponding attribute of the known pixel point, thereby improving the resolution of the image to be matched. Although the processed image obtained in this way does not increase the image information, it improves the resolution of the image, ensuring the smooth progress of the image matching process. If the resolution of the image to be matched is greater than the preset resolution, the preprocessing vector points to the processing method when the resolution is larger. Specifically, for the case where the resolution of the image to be matched is greater than the preset resolution, the processing method can be: averaging, that is, averaging the grayscale values ​​or three-color values ​​of multiple pixels in adjacent areas and merging the multiple pixels into a similar point. The grayscale value or three-color value of the pixel is the average value obtained after averaging the grayscale values ​​or three-color values ​​of multiple pixels. It can be understood that in the actual implementation of this solution, different methods can be used to process the images to be matched belonging to different situations in order to obtain preprocessed images that meet the requirements. The specific method can be determined according to the actual situation and is not limited here.

[0051] 105. Perform matching processing on the preprocessed image and the template image.

[0052] The server performs matching processing on the pre-processed image and the template image. Specifically, the template image may be set as an image of certain buttons, connections or input boxes in a GUI (Graphical User Interface). Therefore, the processing result of matching the pre-processed image and the template image is generally output as the pre-processed image including the template image or the pre-processed image excluding the template image. Based on the processing result of the matching processing, the server can execute the next processing flow, such as clicking a button or connecting to input corresponding characters, etc., which is not limited here.

[0053] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: This solution verifies the resolution of the image to be matched before executing the image matching process to determine whether the image to be matched meets the preset resolution. The preset resolution is set by the template image. If it does not meet the requirements, the resolution of the image to be matched is processed so that the resolution of the image to be matched meets the requirements, and a pre-processed image is obtained. The pre-processed image is used for matching processing to complete the image matching process, thereby avoiding the recognition accuracy problem caused by the large resolution difference during the image matching process, and improving the accuracy and availability of the RPA robot during the business execution process.

[0054] based on Figure 1 The following is a detailed embodiment of the present invention that can be optionally performed during the implementation process. Figure 2 , an embodiment of the present application includes: steps 201 to 208.

[0055] 201. Execute the RPA business process to obtain the image to be matched.

[0056] The RPA robot executes the RPA business process and obtains the image to be matched. The Robotic Process Automation (RPA) system is an application that provides another way to automate the manual operation process of the end user by imitating the manual operation of the end user on the computer. The image to be matched is the image that needs to be matched with the template image. Specifically, the image to be analyzed can be obtained by taking a screenshot of the current user interface. The RPA robot can take a screenshot of the current user interface when executing the corresponding business process to obtain the corresponding image to be matched. It can be understood that the image to be matched can also be an image obtained by the RPA robot executing other business processes. The specific situation can be determined according to the actual situation and is not limited here. Since the image to be analyzed is obtained based on the user's current usage environment, the obtained image to be matched may have a difference in resolution from the template image, and the resolution of the image to be matched needs to be processed.

[0057] 202. Obtain a template image, and obtain a preset resolution of the template image.

[0058] Get a template image and get a preset resolution of the template image. The template image is used for the subsequent matching process. During the implementation of the solution, the template image can be manually selected and preset by the user, such as selecting images such as buttons and input fields. It can also be an image such as a screenshot of the current user node, which can be determined according to the actual situation and is not limited here. When the RPA robot is in use, if an image that is the same or similar to the template image is detected, click and other operations are triggered to complete the function of the RPA robot, reducing the user's manual consumption. The template image can be saved in the server together with the designed RPA robot so that it can work directly on other devices without setting the template image again. The server sets a preset resolution based on the template image. To ensure the efficiency of the image matching process, the server can set a preset resolution based on the template image. Specifically, the server can perform resolution identification on the template image, obtain the resolution data of the template image and set the preset resolution value to be the same as the resolution of the template image. The resolution of the template image can also be adjusted to a certain extent, such as increasing or decreasing the value, to obtain the data range corresponding to the preset resolution. When the resolution of the picture to be matched is within the range of the preset resolution, it means that the effect is good and the image matching process can be performed.

[0059] 203. Perform grayscale processing on the image to be matched.

[0060] The image to be matched is gray-scaled. Generally, in order to ensure the accuracy of the recognition process, the image to be matched needs to be gray-scaled to avoid recognition failure due to color difference and other problems. The specific gray-scale processing process can be processed by calling the corresponding function, which will not be described here. It can be understood that in the case where the template image is a gray-scale image, only the image to be matched can be adjusted to facilitate the subsequent matching process. In the case where the template image is a color image, the template image and the image to be matched can be gray-scaled at the same time to avoid the adverse effects of RGB color factors on the matching process.

[0061] 204. Determine whether the resolution of the image to be matched meets the preset resolution.

[0062] The RPA robot determines whether the resolution of the image to be matched meets the preset resolution. The RPA robot extracts the resolution of the image to be matched. Specifically, after obtaining the screenshot image of the current interface, the resolution as one aspect of the image attribute data has been stored in the screenshot image file. The server can read the image file to obtain the resolution data of the image to be matched, and then compare the resolution data of the image to be matched with the preset resolution to obtain a preprocessing vector. If the preprocessing vector is equal to 1, it means that the resolution of the image to be matched is the same as that of the template image, and the matching process of the image to be matched and the template image can be directly performed without resolution processing of the image to be matched. If the preprocessing vector is less than 1, it means that the resolution of the image to be matched is lower than that of the template image, and step 205 should be executed to interpolate the image to be matched to obtain a preprocessed image. If the preprocessing vector is greater than 1, it means that the resolution of the image to be matched is higher than that of the template image, and step 206 should be executed to perform Gaussian filtering on the image to be matched.

[0063] It can be understood that in the process of determining the resolution of the image to be matched, other processing methods other than step 205 or step 206 can be set for different resolutions. For example, if the resolution of the image to be matched is too small, that is, less than a predetermined value, the image to be matched is deemed invalid, and a matching image that meets the requirements is re-acquired. The specific method can be determined according to the actual situation and is not limited here.

[0064] 205. Perform bicubic interpolation processing on the image to be matched to obtain a preprocessed image.

[0065] The image to be matched is subjected to bicubic interpolation processing to obtain a preprocessed image. If the resolution of the image to be matched is lower than the resolution of the template image, the image to be matched is subjected to bicubic interpolation processing to improve the resolution of the image to be matched, thereby improving the matching accuracy between the subsequent preprocessed image and the template image. The interpolation processing is to insert corresponding pixels in the image using the grayscale values ​​or tricolor values ​​of the known adjacent pixels in the image to be matched. The grayscale values ​​or tricolor values ​​of the inserted pixels are obtained by the corresponding attributes of the known pixels, thereby improving the resolution of the image to be matched. In the actual implementation of this scheme, the GetBicubicVal (bicubic interpolation) function of OPENCV (a cross-platform computer vision and machine learning software library licensed under the Berkeley Software Suite) can be used for interpolation processing. It can be understood that during the interpolation processing, the software used can be adjusted according to the actual situation, and the number of interpolation times can also be adjusted according to its own needs and the attributes of the image to be matched. If the resolution of the image to be matched is too low, the number of interpolation times is increased to obtain a preprocessed image that meets the requirements. The specific method can be determined according to the actual situation and is not limited here. After the RPA robot performs interpolation processing on the image to be matched, it obtains the required pre-processed image, executes step 208 based on the obtained pre-processed image, determines whether the pre-processed image includes the template image, and returns the determination result.

[0066] 206. Perform Gaussian filtering on the image to be matched.

[0067] If the resolution of the image to be matched is higher than that of the template image, you need to perform Gaussian filtering on the image to be matched first. Specifically, the RPA robot can call the Gaussianblur function (Gaussian filter) of OPENCV to smooth and reduce noise on the image to be matched. Gaussian filtering is a filtering method used for noise reduction. Gaussian filtering can effectively reduce Gaussian noise in the image, thereby preventing the Gaussian noise in the high-resolution image to be matched from affecting the resolution reduction process of the matching image, thereby improving the accuracy of image matching between the preprocessed image and the template image.

[0068] It is understandable that in the actual implementation process of this solution, Gaussian filtering may not be performed, that is, step 207 may be directly executed to perform averaging processing on the image to reduce the computing resources consumed by the operation of the RPA robot and improve the processing speed. The specific method may be determined according to the actual situation and is not limited here.

[0069] 207. Perform bitwise averaging processing on the image to be matched to obtain a preprocessed image.

[0070] The image to be matched is subjected to bitwise averaging to obtain a preprocessed image. After the image to be matched is subjected to Gaussian filtering, the image to be matched is subjected to bitwise averaging, and the underlying data of the image is processed by bitwise operation to obtain the result of the averaging process and recompose the image with reduced resolution. Specifically, the grayscale values ​​or three-color values ​​of multiple pixels in adjacent areas can be averaged and the multiple pixels can be calculated to be fused into a similar point. The grayscale value or three-color value of the pixel is the average value obtained after the grayscale value or three-color value of multiple pixels is averaged. In this way, the number of pixels included in the image to be matched is reduced, and the resolution of the image to be matched is reduced to obtain a preprocessed image that meets the preset resolution requirement. It is understandable that the resolution of the preprocessed image obtained after a single averaging process may still not meet the requirements. The resolution of the processed image can be re-identified and repeatedly processed to obtain a preprocessed image that meets the requirements. The specific details can be determined according to the actual situation and are not limited here. After the RPA robot performs averaging processing on the image to be matched, it obtains the required pre-processed image, executes step 208 based on the obtained pre-processed image, determines whether the pre-processed image includes the template image, and returns the determination result.

[0071] 208. Determine whether the pre-processed image includes the template image, and return the determination result.

[0072] The RPA robot determines whether the pre-processed image includes the template image, and returns the judgment result. That is, the RPA robot performs image matching on the pre-processed image and the template image. Specifically, the RPA robot can use the OPENCV image framework and adopt the standard correlation matching algorithm (parameter setting: CV_TM_CCOEFF_NORMED) to match the template image and the detection image, find out whether the template image contains the detection image, and return the detection result. It is understandable that the software and algorithm used in the image matching process can be adjusted according to the actual situation, and the specific details are not limited here. After obtaining the judgment result, the RPA robot can click or enter other information based on the recognition result to complete the use process of the RPA robot, which can be determined according to the actual situation and is not limited here.

[0073] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: This solution verifies the resolution of the image to be matched before executing the image matching process to determine whether the image to be matched meets the preset resolution. The preset resolution is set by the template image. If the resolution of the image to be matched is less than the preset resolution, the image to be matched is interpolated to improve the resolution of the image to be matched. If the resolution of the image to be matched is greater than the preset resolution, the image to be matched is averaged to reduce the resolution of the image to be matched. By processing the images to be processed in different situations in different ways, a pre-processed image that is consistent with or less different from the preset resolution is obtained, and then the pre-processed image is used to match the template image to determine whether the template image exists in the pre-processed image, and the corresponding judgment result is returned. The above solution avoids the recognition accuracy problem caused by the large resolution difference during the image matching process. The error problem caused by the difference in resolution of the input image during the operation and design of the RPA robot is avoided, and the applicability of the RPA robot is improved.

[0074] The above describes the image matching method in the embodiment of the application, and the following describes the image matching device in the embodiment of the present invention. Figure 3 , an embodiment of the image matching device of the present application includes:

[0075] The first acquisition unit 301 is used to execute the RPA business process and acquire the image to be matched;

[0076] The second acquisition unit 302 is used to acquire the image to be matched and acquire the preset resolution of the template image;

[0077] A judging unit 303 is used to judge whether the resolution of the image to be matched meets the preset resolution, and trigger a resolution processing unit 304 if the resolution of the image to be matched does not meet the preset resolution;

[0078] A resolution processing unit 304 is used to perform resolution processing on the image to be matched to obtain a pre-processed image;

[0079] The matching unit 305 is used to perform matching processing on the pre-processed image and the template image.

[0080] In this embodiment, the process performed by each unit in the image matching device is the same as that described above. Figure 1 The method flow described in the corresponding embodiment is similar and will not be repeated here.

[0081] Figure 44 is a schematic diagram of the structure of a computer provided in an embodiment of the present application. The server 400 may include one or more central processing units (CPU) 401 and a memory 405. The memory 405 stores one or more application programs or data.

[0082] In this embodiment, the specific functional module division in the central processing unit 401 can be the same as the above Figure 4 The functional module division method of each unit described in is similar and will not be repeated here.

[0083] The memory 405 may be a volatile storage or a persistent storage. The program stored in the memory 405 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processor 401 may be configured to communicate with the memory 405 and execute a series of instruction operations in the memory 405 on the server 400.

[0084] The server 400 may also include one or more power supplies 402, one or more wired or wireless network interfaces 403, one or more input and output interfaces 404, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0085] The CPU 401 can execute the aforementioned Figure 1 The operations performed by the image matching method in the illustrated embodiment will not be described in detail here.

[0086] The present invention also provides a computer-readable storage medium, which is used to implement the functions of the land area measurement system, and stores a computer program. When the computer program is executed by a processor, the processor can be used to execute the following steps: Figure 1 The image matching method method.

[0087] It is understandable that if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a corresponding computer-readable storage medium or integrated into a computer program product to perform the above method. Based on such an understanding, the present invention implements all or part of the processes in the above corresponding embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0088] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0090] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image matching method, characterized in that: Applications in robotic process automation, including: Execute the RPA business process to obtain the image to be matched; Acquire a template image, and acquire a preset resolution of the template image; After acquiring the image to be matched, grayscale processing is performed on the image to be matched; Determining whether the resolution of the image to be matched meets the preset resolution; If the resolution of the image to be matched does not meet the preset resolution, performing resolution processing on the image to be matched to obtain a preprocessed image; Before obtaining the preprocessed image, if the resolution of the image to be matched is greater than the preset resolution, performing Gaussian filtering on the image to be matched; Performing matching processing on the preprocessed image and the template image; If the resolution of the image to be matched does not meet the preset resolution, resolution processing is performed on the image to be matched to obtain a preprocessed image, including: if the resolution of the image to be matched is less than the preset resolution, bicubic interpolation processing is performed on the image to be matched to obtain a preprocessed image, wherein if the resolution of the image to be matched is less than a preset threshold, the number of interpolation times is increased; if the resolution of the image to be matched is greater than the preset resolution, bitwise averaging processing is performed on the image to be matched to obtain a preprocessed image; The matching process between the pre-processed image and the template image includes: determining whether the pre-processed image includes the template image, and returning the determination result.

2. The image matching method according to claim 1, characterized in that: The image to be matched is obtained by taking a screenshot of the current user interface.

3. An image matching device, applied to robotic process automation, characterized in that: include: A first acquisition unit is used to execute the RPA business process and acquire the image to be matched; A second acquisition unit, used to acquire a template image and acquire a preset resolution of the template image; A resolution processing unit, after acquiring the image to be matched, is used to perform grayscale processing on the image to be matched; A judgment unit, used for judging whether the resolution of the image to be matched meets the preset resolution, and triggering the resolution processing unit if the resolution of the image to be matched does not meet the preset resolution; A resolution processing unit, used for performing resolution processing on the image to be matched to obtain a pre-processed image; Before obtaining the pre-processed image, the resolution processing unit is further used to, if the resolution of the image to be matched is greater than the preset resolution, perform Gaussian filtering on the image to be matched; A matching unit, used for matching the pre-processed image with the template image; The resolution processing unit is further configured to, if the resolution of the image to be matched is less than the preset resolution, perform bicubic interpolation processing on the image to be matched to obtain a preprocessed image, wherein if the resolution of the image to be matched is less than a preset threshold, increase the number of interpolation times; The resolution processing unit is further configured to, if the resolution of the image to be matched is greater than the preset resolution, perform bitwise averaging processing on the image to be matched to obtain a preprocessed image; The matching unit is specifically used to determine whether the pre-processed image includes the template image and return a determination result.

4. A computer, characterized in that: include: CPU, memory; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory, and to execute instructions in the memory on the computer to perform the method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 2.

6. A computer program product, characterized in that When the computer program product is executed on a computer, the computer executes the method according to any one of claims 1 to 2.

Images