Method and system for simulating manual operation based on Java computer language

Abstract

The invention provides a method and a system for simulating manual operation based on Java computer language, the method is operated in a JVM environment, in the operation environment, a Jar packet with built-in screen information, mouse information and keyboard information is firstly obtained, then the Jar packet is set as project dependence, a project document is obtained, then the Jar packet is set as project dependence, and then the Java computer language is used for simulating manual operation according to the project document. Screenshot operation, mouse operation and keyboard operation are automatically executed in parallel. For screenshot operation, according to the method, a screen object is called according to screen information, and screenshot collection, matching and storage operations are automatically executed. For mouse operation, according to the method, a mouse object is called according to mouse information, and mouse moving, clicking and releasing operations are automatically executed. For keyboard operation, the method calls a keyboard object according to keyboard information, and automatically executes key tapping and releasing operation. Compared with the prior art, the operation efficiency of the method is higher.

Description

Technical Field

[0001] The present invention belongs to the technical field of information processing, and particularly relates to a method and system for simulating manual operations based on the Java computer language. Background Art

[0002] RPA, the full English name is Robotic Process Automation, is a technical solution for realizing robotic process automation, aiming to simulate or execute high-repetitive and clearly defined business process tasks originally performed by humans through software.

[0003] In the prior art, RPA is mainly implemented relying on Python and Visual Basic languages. Due to the technical underlying limitations of Python and Visual Basic languages, the multi-threaded execution ability of these two programming languages is poor and the memory occupation is high, resulting in poor execution efficiency of these two programming languages in RPA. The third-party libraries of Python and Visual Basic languages are not mature enough and have a small scale, and the callable functional functions are limited. To add new functions, designers need to reprogram, so these two languages are not conducive to the expansion of functional modules. In addition, the cross-platform execution ability of Python and Visual Basic languages is poor, and their development results are difficult to achieve cross-platform interaction. Summary of the Invention

[0004] The present invention provides a method and system for simulating manual operations based on the Java computer language, mainly used to solve the problem of low running efficiency in the prior art.

[0005] In a first aspect, to achieve the above object, the present invention provides a method for simulating manual operations based on the Java computer language, which runs in a JVM (Java Virtual Machine) environment. The method includes:

[0006] S10: Obtain a Jar package;

[0007] Wherein, the configuration information in the Jar package at least includes screen information, mouse information, and keyboard information;

[0008] S20: Set the Jar package as a project dependency according to the configuration information to obtain a project document;

[0009] S30: According to the project document, execute the following steps in parallel:

[0010] S40: According to the screen information, call a screen object to automatically execute screenshot collection, matching, and storage operations;

[0011] S50: Call the mouse object according to the mouse information, and automatically perform mouse movement, click, and release operations;

[0012] S60: Call the keyboard object according to the keyboard information, and automatically perform key pressing and releasing operations.

[0013] Through the above technical solutions, the method of the present invention implements RPA (Robotic Process Automation) using the Java language. Different from Python and Visual Basic languages, Java is a programming language dedicated to object orientation, with a large-scale third-party library, which is easy to expand functional modules. Relying on the cross-platform characteristics of the JVM technology, the method of the present invention can run on various operating systems such as Windows, Linux, and Mac, and has a wider application range. Compared with the prior art, the method of the present invention realizes parallel image search, acquisition of mouse operation information, and acquisition of keyboard operation information, and utilizes the characteristic that the underlying layer of Java technology occupies less memory, so that the operation efficiency of the method of the present invention is higher.

[0014] Preferably, the operations of automatically performing screenshot acquisition, matching, and storage in S40 are specifically as follows:

[0015] S41: Obtain the template image based on the pre-set template image path;

[0016] S42: Take a screenshot of the content of the screen object to obtain a screenshot;

[0017] S43: Match the screenshot with the pre-processed template image, and determine whether there is a region in the screenshot with a matching degree greater than the expected value; if so, sequentially execute S431 - S432;

[0018] S431: Read the corner coordinates of the region in the screenshot with a matching degree greater than the expected value, and perform offset calculation on the corner coordinates according to the preset user-expected coordinates to obtain an offset result;

[0019] S432: Store the screenshot determined by S43 and the offset result, and return the corner coordinates.

[0020] Different from the existing technologies of automatic screenshot acquisition, matching, and storage based on Python RPA and Visual Basic RPA, the method of the present invention is based on Java kernel for automatic screenshot acquisition, matching, and storage. Utilizing the characteristic that Java technology occupies less memory at the underlying level, the method adopts the random matching method in S43 to screen out screenshots that meet the expected values, enabling the method of the present invention to take less time in terms of graphic image feature recognition and further accelerating the processing rate of RPA type tasks. Moreover, each stored screenshot in S432 will be accompanied by an offset result to provide the return corner coordinates for further correction of the screenshot.

[0021] Preferably, after the S41, the method further includes:

[0022] S411: Determine whether the template image has been loaded in the image cache; if so, load the template image from the image cache and jump to the S42; if not, sequentially execute S412 - S416;

[0023] S412: Read the template image in the target image format to obtain a full-color image;

[0024] S413: Perform grayscale calculation on the full-color image to generate a grayscale image;

[0025] S414: Perform upsampling calculation on the grayscale image to obtain an upsampling result;

[0026] S415: Perform downsampling calculation on the grayscale image to obtain a downsampling result;

[0027] S416: Package the upsampling result and the downsampling result into an image object, and after placing the image object into the image cache, jump to the S42.

[0028] Through the above technical solutions, S411 can reasonably call cache resources to avoid repeated loading of the already loaded template image, which effectively reduces the waste of cache resources and improves the overall performance of the computer. If S411 determines that the newly obtained template image has not been loaded, the image is grayscale processed through S413 to enhance image features, which is beneficial to improving the matching accuracy in the S43 matching process. The upsampling process in S414 is used to further enhance the image features of the grayscale image and also for improving the matching accuracy in the S43 matching process. As for the downsampling process in S415, it is for data dimensionality reduction to reduce the space occupied by the grayscale image data.

[0029] Preferably, the S414 is specifically:

[0030] Continuously upsample multiple grayscale images to obtain an upsampling result;

[0031] Among them, the magnification factor of each grayscale image during the upsampling process is 1.03 - 1.07.

[0032] Preferably, the S415 is specifically:

[0033] Continuously downsample multiple grayscale images to obtain a downsampling result;

[0034] Among them, the reduction factor of each grayscale image during the downsampling process is 0.92 - 0.98.

[0035] Preferably, the image object also encapsulates the original template image. Before the S43, the method further includes:

[0036] Based on the interactive setting, determine whether to use the grayscale mode for matching;

[0037] If so, convert the screenshot to the grayscale format and match the screenshot with the grayscale images in the image cache;

[0038] If not, match the screenshot with the original template image.

[0039] Through the above technical solution, the method of the present invention can also determine whether to use the grayscale mode for matching according to the actual needs of the user (i.e., the interactive setting). For some template images with particularly obvious image features, accurate matching can already be achieved using the original template image. Therefore, there is no need to additionally enable the grayscale mode matching function. At this time, directly matching the original template image with the screenshot can further improve the overall operation efficiency of the method of the present invention.

[0040] Preferably, if there is no area in the screenshot in S43 with a matching degree greater than the expected value, the method further includes:

[0041] Take out the sampled image list from the image object;

[0042] Perform template matching on the screenshot with each image in the sampled image list, and collect multiple matching results;

[0043] According to the multiple matching results, obtain the area with the highest matching degree;

[0044] Query whether any one of the matching results in the area is greater than the expected value; if so, sequentially execute the S431 - S432; if not, return the empty coordinate value.

[0045] Through the above technical solution, the method of the present invention also has a rejudgment function, which can avoid the "false killing" caused by matching oversights in S43, thereby improving the matching accuracy.

[0046] Preferably, the S50 is specifically:

[0047] According to the mouse information, based on the mouse call module in the Java standard library, call the mouse object, and automatically perform mouse movement, click, and release operations.

[0048] Preferably, the S60 is specifically:

[0049] According to the keyboard information, based on the keyboard call module in the Java standard library, call the keyboard object, and automatically perform key pressing and releasing operations.

[0050] In a second aspect, to achieve the above object, the present invention further provides a system for simulating manual operations based on the Java computer language, including:

[0051] A Jar package acquisition module for acquiring a Jar package;

[0052] Wherein, the configuration information in the Jar package at least includes screen information, mouse information, and keyboard information;

[0053] A project dependency setting module for setting the Jar package as a project dependency according to the configuration information to obtain a project document;

[0054] A multi-threaded execution module for parallel running the following modules according to the project document:

[0055] A screen call module for calling a screen object according to the screen information and automatically performing screenshot acquisition, matching, and storage operations;

[0056] A mouse call module for calling a mouse object according to the mouse information and automatically performing mouse movement, click, and release operations;

[0057] A keyboard call module for calling a keyboard object according to the keyboard information and automatically performing key pressing and releasing operations.

[0058] Through the above technical solutions, the system of the present invention is a cross-platform system relying on a method for simulating manual operations based on the Java computer language. Existing Python and Visual Basic languages are restricted by the operating systems they belong to and have poor compatibility, while the system of the present invention can run on multiple operating systems such as Windows, Linux, and Mac. Moreover, the system of the present invention has relevant modules with functions of parallel image search, obtaining operation mouse information, and obtaining operation keyboard information, and also utilizes the characteristic that the Java technology occupies less memory at the bottom layer, making the operation efficiency of the system of the present invention higher.

[0059] The present invention has at least the following advantages:

[0060] 1) Before the specific steps of the method of the present invention are executed, the JVM is used as the running environment to utilize the cross-platform characteristics of the JVM, so that the method of the present invention can run on multiple operating systems such as Windows, Linux, and Mac, with stronger compatibility. After using the JVM as the running environment, the method of the present invention first obtains a Jar package with screen information, mouse information, and keyboard information built in, then sets the Jar package as a project dependency to obtain a project document, and then, according to the project document, automatically and parallelly executes screenshot operations, mouse operations, and keyboard operations. This parallel method makes full use of the characteristic that the Java concurrent framework occupies less memory, greatly improving the running efficiency, thus solving the problem of low running efficiency in the prior art.

[0061] 2) For the automatic screenshot operation of images and graphics, the method of the present invention develops a new screenshot processing logic based on the JVM environment. First, based on the pre-set template image path, the template image is obtained, then the content of the screen object is intercepted to obtain a screenshot, and then the screenshot is matched with the pre-processed template image. This way of obtaining screenshots by matching with a template image will perform a simplified determination for each screenshot, which is beneficial to improving the accuracy and efficiency of the automatic screenshot operation. During the matching process, it is necessary to determine whether there is a region in the screenshot with a matching degree greater than the expected value. If so, perform screenshot storage, corner point return, and offset return operations. This method not only stores screenshots that meet the matching requirements, but also can record the corner points and offset information of relevant screenshots in real time, so as to further correct the screenshots.

[0062] 3) After the method of the present invention obtains the template image, it is necessary to determine whether the template image has been loaded in the image cache; if so, load the template image from the image cache and jump to S42, if not, sequentially execute S412 - S416. This method can flexibly call the cache content, reduce the repeated pattern sampling process, and further improve the running efficiency of the method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0064] Figure 1 It is a flowchart of the method for simulating manual operations based on the Java computer language in the first embodiment of the present invention;

[0065] Figure 2This is the flowchart of S41 - S42 in the method for simulating manual operations based on the Java computer language in the second embodiment of the present invention;

[0066] Figure 3 This is the flowchart of S42 - S43 in the method for simulating manual operations based on the Java computer language in the second embodiment of the present invention;

[0067] Figure 4 This is the structural schematic diagram of the system for simulating manual operations based on the Java computer language in the fifth embodiment of the present invention. Detailed implementation manners

[0068] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0069] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0070] This embodiment discloses a method and a system for simulating manual operations based on the Java computer language, mainly used to solve the problem of low operating efficiency in the prior art.

[0071] Embodiment 1

[0072] As Figure 1 shown, this embodiment discloses a method for simulating manual operations based on the Java computer language, which runs in the JVM environment. The method of the present invention includes:

[0073] S10: Obtain the Jar package.

[0074] It should be noted that the JVM, also known as the Java Virtual Machine, is a fictional computer dedicated to executing Java bytecodes. The JVM is a core component of the Java platform, providing a runtime environment that enables Java programs to run seamlessly on different hardware and operating system platforms, achieving the cross-platform feature of "write once, run anywhere". A Jar is a file format used to package multiple Java class files, related metadata, and resources. Third-party libraries in Java are usually provided in the form of JAR files, and developers can add these JAR files to the classpath of a project to use the classes and methods provided by these libraries in the project.

[0075] In this embodiment, the configuration information in the Jar package includes at least screen information, mouse information, and keyboard information to facilitate providing paths for object calls of the screen, mouse, and keyboard.

[0076] Specifically, the screen information may include size, aspect ratio, resolution, refresh rate, response time, panel type, number, number, name, etc. The mouse information may include number, name, and driver information, etc. The keyboard information may include keyboard number, name, and driver information, etc.

[0077] S20: Set the Jar package as a project dependency according to the configuration information to obtain a project document.

[0078] It should be noted that in a Java project, using the Jar package as a project dependency facilitates project compilation and running, and also enables the project to access the classes, interfaces, methods, and resources provided in the Jar package, thereby accelerating the development speed.

[0079] S30: Execute the following steps in parallel according to the project document:

[0080] S40: Call the screen object according to the screen information and automatically perform screenshot capture, matching, and storage operations;

[0081] S50: Call the mouse object according to the mouse information and automatically perform mouse movement, click, and release operations;

[0082] S60: Call the keyboard object according to the keyboard information and automatically perform key pressing and releasing operations.

[0083] Through the above technical solution, the method of the present invention implements RPA using the Java language. Different from Python and Visual Basic languages, as an object-oriented programming language, Java has a large-scale third-party library, which is easy to expand functional modules. On this basis, due to the excellent cross-platform characteristics of the JVM technology, the method of the present invention can run on various operating systems such as Windows, Linux, and Mac, thereby making the application scope of the present invention wider. In addition, the method of the present invention solves the problem that software with an operation interface cannot work together without an application programming interface (hardware interface). Through the method of the present invention, repetitive computer operations performed manually can be automated, improving work efficiency.

[0084] Compared with the prior art, the method of the present invention realizes the functions of parallel image capture, operation mouse information acquisition, and operation keyboard information acquisition, and utilizes the characteristic that the underlying Java technology occupies less memory, greatly improving the running efficiency, thereby solving the problem of low running efficiency in the prior art.

[0085] Embodiment 2

[0086] As Figure 2 and Figure 3 shown, on the basis of Embodiment 1, the operations of automatically performing screenshot acquisition, matching, and storage in S40 are specifically as follows:

[0087] S41: Based on the pre-set template image path, obtain the template image.

[0088] In this embodiment, the template image path needs to be pre-input or set by the user, that is, the user sets the search path of the template graphic in the JVM.

[0089] After obtaining the template image, in order to improve the matching accuracy of the subsequent screenshots and the template image, and flexibly call the cache resources to improve the overall running speed, the following steps need to be executed:

[0090] S411: Determine whether the template image has been loaded in the image cache; if so, load the template image from the image cache and jump to S42; if not, sequentially execute S412 - S416.

[0091] Through the above S411, the method of the present invention can flexibly call the cache content, reduce the repeated pattern sampling process, and further improve the running efficiency of the method of the present invention.

[0092] S412: Read the template image in the target image format to obtain a full-color image.

[0093] S413: Perform grayscale calculation on the full-color image to generate a grayscale image.

[0094] S414: Perform upsampling calculation on the grayscale image to obtain an upsampling result.

[0095] S415: Perform downsampling calculation on the grayscale image to obtain a downsampling result.

[0096] S416: Encapsulate the down-sampling result of the up-sampling result into an image object, and after the image object is placed in the image cache, jump to S42.

[0097] In this embodiment, the target image format mainly refers to the GBR format. The up-sampling result and the down-sampling result need to be put into the sampling image list.

[0098] It should be explained that, in the process of grayscale calculation of a full-color image, the color information (usually including three components of red, green and blue) of each pixel of the full-color image will be converted into a single grayscale value.

[0099] Furthermore, in order to enhance the image features of the grayscale image, the method of the present invention upsamples the grayscale image through S414. In this embodiment, S414 is specifically:

[0100] 40 grayscale images are continuously upsampled to obtain the upsampling result.

[0101] Preferably, the magnification of each grayscale image during upsampling is 1.05.

[0102] Furthermore, in order to extract the features of the grayscale image and achieve data dimensionality reduction, the method of the present invention performs downsampling through S415. In this embodiment, S415 is specifically:

[0103] Continuously downsample 10 grayscale images to get the downsampling results.

[0104] Preferably, the reduction factor of each grayscale image during the downsampling process is 0.95.

[0105] After the above S41, the method of the present invention further comprises:

[0106] S42: Capture the content of the screen object to obtain a screenshot.

[0107] It should be noted that the image capture mode can be automatically captured based on a predetermined size or can be automatically captured randomly.

[0108] S43: Match the screenshot with the preprocessed template image to determine whether there is an area in the screenshot with a matching degree greater than an expected value; if so, execute S431-S432 in sequence;

[0109] S431: Read the corner coordinates of the region in the screenshot with a matching degree greater than the expected value, and perform an offset calculation on the corner coordinates according to the preset user-expected coordinates to obtain an offset result;

[0110] S432: Store the screenshot and the offset result determined by S43 and return the corner coordinates.

[0111] In this embodiment, the above corner coordinates are the point coordinates of the upper left corner of the region. The above preprocessing includes grayscale operation, upsampling and downsampling processes, and the template images used for matching are mainly divided into the above grayscale images and the original template images.

[0112] Different from the existing Python RPA and Visual Basic RPA tools, the time required for S41 - S43 in the method of the present invention for graphic image feature recognition is shorter, further accelerating the processing rate of RPA type tasks. Additionally, each stored screenshot will return the corner coordinates with the offset result attached, facilitating further correction of the screenshot.

[0113] Preferably, the image object also encapsulates the original template image. Before S43, the method of the present invention further includes:

[0114] Based on the interactive setting, determine whether to use the grayscale mode for matching;

[0115] If so, convert the screenshot to grayscale format and match the screenshot with the grayscale image in the image cache;

[0116] If not, match the screenshot with the original template image.

[0117] Through the above technical solution, the method of the present invention can, based on the user interaction setting, determine whether to enable the grayscale mode for matching. For example, for some template images with obvious image features, the original template image without grayscale processing can be directly selected to match the screenshot. Compared with the prior art, the above technical solution further improves the operation efficiency.

[0118] Preferably, if there is no region in the screenshot with a matching degree greater than the expected value in the above S43, the method of the present invention further includes:

[0119] Take out the sampling image list from the image object;

[0120] Perform template matching between the screenshot and each image in the sampling image list, and collect multiple matching results;

[0121] According to the multiple matching results, obtain the region with the highest matching degree;

[0122] Whether any matching result within the query area is greater than the expected value; if so, sequentially execute S431 - S432; if not, return an empty coordinate value.

[0123] Through the above technical solution, if there is no area with a matching degree greater than the expected value in the S43 screenshot, it indicates that there may be a problem with the template image used for matching. At this time, the method of the present invention will call all the images in the sampling image list (including grayscale images and the original template image), and these images will be rematched with the screenshot to reduce the false detection rate of the method of the present invention.

[0124] Embodiment III

[0125] Based on Embodiment I, S50 is specifically:

[0126] According to the mouse information, based on the mouse call module in the Java standard library, call the mouse object and automatically execute mouse movement, click, and release operations.

[0127] It should be explained that there is a built - in mouse call module in the Java standard library, namely java.awt.Robot, and its specific existing function functions are:

[0128] mouseMove(int x, int y)

[0129] mousePress(int buttons)

[0130] mouseRelease(int buttons)

[0131] Through the above technical solution, the function of calling the mouse object and automatically executing mouse movement, click, and release operations can be realized.

[0132] Embodiment IV

[0133] Based on Embodiment I, S60 is specifically:

[0134] According to the keyboard information, based on the keyboard call module in the Java standard library, call the keyboard object and automatically execute key pressing and releasing operations.

[0135] It should be explained that there is a built - in keyboard call module in the Java standard library, namely java.awt.Robot, and its specific existing function functions are:

[0136] 1keyPress(int keycode)

[0137] keyRelease(int keycode)

[0138] Through the above technical solution, it is possible to call the keyboard object and automatically execute the key pressing and releasing operations.

[0139] Embodiment 5

[0140] As Figure 4 shown, this embodiment discloses a system for simulating manual operations based on the Java computer language, including:

[0141] A Jar package acquisition module for acquiring the Jar package:

[0142] wherein, the configuration information in the Jar package includes at least screen information, mouse information, and keyboard information;

[0143] A project dependency setting module for setting the Jar package as a project dependency according to the configuration information to obtain a project document;

[0144] A multi-threaded execution module for parallelly running the following modules according to the project document:

[0145] A screen call module for calling a screen object according to the screen information and automatically executing screenshot acquisition, matching, and storage operations;

[0146] A mouse call module for calling a mouse object according to the mouse information and automatically executing mouse movement, clicking, and releasing operations;

[0147] A keyboard call module for calling a keyboard object according to the keyboard information and automatically executing key pressing and releasing operations.

[0148] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes various media such as ROM, RAM, magnetic disk, or optical disc that can store program codes.

[0149] For those skilled in the art, according to the technical solutions and concepts described above, various corresponding changes and deformations can be made, and all such changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. A method based on the Java computer language for simulating manual operations, characterized in that, Run in the JVM environment, the method includes: S10: Obtain the Jar package; Wherein, the configuration information in the Jar package includes at least screen information, mouse information, and keyboard information; S20: Set the Jar package as a project dependency according to the configuration information to obtain a project document; S30: According to the project document, execute the following steps in parallel: S40: According to the screen information, call the screen object to automatically perform screenshot acquisition, matching, and storage operations; S50: According to the mouse information, call the mouse object to automatically perform mouse movement, click, and release operations; S60: According to the keyboard information, call the keyboard object to automatically perform key tapping and release operations.

2. The method for simulating manual operations based on the Java computer language according to claim 1, wherein The operations of automatically performing screenshot acquisition, matching, and storage in S40 are specifically: S41: Obtain the template image based on the pre-set template image path; S42: Capture the content of the screen object to obtain a screenshot; S43: Match the screenshot with the pre-processed template image to determine whether there is a region in the screenshot with a matching degree greater than the expected value; if so, sequentially execute S431 - S432; S431: Read the corner coordinates of the region in the screenshot with a matching degree greater than the expected value, and perform offset calculation on the corner coordinates according to the pre-set user expected coordinates to obtain an offset result; S432: Store the screenshot determined by S43 and the offset result and return the corner coordinates.

3. The method for simulating manual operations based on the Java computer language according to claim 2, wherein, After S41, the method further includes: S411: Determine whether the template image has been loaded in the image cache; if so, load the template image from the image cache and jump to S42; if not, sequentially execute S412 - S416; S412: Read the template image in the target image format to obtain a full-color image; S413: Perform grayscale calculation on the full-color image to generate a grayscale image; S414: Perform upsampling calculation on the grayscale image to obtain an upsampling result; S415: Perform downsampling calculation on the grayscale image to obtain a downsampling result; S416: Package the upsampling result and the downsampling result into an image object, and after placing the image object in the image cache, jump to S42.

4. The method for simulating manual operations based on the Java computer language according to claim 3, characterized in that, S414 is specifically: Continuously upsample multiple grayscale images to obtain an upsampling result; Wherein, the magnification factor of each grayscale image during upsampling is 1.03 - 1.

07.

5. The method for simulating manual operations based on the Java computer language according to claim 3, wherein S415 is specifically: Continuously downsample multiple grayscale images to obtain a downsampling result; Wherein, the reduction factor of each grayscale image during downsampling is 0.92 - 0.

98.

6. The method for simulating manual operations based on the Java computer language according to claim 3, characterized in that, The image object also packages the original template image. Before S43, the method further includes: Based on the interactive setting, determine whether to perform matching in grayscale mode; If so, convert the screenshot to grayscale format and match the screenshot with the grayscale image in the image cache; If not, match the screenshot with the original template image.

7. The method for simulating manual operations based on the Java computer language according to claim 6, wherein If there is no region in the screenshot in S43 with a matching degree greater than the expected value, the method further includes: Take out the list of sampled images from the image object; Perform template matching between the screenshot and each image in the list of sampled images, and collect multiple matching results; Obtain the region with the highest matching degree according to multiple matching results; Query whether any one of the matching results in the region is greater than the expected value; if so, sequentially execute S431 - S432; if not, return the empty coordinate value.

8. The method for simulating manual operations based on the Java computer language according to claim 1, wherein The specific content of S50 is as follows: According to the mouse information, based on the mouse call module in the Java standard library, call the mouse object to automatically perform mouse movement, click, and release operations.

9. The method for simulating manual operations based on the Java computer language according to claim 1, wherein, The specific content of S60 is as follows: According to the keyboard information, based on the keyboard call module in the Java standard library, call the keyboard object to automatically perform key pressing and releasing operations.

10. A system based on the Java computer language for simulating manual operations, characterized in that, It includes: The Jar package acquisition module is used to acquire the Jar package; Among them, the configuration information in the Jar package at least includes screen information, mouse information, and keyboard information; The project dependency setting module is used to set the Jar package as a project dependency according to the configuration information to obtain a project document; The multi-thread execution module is used to parallelly run the following modules according to the project document: The screen call module is used to call the screen object according to the screen information to automatically perform screenshot acquisition, matching, and storage operations; The mouse call module is used to call the mouse object according to the mouse information to automatically perform mouse movement, click, and release operations; The keyboard call module is used to call the keyboard object according to the keyboard information to automatically perform key pressing and releasing operations.

Images