A method for implementing automatic operation of Java Applet

Abstract

The application discloses a kind of methods for realizing Java Applet automation operation, comprising the following steps: step one: write a UDP Server, define interface element object structure, structure contains the handle that element holds, on the location of interface, element type data;Step two: write Java Plugin by Java Accessibility API, this Java Plugin is mainly loaded by JVM when starting;Step three: write Native Java Bridge, for JVM, operating system, and automation program communication, through HOOK technology mounts global mouse event, when mouse moves to specified Java Applet, call Plugin, and Java interface element object instance is reflected;Step four: construct UDP Client in Native Java Bridge, when Java interface element object is reflected, it is serialized into the same data type agreed with UDP Server, and is sent to UDP Server by UDP Client;The application realizes the automation operation of system developed with Java Applet technology, builds overall automatic platform, so that each system can be automatically connected, instead of traditional manual operation.

Description

Technical Field

[0001] This invention belongs to the field of RPA automation technology, specifically relating to a method for automating Java Applet operations. Background Technology

[0002] When developing business process automation (RPA) systems using Java Applet technology, there are currently no suitable solutions in the industry. Therefore, after other systems or platforms capable of automation have completed their tasks, it's impossible to directly connect to the Java Applet system for further automation, leading to a break in the RPA automation process and preventing the automation from continuing. Summary of the Invention

[0003] The purpose of this invention is to provide a method for automating Java Applet operations, thereby solving the problems mentioned in the background art, such as the inability to automate Java Applet applications and the inability of Java Applets to connect with other automated testing systems for testing.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for automating Java Applet operations, comprising the following steps:

[0005] Step 1: Write a UDP server and define the structure of the interface element objects. The structure includes the handle held by the element, its position on the interface, and the element type data.

[0006] Step 2: Write a Java Plugin using the Java Accessibility API. This Java Plugin is primarily loaded at JVM startup.

[0007] Step 3: Write a Native Java Bridge for communication between the JVM, the operating system, and automation programs. Use hooking technology to attach global mouse events. When the mouse moves over a specified Java Applet, call the Plugin to reflect the Java UI element object instance.

[0008] Step 4: Construct a UDP Client in the Native Java Bridge. When Java UI element objects are reflected, serialize them into the same data type agreed upon with the UDP Server, and send them to the UDP Server through the UDP Client.

[0009] Step 5: Construct a TCP Server bound to Localhost in the Native Java Bridge. In the Server, add mouse click and keyboard input functions by connecting to the win32 dynamic library. For the passed object, encapsulate it into a separate action. The input parameter is a serialized object. If you need to press a button in the Java Applet, get the position of the element on the screen through the serialized object, pass its position as a parameter to the MouseEvent of the mouse and trigger the event to complete the automatic click operation.

[0010] Step 6: Package the Java Plugin into a JAR file and the Native Java Bridge into a DLL file;

[0011] Step 7: Locate the JRE installation directory on the machine, place the written jar and dll files in the bin and ext directories of the JRE respectively, and restart the Java process and Java Applet program;

[0012] Step 8: Write a TCP client to automate communication operations, and wrap it into automated action export methods for Java Applet interface elements for use by other applications.

[0013] Preferably, the above steps can form an automated operation application framework for Java Applet. The interface elements can be serialized and saved through a UDP Server and used as input parameters for automated operations. By referencing a TCPClient, the clicking of elements, a series of input operations, and the content displayed in the elements can be obtained to achieve automated operations.

[0014] Preferably, in step one, the UDP Server is used to receive element features transmitted from the Native Java Bridge, and can be written in Python, C++, or C#.

[0015] Preferably, in step two, the Java Plugin is written in Java and is loaded by the Class Loader when the JVM starts.

[0016] Preferably, in step four, the same data type, namely the JSON data structure defined in the UDP Server, can be sent to the UDP Server and then the user can decide whether to save the element characteristics as input elements for subsequent automated operations based on the automation use case.

[0017] Preferably, in step eight, the TCP Client can be written in Python, C++, or C# to maintain the consistency of the transmitted data structure and achieve the purpose of heterogeneous system calls.

[0018] Preferably, the TCP Client may include automated operation use cases, such as clicking a button in a Java Applet, obtaining some content from the interface element, and inputting some content into the Java Applet element to achieve the purpose of automated operation.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] 1. This invention automates the operation of systems developed using Java Applet technology, builds an overall automated platform, enables automated connections between various systems, replaces traditional manual operations, and improves work efficiency;

[0021] 2. This invention provides a tool for collecting and identifying Java Applet elements and a method for automating their operation, enabling the operation of Java Applets by humans in business process automation to be implemented by a program, which is more convenient and efficient than traditional manual operation. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention;

[0023] Figure 2 This is a schematic diagram illustrating the feature description of Java Applet element generation in this invention;

[0024] Figure 3 This is a schematic diagram illustrating how the present invention performs automated operations based on element features. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] First, the technical terms used in this article are explained as follows:

[0027] Java Applet: This is equivalent to an application, typically hosted in the browser process of browsers such as IE, and implemented using Java.

[0028] JAB: Java Access Bridge is a technology that exposes the Java Accessibility API in a Microsoft Windows DLL, making Java applications and applets that implement the Java Accessibility API visible to accessibility technologies on Microsoft Windows systems.

[0029] UDP Server: A service built using the UDP protocol that can receive data sent via the UDP protocol.

[0030] UDP Client: A client implemented using the UDP protocol that can send data via UDP.

[0031] TCP Server: A service built using the TCP protocol that can receive data sent via the TCP protocol.

[0032] TCP Client: A client implemented using the TCP protocol that can send data via the TCP protocol.

[0033] Example

[0034] Please see Figures 1 to 3 This invention provides a technical solution: a method for automating Java Applet operations, comprising:

[0035] Step 1: Write a UDP server and define the structure of the interface element objects. The structure contains data such as the handle held by the element, its position on the interface, and the element type.

[0036] Furthermore, the UDP Server is used to receive element characteristics transmitted from the Native Java Bridge, and can be written in languages ​​such as Python, C++, and C#. Within the UDP Server, a data structure describing the element characteristics needs to be defined, typically using a JSON data structure type.

[0037] Step 2: Write a Java Plugin using the Java Accessibility API. This Java Plugin is primarily loaded at JVM startup.

[0038] Furthermore, the Java Plugin is written in Java and is loaded by the Class Loader when the JVM starts. Its main purpose is to use the Class Loader to reflect the Classes of UI elements, and then iterate through the corresponding element's properties, methods, and states using these Classes to form a JSON data structure.

[0039] Step 3: Write a Native Java Bridge for communication between the JVM, the operating system, and automation programs. Use hooking technology to attach global mouse events. When the mouse moves over a specified Java Applet, call the Plugin to reflect the Java UI element object instance.

[0040] Step 4: Construct a UDP Client in the Native Java Bridge. When Java UI element objects are reflected, serialize them into the same data type agreed upon with the UDP Server, and send them to the UDP Server through the UDP Client.

[0041] Furthermore, the same data type here refers to the JSON data structure defined in the UDP Server. After being sent to the UDP Server, it can be determined whether to save the element characteristics as input elements for subsequent automated operations based on the automation use case.

[0042] Step 5: Construct a TCP Server bound to the Localhost within the Native Java Bridge. The Server connects to a Win32 dynamic library and adds functionality such as mouse clicks and keyboard input. For each input object, encapsulate it into a separate action, with a serialized object as the input parameter. For example, to press a button in a Java Applet, obtain the element's position on the screen through the serialized object, pass that position as a parameter to the mouse's MouseEvent event, and trigger the event to complete the automated click operation.

[0043] Step 6: Package the Java Plugin into a JAR file and the Native Java Bridge into a DLL file;

[0044] Step 7: Locate the JRE installation directory of the execution machine, place the written jar and dll files in the bin and ext directories of the JRE respectively, and restart the Java process and Java Applet program;

[0045] Step 8: Write a TCP Client to automate communication operations, and wrap it into automated action export methods for Java Applet UI elements for use by other applications.

[0046] Furthermore, TCP clients can also be written in languages ​​such as Python, C++, and C#. As long as the consistency of the transmitted data structure is maintained, the purpose of heterogeneous system calls can be achieved. At the same time, TCP clients can include automated operation test cases, such as clicking a button in a Java Applet, retrieving some content from the interface element, and inputting some content into the Java Applet element to achieve the purpose of automated operation.

[0047] For example, logging into a Java Applet application requires entering a username and password:

[0048] 1) Select the username, password, and login button elements in the Java Applet and save the element characteristics;

[0049] 2) Configure the input content corresponding to the element, and use the TCP Client to send the element characteristics and input method to input the account and password;

[0050] 3) The Native Java Bridge input method calls the JAB to find the coordinates of the username and password on the Java Applet. Users can then click on these coordinates with the mouse and enter the username and password using the keyboard.

[0051] 4) Similarly, find the coordinates of the login button, move the mouse to these coordinates, and left-click.

[0052] 5) Login complete.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for automating Java Applet operations, characterized in that: Includes the following steps: Step 1: Write a UDP server and define the structure of the interface element objects. The structure includes the handle held by the element, its position on the interface, and the element type data. Step 2: Write a Java Plugin using the Java Accessibility API. This Java Plugin is primarily loaded at JVM startup. Step 3: Write a Native Java Bridge for communication between the JVM, the operating system, and automation programs. Use hooking technology to attach global mouse events. When the mouse moves over a specified Java Applet, call the Plugin to reflect the Java UI element object instance. Step 4: Construct a UDP Client within the Native Java Bridge. When Java UI element objects are reflected, serialize them into the same data type agreed upon with the UDP Server, and send them to the UDP Server through the UDP Client. Step 5: Construct a TCP Server bound to Localhost within the Native Java Bridge. The Server connects to a Win32 dynamic library and adds mouse click and keyboard input functionality. For each input object, encapsulate it into a separate action, with a serialized object as the input parameter. When a button in the Java Applet is pressed, the element's position on the screen is obtained through the serialized object, and this position is passed as a parameter to the MouseEvent event to trigger the event, thus completing the automated click operation. Step 6: Package the Java Plugin into a JAR file and the Native Java Bridge into a DLL file; Step 7: Locate the JRE installation directory on the machine, place the written jar and dll files in the bin and ext directories of the JRE respectively, and restart the Java process and Java Applet program; Step 8: Write a TCP client to automate communication operations, and wrap it into automated action export methods for Java Applet interface elements for use by other applications.

2. The method for automating Java Applet operations according to claim 1, characterized in that: The above steps form a Java Applet automation application framework. The interface elements are serialized and saved via a UDP server to serve as input parameters for automation operations. A TCP client is used to trigger a series of operations such as clicking elements, inputting data, and retrieving the content displayed in the elements, thereby achieving automation.

3. The method for automating Java Applet operations according to claim 1, characterized in that: In step one, the UDP Server is used to receive element features transmitted from the Native Java Bridge and is written in Python, C++, or C#.

4. The method for implementing automated operations of Java Applets according to claim 1, characterized in that: In step two, the Java Plugin is written in Java and is loaded by the Class Loader when the JVM starts.

5. The method for automating Java Applet operations according to claim 1, characterized in that: In step four, the same data type, namely the JSON data structure defined in the UDP Server, is sent to the UDP Server. After that, the system decides whether to save the element characteristics as input elements for subsequent automated operations based on the automation use case.

6. The method for implementing automated operations of Java Applets according to claim 1, characterized in that: In step eight, the TCP Client is written in Python, C++, or C# to maintain the consistency of the transmitted data structure and achieve the purpose of heterogeneous system calls.

7. A method for automating Java Applet operations according to claim 6, characterized in that: The TCP Client includes automated operation use cases. When a button in a Java Applet is clicked, some content from the interface element is retrieved, and some content is entered into the Java Applet element to achieve the purpose of automated operation.

Images