Data processing method and apparatus

Abstract

The application provides a data processing method and device, the method comprises the following steps: identifying an interface element and obtaining a flow control table; judging whether the interface element is an interface object with an identifiable interface object ID based on the flow control table; if yes, executing an operation driving script and a judgment operation script in the flow control table based on the interface element to archive a target interface; if no, executing a fitting operation to obtain an operation result; judging whether the interface element is an operation object based on the operation result; if yes, executing the operation driving script and the judgment operation script to archive the target interface. The method provided by the application can archive an interface without obtaining an interface object ID through an open API and manipulating the interface across application windows.

Description

Technical Field

[0010]

[0001] The present invention relates to the field of computer technology, and particularly to a data processing method and apparatus. Background Art

[0002] In the selection and testing of information technology products, it is necessary to perform video monitoring on the test site and the execution process of some key test cases, and upload the videos for archiving and backup. Since there are various monitoring tools and a large number of monitoring screens, and it is necessary to ensure that the archived video interfaces and interface elements are consistent in time. Since the operations of sorting, archiving, and backing up the monitoring videos often need to be performed on a single monitoring console and involve associated operations on the client interfaces of multiple monitoring tools. If associated operations are required, the interface object IDs need to be obtained through the API to intercept the interface elements in the interface. At this time, problems such as cross-window execution and technical differences in the implementation of client interfaces of different monitoring tools will be involved. Especially when the monitoring tool adopts a C / S architecture, it is very difficult to implement an automated process by directly calling the interface elements in an automated script. Summary of the Invention

[0003] In view of this, the present invention provides a data processing method, by which it is not necessary to obtain the interface object ID through an open API and manipulate the interface across application windows.

[0004] The present invention also provides a data processing apparatus to ensure the implementation and application of the above method in practice.

[0005] A data processing method includes:

[0006] Identify the interface elements of the target interface and obtain a preset process control table;

[0007] Based on the process control table, determine whether the interface element is an interface object that can identify the interface object ID;

[0008] If the interface element is an interface object that can identify the interface object ID, then based on the interface element, execute the operation drive script and the determination operation script corresponding to the target interface in the process control table to archive the target interface;

[0009] If the interface element is not an interface object that can identify the interface object ID, then perform the fitting operation corresponding to the target interface to obtain an operation result;

[0010] Based on the operation result, determine whether the interface element is an operable object;

[0011] If the interface element is an operable object, then execute the operation drive script and the determination operation script corresponding to the target interface in the process control table to archive the target interface.

[0012] Optionally, the above method may further include, before identifying the interface elements of the target interface:

[0013] Construct the process control table;

[0014] Obtain the process control parameters uploaded by the user in advance. The process control parameters include the interface object ID or interface element diagram corresponding to multiple interface elements to be operated.

[0015] Based on the process control parameters, construct the operation driving script and the judgment operation script corresponding to each of the interface elements to be operated, which are included in the process control parameters.

[0016] Each of the interface element images is saved to a preset interface element image library, and the interface object ID, interface element image, operation driving script, and judgment operation script of each interface element to be operated are written into the process control table.

[0017] Optionally, in the above method, performing the fitting operation corresponding to the target interface to obtain the operation result includes:

[0018] Obtain the target interface element image corresponding to the interface element in the interface element image library;

[0019] Obtain a screenshot of the target interface;

[0020] The screenshot of the interface is fitted to the target interface element image to obtain the matching degree between the screenshot of the interface and the target interface element image. The matching degree is the operation result.

[0021] Optionally, in the above method, determining whether the interface element is an operable object based on the operation result includes:

[0022] Get the pre-set standard matching threshold;

[0023] Determine whether the matching degree between the screenshot and the target interface element image falls within the standard matching threshold;

[0024] If the matching degree between the interface element and the target interface element image is within the standard matching threshold, then the interface element is determined to be an operable object.

[0025] If the matching degree between the interface element and the target interface element image does not fall within the standard matching threshold, then the interface element is determined to be an operable object.

[0026] The above methods may also include:

[0027] The target interface element image and the interface screenshot are processed into grayscale to obtain grayscale images corresponding to the target interface element image and the interface screenshot, respectively.

[0028] A data processing apparatus, comprising:

[0029] The identification unit is used to identify the interface elements of the target interface and obtain the preset flow control table.

[0030] The first judgment unit is used to determine whether the interface element is an interface object with an identifiable interface object ID based on the process control table.

[0031] The first execution unit is configured to, if the interface element is an interface object with an identifiable interface object ID, execute the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table based on the interface element, so as to archive the target interface.

[0032] The second execution unit is used to perform the fitting operation corresponding to the target interface and obtain the operation result if the interface element is not an interface object with an identifiable interface object ID.

[0033] The second judgment unit is used to determine whether the interface element is an operable object based on the operation result.

[0034] The third execution unit is used to execute the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table if the interface element is an operable object, so as to archive the target interface.

[0035] The aforementioned apparatus may optionally further include:

[0036] The first construction unit is used to construct the process control table;

[0037] The acquisition unit is used to acquire the process control parameters pre-uploaded by the user. The process control parameters include the interface object ID or interface element diagram corresponding to multiple interface elements to be operated.

[0038] The second construction unit is used to construct operation driving scripts for each of the identifiable interface elements included in the process control parameters based on the process control parameters.

[0039] The writing unit is used to save each of the interface element images to a preset interface element image library, and to write the interface object ID, interface element image, operation driving script and judgment operation script of each interface element to be operated into the flow control table.

[0040] Optionally, in the aforementioned apparatus, the second execution unit includes:

[0041] The first acquisition subunit is used to acquire the target interface element image corresponding to the interface element in the interface element image library;

[0042] The second acquisition subunit is used to acquire a screenshot of the target interface;

[0043] The fitting subunit is used to fit the interface screenshot with the target interface element image to obtain the matching degree between the interface elements and the target interface element image, and the matching degree is the operation result.

[0044] Optionally, the second determining unit in the aforementioned apparatus includes:

[0045] The third acquisition subunit is used to acquire a pre-set standard matching threshold.

[0046] The judgment subunit is used to determine whether the matching degree between the interface screenshot and the target interface element image belongs to the standard matching threshold.

[0047] The first determining subunit is used to determine that the interface element is an operable object if the matching degree between the interface element and the target interface element image is within the standard matching threshold.

[0048] The second determining subunit is used to determine that the interface element is not an operable object if the matching degree between the interface element and the target interface element graph does not belong to the standard matching threshold.

[0049] The aforementioned apparatus may optionally further include:

[0050] The processing unit is used to perform grayscale processing on the target interface element image and the interface screenshot to obtain grayscale images corresponding to the target interface element image and the interface screenshot respectively.

[0051] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides controls the execution of the data processing method described above.

[0052] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the data processing method described above.

[0053] Compared with the prior art, the present invention has the following advantages:

[0054] This invention provides a data processing method, comprising: identifying interface elements of a target interface and obtaining a preset flow control table; determining, based on the flow control table, whether the interface element is an interface object with an identifiable interface object ID; if the interface element is an interface object with an identifiable interface object ID, then, based on the interface element, executing the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table to archive the target interface; if the interface element is not an interface object with an identifiable interface object ID, then executing the fitting operation corresponding to the target interface to obtain an operation result; based on the operation result, determining whether the interface element is an operation object; if the interface element is an operable object, then executing the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table to archive the target interface. By applying the method provided by this invention, interface archiving can be performed without obtaining interface object IDs through open APIs and manipulating the interface across application windows. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 A flowchart of a data processing method provided in an embodiment of the present invention;

[0057] Figure 2 Another flowchart of a data processing method provided in an embodiment of the present invention;

[0058] Figure 3 A further flowchart of a data processing method provided in an embodiment of the present invention;

[0059] Figure 4 This is a structural diagram of a data processing apparatus provided in an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0063] This invention can be used in a wide variety of general-purpose or special-purpose computing environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0064] This invention provides a data processing method that can be applied to various system platforms. The executing entity can be a computer terminal or a processor of various mobile devices. The method flowchart is shown below. Figure 1 As shown, it specifically includes:

[0065] S101: Identify the interface elements of the target interface and obtain the preset flow control table.

[0066] It should be noted that the target interface can be the monitoring screen that the user can view, and the interface elements are the objects that can be operated in the monitoring.

[0067] Specifically, before identifying the interface elements of the target interface, it may also include:

[0068] Construct the process control table;

[0069] Obtain the process control parameters uploaded by the user in advance. The process control parameters include the interface object ID or interface element diagram corresponding to multiple interface elements to be operated.

[0070] Based on the process control parameters, construct the operation driving script and the judgment operation script corresponding to each of the interface elements to be operated, which are included in the process control parameters.

[0071] Each of the interface element images is saved to a preset interface element image library, and the interface object ID, interface element image, operation driving script, and judgment operation script of each interface element to be operated are written into the process control table.

[0072] It should also be noted that the process control table can be an Excel file or a database table, used for automatically scheduling operation processes. This process control table may include information such as whether the driving interface is an object with a recognizable ID, whether the determining interface is an object with a recognizable ID, the driving script identifier based on the interface object ID, the determining script identifier based on the interface object ID, the image identifiers captured by the interface and their access paths, the operation events corresponding to the image identifiers, and the determining status corresponding to the interface identifiers.

[0073] In this invention, a process control table is pre-purchased. The user uploads all interface elements requiring operation, along with the corresponding interface object ID or interface element diagram for each element. Based on the process control parameters, corresponding operation-driven scripts and decision-making scripts are written. The interface object IDs, interface element diagrams, operation-driven scripts, and decision-making scripts are then written into the process control parameters. When certain interface elements in the target interface need to be processed, the need for operation is determined by identifying the interface elements.

[0074] Furthermore, the operation-driven script is used to perform driving operations on interface elements, and the judgment-driven script is used to determine whether the driving operation of the operation-driven script is successful.

[0075] S102: Based on the process control table, determine whether the interface element is an interface object with an identifiable interface object ID.

[0076] In this invention, the ID corresponding to the identifiable interface is obtained from the process control table, and the interface object ID is used to determine whether the interface element is an interface object that needs to be operated.

[0077] It should be noted that a UI object refers to a UI element on the screen that needs to perform a specific operation. The processor can determine whether the currently identified UI element belongs to the UI object with that UI object ID by using the UI object ID.

[0078] If the interface element is an interface object with a recognizable interface object ID, then S103 is executed; if the interface identifier is not a recognizable interface object ID, then S104 is executed.

[0079] S103: Based on the interface elements, execute the operation-driven script and judgment operation script corresponding to the target interface in the process control table to archive the target interface.

[0080] In this invention, the operation-driven script can be used to trigger interface events and execute interface operations. The determination script is mainly used to obtain the operation results of the operation-driven script.

[0081] S104: Perform the fitting operation corresponding to the target interface to obtain the operation result.

[0082] It should be noted that the fitting operation refers to fitting the interface elements in the target interface to the relevant image.

[0083] S105: Based on the operation result, determine whether the interface element is an operable object.

[0084] If the interface element is an operable object, or if the interface element is changed to an interface object with an identifiable interface object ID, then S106 is executed; otherwise, the target interface is skipped.

[0085] S106: Execute the operation-driven script and the judgment operation script corresponding to the target interface in the process control table to archive the target interface.

[0086] In this embodiment of the invention, the execution method of the operation driving script and the determination operation script in S106 is the same as that of the operation driving script and the determination operation script in S102 above.

[0087] It should be noted that during the fitting process, screenshots of the interface elements in the target interface can be captured and saved.

[0088] In the method provided by this invention embodiment, when archiving a target interface, interface elements in the target interface are identified, and a flow control table is obtained. It is determined whether an interface element is an interface object with an identifiable interface object ID. If so, the interface needs to be archived according to the operation-driven script and the judgment operation script; otherwise, a fitting operation is required. Through the fitting operation, it is re-determined whether the interface element is an operable object. If so, the operation-driven script and the judgment operation script are executed to archive the target interface, skipping the target interface.

[0089] By applying the method provided in this embodiment of the invention, it is possible to automate processes without obtaining the interface object ID and manipulating the interface across application windows through open APIs.

[0090] In the method provided by this embodiment of the invention, the process of performing a fitting operation corresponding to the target interface and obtaining the operation result is as follows: Figure 2 As shown, it can specifically include:

[0091] S201: Obtain the target interface element image corresponding to the interface element in the interface element image library.

[0092] It should be noted that the interface element image library stores multiple interface element images.

[0093] S202: Obtain a screenshot of the target interface.

[0094] It should be noted that when updating to the target interface, the processor can automatically capture a screenshot of the target interface.

[0095] S203: The screenshot of the interface is fitted to the target interface element image to obtain the matching degree between the screenshot of the interface and the target interface element image, and the matching degree is the operation result.

[0096] In the method provided by this invention, when it is uncertain whether an interface element can be operated, it is necessary to determine whether the user should upload an image of the interface element that needs to be operated on, based on the relevant parameters recorded in the process control table. The saved target interface element image is retrieved from the interface element image library; then, a screenshot of the interface element on the target interface is captured, and the two images are fitted to obtain the matching degree. This matching degree is the operation result.

[0097] In the method provided by this embodiment of the invention, the process of determining whether the interface element is an operable object based on the operation result is as follows: Figure 3 As shown, it can specifically include:

[0098] S301: Obtain the pre-set standard matching threshold.

[0099] It should be noted that the standard matching threshold can be 95% to 100%.

[0100] S302: Determine whether the matching degree between the screenshot and the target interface element image falls within the standard matching threshold.

[0101] If the matching degree between the interface element and the target interface element image is within the standard matching threshold, then S303 is executed; if the matching degree between the interface element and the target interface element image is not within the standard matching threshold, then S304 is executed.

[0102] S303: It is determined that the interface element has been changed to an operable object.

[0103] S304: It is determined that the interface element is not an operable object.

[0104] The method provided in this embodiment of the invention further includes:

[0105] The target interface element image and the interface screenshot are processed into grayscale to obtain grayscale images corresponding to the target interface element image and the interface screenshot, respectively.

[0106] It should be noted that after grayscale processing of the target interface element image and the interface screenshot, the two grayscale images have the same resolution. That is, the target interface element image and the interface screenshot are grayscale processed at the same resolution.

[0107] Based on the method provided in the above embodiments, the data processing process may include the following execution:

[0108] 1. Operation Flow Compilation: Compile parameterized operation flow control tables (which can be in Excel or database tables) for the operation flow scheduling module to automatically schedule operation flows. This mainly includes whether the driving interface is a recognizable interface object with an interface ID, the determination of whether it is an object with a recognizable ID, the driving script identifier based on the interface object ID, the determination script identifier based on the interface object ID, the image identifiers captured by the interface and their access paths, the operation events corresponding to the image identifiers, and the determination states corresponding to the interface identifiers, etc.

[0109] 2. Prepare event scripts: For recognizable interface object IDs that do not require the collection of interface images, prepare automated scripts, including operation-driven scripts and decision-making scripts (for execution in points 8 and 10 below), through recording or manual compilation.

[0110] 3. Interface Image Acquisition and Processing: Using an interface screenshot tool, images are acquired to identify specific interface objects (interface elements). The original images are converted into standardized black and white pixel images to reduce storage requirements and improve computational efficiency in subsequent image fitting operations. After storing the processed images and their parameters in the interface element image library, the image identifier and access path are reflected as parameters in the flow control table.

[0111] 4. Interface element identification: For driver interfaces that can identify interface object IDs, execute point 5 below directly based on the interface object identifier in the flow control table; otherwise, obtain the interface object coordinate data required to execute point 8 through points 6 and 7 below.

[0112] 5. Event script operation: For interface events whose interface object IDs can be directly identified, the automated script can be directly called to complete the interface event submission operation.

[0113] 6. User Interface Image Acquisition: Based on the operation flow, acquire images of the current specific application window or the current screen, and perform grayscale processing on the images. This is used to identify and drive interface elements through image fitting methods.

[0114] 7. Image Fitting of Interface Elements: According to the operation flow arrangement, the corresponding driving interface element images in the interface element image library are image fitted with the interface images obtained in point 6. If the degree of fitting meets the judgment criteria, the position information of the interface element images in the driving interface is output to point 8, thus completing the interface operation simulation action.

[0115] 8. Interface Operation Simulation: Based on the interface element positioning information given in point 7, and according to the operation flow arrangement, simulate the corresponding mouse or keyboard operations to achieve flow-driven operation.

[0116] 9. Determining the Result of Interface Operation: For interfaces that can identify the interface object ID, directly execute point 10 to determine the result based on the object identifier in the parameter table; otherwise, determine the result of the event operation simulation through points 11 and 12.

[0117] 10. Result Judgment Script Operation: For result interfaces where the object ID can be directly identified, the automated script is directly called to determine whether the event operation result meets expectations based on the value of the response object on the interface.

[0118] 11. Response Interface Image Acquisition: Based on the operation flow, acquire the image of the current specific application window or the current screen, and perform grayscale processing on the image. Used for determining the operation result.

[0119] 12. Image Fitting of Response Interface Elements: According to the operation process, the corresponding response interface element images in the interface element image library are fitted with the response interface images obtained in point 11. The result of the event operation is determined based on whether the degree of fitting meets the judgment criteria.

[0120] The specific implementation processes and derivative methods of the above embodiments are all within the protection scope of this invention.

[0121] and Figure 1 Corresponding to the method described above, embodiments of the present invention also provide a data processing apparatus for processing data... Figure 1 The specific implementation of the method, the data processing device provided in this embodiment of the invention can be applied to computer terminals or various mobile devices, and its structural schematic diagram is shown below. Figure 4 As shown, it specifically includes:

[0122] The identification unit 401 is used to identify the interface elements of the target interface and obtain the preset flow control table.

[0123] The first judgment unit 402 is used to determine whether the interface element is an interface object with an identifiable interface object ID based on the process control table.

[0124] The first execution unit 403 is used to archive the target interface based on the interface element if the interface element is an interface object with an identifiable interface object ID.

[0125] The second execution unit 404 is used to perform the fitting operation corresponding to the target interface and obtain the operation result if the interface element is not an interface object with an identifiable interface object ID.

[0126] The second judgment unit 405 is used to determine whether the interface element is an operable object based on the operation result.

[0127] The third execution unit 406 is used to execute the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table if the interface element is an operable object, so as to archive the target interface.

[0128] In the apparatus provided in this embodiment of the invention, when archiving a target interface, interface elements in the target interface are identified, and a flow control table is obtained. It is determined whether an interface element is an interface object with an identifiable interface object ID. If so, the interface needs to be archived according to the operation-driven script and the judgment operation script; otherwise, a fitting operation is required. Through this fitting operation, it is determined whether the interface element is an operable object. If so, the operation-driven script and the judgment operation script are executed to archive the target interface.

[0129] The apparatus provided in this embodiment of the invention can automate processes without obtaining the interface object ID and manipulating the interface across application windows through an open API.

[0130] The apparatus provided in this embodiment of the invention further includes:

[0131] The first construction unit is used to construct the process control table;

[0132] The acquisition unit is used to acquire the process control parameters pre-uploaded by the user. The process control parameters include the interface object ID or interface element diagram corresponding to multiple interface elements to be operated.

[0133] The second construction unit is used to construct operation driving scripts for each of the identifiable interface elements included in the process control parameters based on the process control parameters.

[0134] The writing unit is used to save each of the interface element images to a preset interface element image library, and to write the interface object ID, interface element image, operation driving script and judgment operation script of each interface element to be operated into the flow control table.

[0135] In the apparatus provided in this embodiment of the invention, the second execution unit includes:

[0136] The first acquisition subunit is used to acquire the target interface element image corresponding to the interface element in the interface element image library;

[0137] The second acquisition subunit is used to acquire a screenshot of the target interface;

[0138] The fitting subunit is used to fit the interface screenshot with the target interface element image to obtain the matching degree between the interface elements and the target interface element image, and the matching degree is the operation result.

[0139] In the apparatus provided in this embodiment of the invention, the second determining unit includes:

[0140] The third acquisition subunit is used to acquire a pre-set standard matching threshold.

[0141] The judgment subunit is used to determine whether the matching degree between the interface screenshot and the target interface element image belongs to the standard matching threshold.

[0142] The first determining subunit is used to determine that the interface element is an operable object if the matching degree between the interface element and the target interface element image is within the standard matching threshold.

[0143] The second determining subunit is used to determine that the interface element is not an operable object if the matching degree between the interface element and the target interface element graph does not belong to the standard matching threshold.

[0144] The apparatus provided in this embodiment of the invention further includes:

[0145] The processing unit is used to perform grayscale processing on the target interface element image and the interface screenshot to obtain grayscale images corresponding to the target interface element image and the interface screenshot respectively.

[0146] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides controls the execution of the data processing method described above.

[0147] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the data processing method described above.

[0148] The specific working process of each unit and subunit in the data processing device disclosed in the above embodiments of the present invention can be found in the corresponding content of the data processing method disclosed in the above embodiments of the present invention, and will not be repeated here.

[0149] This invention also provides a storage medium, which includes stored instructions, wherein the execution of the instructions controls the device where the storage medium is located to perform the above-described data processing method.

[0150] This invention also provides an electronic device, the structural schematic of which is shown below. Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein one or more instructions 502 are stored in the memory 501 and configured to be executed by one or more processors 503 to perform the following operations:

[0151] Identify the interface elements of the target interface and obtain the preset flow control table;

[0152] Based on the process control table, determine whether the interface element is an interface object with a recognizable interface object ID;

[0153] If the interface element is an interface object with an identifiable interface object ID, then based on the interface element, the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table are executed to archive the target interface.

[0154] If the interface element is not an interface object with an identifiable interface object ID, then the fitting operation corresponding to the target interface is performed to obtain the operation result.

[0155] Based on the operation result, determine whether the interface element is an operable object;

[0156] If the interface element is an operable object, then the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table are executed to archive the target interface.

[0157] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0158] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both.

[0159] To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality above. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0160] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that, include: Identify the interface elements of the target interface and obtain the preset flow control table. The target interface is the monitoring screen that the user can view, and the interface elements are the operable objects in the monitoring. Based on the process control table, determine whether the interface element is an interface object with a recognizable interface object ID; If the interface element is an interface object with an identifiable interface object ID, then based on the interface element, the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table are executed to archive the target interface. The operation-driven script is used to trigger interface events and execute interface operations, and the judgment operation script is used to obtain the operation result of the operation-driven script. If the interface element is not an interface object with an identifiable interface object ID, then the fitting operation corresponding to the target interface is performed to obtain the operation result. Based on the operation result, determine whether the interface element is an operable object; If the interface element is an operable object, then the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table are executed to archive the target interface. Before identifying the interface elements of the target interface, the process also includes: Construct the process control table; Obtain the process control parameters uploaded by the user in advance. The process control parameters include the interface object ID or interface element diagram corresponding to multiple interface elements to be operated. Based on the process control parameters, construct the operation driving script and the judgment operation script corresponding to each of the interface elements to be operated, which are included in the process control parameters. Each of the interface element images is saved to a preset interface element image library, and the interface object ID, interface element image, operation driving script, and judgment operation script of each interface element to be operated are written into the process control table.

2. The method according to claim 1, characterized in that, The step of performing the fitting operation corresponding to the target interface and obtaining the operation result includes: Obtain the target interface element image corresponding to the interface element in the interface element image library; Obtain a screenshot of the target interface; The screenshot of the interface is fitted to the target interface element image to obtain the matching degree between the screenshot of the interface and the target interface element image. The matching degree is the operation result.

3. The method according to claim 2, characterized in that, The step of determining whether the interface element is an operable object based on the operation result includes: Get the pre-set standard matching threshold; Determine whether the matching degree between the screenshot and the target interface element image falls within the standard matching threshold; If the matching degree between the interface element and the target interface element image is within the standard matching threshold, then the interface element is determined to be an operable object. If the matching degree between the interface element and the target interface element image does not fall within the standard matching threshold, then the interface element is determined to be an operable object.

4. The method according to claim 2, characterized in that, Also includes: The target interface element image and the interface screenshot are processed into grayscale to obtain grayscale images corresponding to the target interface element image and the interface screenshot, respectively.

5. A data processing apparatus, characterized in that, include: The identification unit is used to identify the interface elements of the target interface and obtain a preset flow control table. The target interface is a monitoring screen that the user can view, and the interface elements are operable objects in the monitoring. The first judgment unit is used to determine whether the interface element is an interface object with an identifiable interface object ID based on the process control table. The first execution unit is configured to, if the interface element is an interface object with an identifiable interface object ID, execute the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table based on the interface element, so as to archive the target interface. The operation-driven script is used to trigger interface events and execute interface operations, and the judgment operation script is used to obtain the operation result of the operation-driven script. The second execution unit is used to perform the fitting operation corresponding to the target interface and obtain the operation result if the interface element is not an interface object with an identifiable interface object ID. The second judgment unit is used to determine whether the interface element is an operable object based on the operation result. The third execution unit is used to execute the operation-driven script and the judgment operation script corresponding to the target interface in the flow control table if the interface element is an operable object, so as to archive the target interface. The first construction unit is used to construct the process control table; The acquisition unit is used to acquire the process control parameters pre-uploaded by the user. The process control parameters include the interface object ID or interface element diagram corresponding to multiple interface elements to be operated. The second construction unit is used to construct operation driving scripts for each of the identifiable interface elements included in the process control parameters based on the process control parameters. The writing unit is used to save each of the interface element images to a preset interface element image library, and to write the interface object ID, interface element image, operation driving script and judgment operation script of each interface element to be operated into the flow control table.

6. The apparatus according to claim 5, characterized in that, The second execution unit includes: The first acquisition subunit is used to acquire the target interface element image corresponding to the interface element in the interface element image library; The second acquisition subunit is used to acquire a screenshot of the target interface; The fitting subunit is used to fit the interface screenshot with the target interface element image to obtain the matching degree between the interface screenshot and the target interface element image, and the matching degree is the operation result.

7. The apparatus according to claim 6, characterized in that, The second judgment unit includes: The third acquisition subunit is used to acquire a pre-set standard matching threshold. The judgment subunit is used to determine whether the matching degree between the interface screenshot and the target interface element image belongs to the standard matching threshold. The first determining subunit is used to determine that the interface element is an operable object if the matching degree between the interface element and the target interface element image is within the standard matching threshold. The second determining subunit is used to determine that the interface element is not an operable object if the matching degree between the interface element and the target interface element graph does not belong to the standard matching threshold.

8. The apparatus according to claim 6, characterized in that, Also includes: The processing unit is used to perform grayscale processing on the target interface element image and the interface screenshot to obtain grayscale images corresponding to the target interface element image and the interface screenshot respectively.

Images