Method and device for automatically testing user interface

Abstract

The invention discloses a user interface automatic testing method and device.The method comprises the steps that operation elements in a user interface involved in testing are modified from initial styles to corresponding custom styles, and the custom styles are used for highlighting the operation elements so that the operation elements can be rapidly recognized; after each test action in the test script is executed, performing screenshot on the user interface, generating annotation information according to the operation type, the operation element identifier, the operation parameter and the expected result of the current test action, and adding the annotation information to the screenshot; splicing the screenshots corresponding to the test actions into a video according to the sequence of the test actions in the test script; according to the video, a test result is confirmed, and the test result comprises whether the test is successful or not and the position where a problem occurs when the problem occurs in the test. According to the scheme, the problem positioning efficiency and accuracy in the automatic test of the user interface are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of automated user interface testing, and more specifically, to a method and apparatus for automated user interface testing. Background Technology

[0002] Currently, user interface automated testing typically relies on the execution of automated scripts to verify whether the interface functions as expected. An automated script consists of multiple actions executed sequentially, each generally corresponding to a specific HTML element and its operation type, such as a click, input, or assertion. By automating these operations through scripts, a large number of interface functionalities can be verified in a shorter time, thus significantly improving testing efficiency.

[0003] However, when UI automation scripts encounter problems during execution, testers often can only determine the cause of the failure by analyzing log information or viewing static error screenshots. This approach has significant shortcomings: log content is abstract and lacks intuitiveness, requiring testers to infer the problem based on line numbers and output information, resulting in low localization efficiency; static screenshots can only reflect the state at a specific moment when the error occurs, failing to demonstrate the preceding and following processes and the order of operations.

[0004] In real-world, complex business scenarios, such as corporate lending systems with numerous page interactions and lengthy workflows, problem localization becomes even more challenging. Testing often spans multiple pages and input stages, and relying solely on logs or static screenshots makes it difficult to quickly reconstruct the specific steps and context in which the problem occurred. Typically, manual replaying of the script and comparison of execution results are required to locate the anomaly, which is time-consuming and error-prone.

[0005] Therefore, improving the efficiency and accuracy of problem localization in automated user interface testing has become a technical problem that needs to be solved by existing technologies. Summary of the Invention

[0006] In order to solve at least one of the technical problems in the background art, the present invention proposes an automated testing method and apparatus for user interfaces.

[0007] One aspect of the present invention provides an automated testing method for user interfaces, the method comprising:

[0008] Modify the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, wherein the custom style is used to highlight the operation elements for quick identification;

[0009] After each test action in the test script is executed, a screenshot of the user interface is taken, and annotation information is generated based on the operation type, operation element identifier, operation parameters and expected results of the current test action, and the annotation information is added to the screenshot.

[0010] According to the order of the test actions in the test script, the screenshots corresponding to each test action are stitched together into a video;

[0011] The test results are confirmed based on the video, including whether the test was successful and, if any, the location of the problem when the test failed.

[0012] Optionally, confirming the test results based on the video includes:

[0013] The video and the test rule set are input into a large artificial intelligence model with multimodal understanding capabilities. The large artificial intelligence model performs semantic reasoning based on the video frame content, inter-frame temporal relationship and test rule set, and outputs the execution effect judgment result of each test action. The execution effect judgment result includes the achievement status, achievement probability, anomaly category and corresponding anomaly frame position of each test action.

[0014] The test is considered successful if all test actions in the execution effect judgment result are in the achieved state; otherwise, the test is considered to have failed and an exception report is generated.

[0015] Optionally, modifying the operational elements in the user interface involved in the test from the initial style to the corresponding custom style includes:

[0016] Before the test script runs, the front-end style of the operation elements in the user interface involved in the test script is modified from the initial style to the corresponding custom style;

[0017] The method further includes:

[0018] After the test script finishes running, the front-end style of the user interface elements involved in the test script will be modified back to their corresponding initial style.

[0019] Optionally, modifying the operational elements in the user interface involved in the test from the initial style to the corresponding custom style includes:

[0020] Before each test action in the test script is executed, the front-end style of the operation element corresponding to that test action in the user interface is changed from the initial style to the corresponding custom style.

[0021] The method further includes:

[0022] After the test action is completed and a screenshot is taken, the front-end style of the operation element corresponding to the test action in the user interface is changed back to the corresponding initial style.

[0023] Optionally, after each test action in the test script is executed, a screenshot of the user interface is taken, and annotation information is generated based on the operation type, operation element identifier, operation parameters, and expected result of the current test action, including:

[0024] For multiple consecutive test actions targeting the same operation element on the user interface, after the last test action in the series is executed, a screenshot of the user interface is taken, and unified annotation information is generated based on the operation type, operation element identifier, operation parameters, and expected results of the series of test actions.

[0025] Optionally, confirming the test results based on the video includes:

[0026] The video is sent to the testers so that they can confirm whether the test was successful based on the custom style and annotation information displayed in the video, and identify the location of the problem if an issue occurs during the test.

[0027] Optionally, the custom style includes at least one of the following: border highlighting, background color change, animation effect, font bolding, and font color change.

[0028] In another aspect, the present invention provides an automated user interface testing apparatus, the apparatus comprising:

[0029] The style modification unit is used to modify the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, wherein the custom style is used to highlight the operation elements for quick identification;

[0030] The screenshot processing unit is used to take a screenshot of the user interface after each test action in the test script is executed, generate annotation information based on the operation type, operation element identifier, operation parameters and expected results of the current test action, and add the annotation information to the screenshot.

[0031] The video generation unit is used to stitch together the screenshots corresponding to each test action into a video according to the order of each test action in the test script.

[0032] The test result determination unit is used to confirm the test result based on the video, wherein the test result includes whether the test was successful and, if a problem occurs during the test, the location of the problem.

[0033] To achieve the above objectives, according to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described user interface automated testing method.

[0034] To achieve the above objectives, according to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program / instructions are stored, which, when executed by a processor, implement the steps of the above-described user interface automated testing method.

[0035] To achieve the above objectives, according to another aspect of the present invention, a computer program product is also provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the above-described user interface automated testing method.

[0036] The beneficial effects of this invention are as follows:

[0037] This invention achieves a visual record of the entire test execution process by customizing the styles of user interface elements during testing, generating screenshots with annotations after each test action, and stitching these screenshots together in sequence into a video. By adding annotations to the screenshots containing the operation type, operation element identifier, operation parameters, and expected results, the execution content of each test action corresponds one-to-one with the interface changes. This allows for intuitive confirmation of test success through the generated video after the test, and accurate location of problems when they occur. Therefore, this invention effectively improves the efficiency and accuracy of problem localization in automated user interface testing. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0039] Figure 1 This is a flowchart of the user interface automated testing method according to an embodiment of the present invention;

[0040] Figure 2 This is a flowchart illustrating the process of confirming test results based on video according to an embodiment of the present invention;

[0041] Figure 3 This is a flowchart of the overall processing of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the generation and processing of annotation information according to an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram illustrating problem localization in an embodiment of the present invention;

[0044] Figure 6This is a structural block diagram of the user interface automated testing device according to an embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0048] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] It should be noted that the information collected in the technical solution of this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation portals are provided for users to choose to authorize or refuse.

[0051] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0052] It should be noted that the technical solution of this application provides users with corresponding operation entry points for users to choose to agree to or reject the automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0053] It should be noted that the user interface automated testing method and apparatus of the present invention can be used in the financial field, or in any field other than the financial field. The application field of the user interface automated testing method and apparatus of the present invention is not limited.

[0054] Figure 1 This is a flowchart of the user interface automated testing method according to an embodiment of the present invention, such as... Figure 1 As shown, in one embodiment of the present invention, the user interface automated testing method of the present invention includes steps S101 to S104.

[0055] Step S101: Modify the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, wherein the custom style is used to highlight the operation elements for quick identification.

[0056] In one embodiment of the present invention, the custom style includes at least one of the following: border highlighting, background color change, animation effect, font bolding, and font color changing, making the target element clearly distinguishable from other interface elements during test execution. The system can automatically apply the corresponding styles to the interface according to a preset style configuration table during the test script loading phase. In this way, the test script can identify the target element more quickly during operation, reducing identification errors caused by interface complexity and a large number of elements, and improving the accuracy and stability of test actions.

[0057] In some embodiments, modifications to custom styles can be achieved through script injection or DOM attribute replacement. After modification, the original functional logic of the page is not affected, and the elements are only visually identified.

[0058] Step S102: After each test action in the test script is executed, a screenshot of the user interface is taken, and annotation information is generated based on the operation type, operation element identifier, operation parameters and expected results of the current test action, and the annotation information is added to the screenshot.

[0059] In one embodiment of the present invention, the system automatically captures the current interface image after each test action is completed to record the interface state after the operation is executed. The annotation information is generated by the test control module based on the attributes of the current action, such as operation type (click, input, selection, etc.), operation element identifier (button name, input box number, etc.), operation parameters (input content or operation value), and the corresponding expected result.

[0060] In one embodiment of the present invention, after generating a screenshot, the system overlays annotation information as text boxes or graphic icons onto the corresponding area of ​​the screenshot, typically next to the operated element or at the edge of the screenshot. This visual annotation method allows each screenshot to reflect interface changes and intuitively explain the current action, providing a clear basis for subsequent problem analysis.

[0061] Step S103: According to the order of the test actions in the test script, stitch the screenshots corresponding to each test action into a video.

[0062] In one embodiment of the present invention, in this step, the system stitches together the multiple screenshots generated in step S102 according to the execution time sequence of the test actions to generate a complete video file. Each frame in the video corresponds to a screenshot of a test action, thereby intuitively displaying the entire execution process of the test script from start to finish.

[0063] In one embodiment of the invention, the video generation module can also add a timestamp or action number to each frame, facilitating quick location of the corresponding test step during video playback. When the test script is long, the system can also compress the screenshots during the video generation stage to reduce file size while maintaining necessary visual clarity. By converting the screenshot sequence into video format, testers can quickly review the test process after the test is completed without comparing screenshots one by one, significantly improving the efficiency of problem reproduction and analysis.

[0064] Step S104: Confirm the test results based on the video, wherein the test results include whether the test was successful and, if a problem occurred during the test, the location of the problem.

[0065] like Figure 5 As shown, in one embodiment of the present invention, testers can intuitively judge whether the execution of each test action has achieved the expected result by replaying the generated video. When an action in the test script is executed abnormally or the interface result does not match the expectation, the abnormal action step and the corresponding interface position can be located directly based on the annotation information superimposed in the video frame.

[0066] In some embodiments, the system can automatically record the frame number where the anomaly occurred by combining a timestamp or action number, and generate a corresponding test report. The report includes a screenshot of the anomaly frame, action information, and a problem description. This method can effectively reduce the workload of manually searching for problems and improve the efficiency and accuracy of problem localization in automated user interface testing.

[0067] Therefore, it can be seen that through the synergistic effect of the above steps S101 to S104, the present invention realizes a complete process from test execution, information annotation to video display and problem confirmation, which can significantly improve the visualization of user interface automated testing and the efficiency of problem analysis.

[0068] In one embodiment of the present invention, the present invention can use an artificial intelligence large model to automatically confirm the test results based on the video.

[0069] like Figure 2 As shown, in one embodiment of the present invention, the above step S104 of confirming the test result based on the video includes steps S201 and S202.

[0070] Step S201: Input the video and the test rule set into an AI big model with multimodal understanding capabilities. The AI ​​big model performs semantic reasoning based on the video frame content, inter-frame temporal relationship and test rule set, and outputs the execution effect judgment result of each test action. The execution effect judgment result includes the achievement status, achievement probability, anomaly category and corresponding anomaly frame position of each test action.

[0071] In one embodiment of the present invention, the system first performs frame-level decoding on the video generated in step S103, synchronizing the consecutive frames with the action sequence in the test script. Then, the synchronized video frame data and the corresponding test rule set are input into the large-scale artificial intelligence model. The test rule set typically includes information such as the target element, operation type, operation parameters, and expected result for each test action. The large-scale model can simultaneously understand image, text, and temporal information, utilizing its multimodal semantic reasoning capabilities to analyze whether the changes between video frames are consistent with the action descriptions.

[0072] The model generates a corresponding execution effect judgment result for each test action. The achievement status indicates whether the action was successfully completed, the achievement probability represents the model's confidence in the judgment, and the exception category distinguishes different types of execution exceptions, such as "no response," "result not updated," or "interface display error." The corresponding exception frame position indicates the time point in the video where the exception occurred. In this way, the system can automatically identify the actual execution status of test actions from the video content, providing a basis for subsequent test result judgment.

[0073] Step S202: When all test actions in the execution effect determination result are in the achieved state, the test is determined to be successful; otherwise, the test is determined to be unsuccessful and an exception report is generated.

[0074] In one embodiment of the present invention, the system statistically analyzes and judges all execution effect judgment results output by the large artificial intelligence model. When all actions are successful, the system automatically outputs a test pass result. If one or more actions are unsuccessful or have a success probability lower than a set threshold, the test script is considered to have executed abnormally.

[0075] When an anomaly occurs, the system automatically extracts relevant screenshots and frames based on the anomaly category and corresponding frame location output by the large-scale artificial intelligence model, generating an anomaly report. The anomaly report may include the anomaly type, the time and location of the problem, the corresponding screenshot, and possible causes analyzed by the model. Through this automated anomaly identification and report generation mechanism, testers can quickly locate the action steps and interface locations where problems occur without manually comparing or replaying the entire test video, thus significantly improving the efficiency and accuracy of problem localization in automated user interface testing.

[0076] Through the above steps S201 and S202, this invention achieves the function of automatically analyzing test videos and determining test results using a large artificial intelligence model, enabling the test system to automatically complete result identification and anomaly report generation without human intervention, further improving the intelligence and automation level of the testing process.

[0077] In one embodiment of the present invention, in step S201 above, the large artificial intelligence model performs semantic reasoning based on video frame content, inter-frame temporal relationships, and test rule sets, and outputs the execution effect judgment results of each test action, including the following process:

[0078] First, the video is decoded frame by frame, and keyframes are extracted. Keyframes can be determined based on the trigger time of the test action or the magnitude of interface changes to ensure that the selected frames represent changes in the interface state. Then, combined with the timestamp information recorded in the test script, the continuous frames are divided into multiple time segments, each time segment corresponding to the execution interval of a test action.

[0079] Next, the system automatically generates natural language prompts corresponding to each test action based on the test rule set. These prompts can be composed of the operation type, target operation element identifier, expected interaction result, and judgment criteria, and are used to guide the large-scale artificial intelligence model in semantic analysis of the video content.

[0080] Subsequently, the keyframe sequences corresponding to each test action and their natural language prompts are input into the large-scale artificial intelligence model. The model outputs the success probability and anomaly category for each test action through multimodal reasoning of video images, action semantics, and inter-frame changes. The success probability is used to characterize the model's confidence in the success of the action, and the anomaly category is used to distinguish different types of execution anomalies, such as unresponsive interface, incorrect result display, or outdated elements.

[0081] When the probability of achieving a certain test action is lower than a preset threshold, the system identifies the last frame of the corresponding time segment as an abnormal frame. At this point, the system can determine the salient region based on the attention weight distribution or visual feature response map of the large-scale artificial intelligence model, thereby pinpointing the specific location of the problem within the interface. Through this process, the system ultimately generates an execution effect judgment result for each test action, providing data support for subsequent test result determination.

[0082] In one embodiment of the present invention, step S101, which modifies the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, includes:

[0083] Before the test script runs, the front-end style of the operation elements in the user interface involved in the test script is modified from the initial style to the corresponding custom style;

[0084] In one embodiment of the present invention, the method further includes:

[0085] After the test script finishes running, the front-end style of the user interface elements involved in the test script will be modified back to their corresponding initial style.

[0086] In one embodiment of the present invention, when loading the test script, the system automatically scans all interface element identifiers referenced in the script and performs batch replacement of the front-end style attributes of these elements before the interface is rendered. Specifically, style parameters associated with the target element, such as border color, background color, transparency, or font style, can be modified on the front-end stylesheet (CSS file) or page DOM node to make the element visually more prominent.

[0087] Custom styles can be defined through configuration files or style templates. Different operational elements can correspond to different style schemes to distinguish their type or operational nature. For example, input box elements can have highlighted borders, button elements can have a blinking background effect, and menu item elements can have text color variations. By applying these custom styles once before the test script runs, the consistency and stability of element recognition can be maintained throughout the entire test execution process, avoiding recognition deviations caused by style changes or theme switching during testing, thereby improving the reliability of test execution.

[0088] In some embodiments, the aforementioned style replacement can be achieved through script injection. For example, in a browser or front-end framework environment, the style attributes of elements can be dynamically modified by executing a style injection script without altering the original page code structure. This ensures the independence of test execution without affecting the normal business logic of the system under test.

[0089] In one embodiment of the present invention, after the test is completed, the system automatically calls the style restoration module to replace the custom styles of each operation element back to the original styles according to the pre-recorded style mapping table. This style mapping table can be automatically generated when the custom style is applied, recording the initial style attributes and modified style attributes of each element for accurate matching during restoration.

[0090] By restoring the initial style after testing, custom styles can be left in the interface to avoid interfering with subsequent tests or manual operations. For example, when multiple automated scripts are executed consecutively on the same system, if the initial style is not restored, subsequent scripts may misidentify modified style elements, resulting in test errors. This style restoration process can be automatically triggered at the end of the test script, requiring no manual intervention and ensuring that the test environment is in a standard state after each execution.

[0091] In some embodiments, the recovery operation can be achieved by undoing the style injection script or reloading the original stylesheet. For single-page applications (SPAs), a global hook function can also be set in the test framework to automatically clear the custom style cache after the script execution is complete. This ensures that the system interface maintains a consistent visual and structural state before and after testing, improving the repeatability and stability of automated testing.

[0092] By applying custom styles to target elements before test execution and restoring the initial styles after test, the test system can more accurately identify the operation elements without affecting the normal display logic of the system. At the same time, it ensures the consistency of the test environment and provides a stable foundation for subsequent repeated tests or manual verification.

[0093] In another embodiment of the present invention, step S101, which modifies the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, includes:

[0094] Before each test action in the test script is executed, the front-end style of the operation element corresponding to that test action in the user interface is changed from the initial style to the corresponding custom style.

[0095] In one embodiment of the present invention, the method further includes:

[0096] After the test action is completed and a screenshot is taken, the front-end style of the operation element corresponding to the test action in the user interface is changed back to the corresponding initial style.

[0097] In one embodiment of the present invention, the system dynamically applies custom styles during the execution of the test script. That is, before each test action is executed, the front-end style of the target element to be operated on by the current action is temporarily modified. The system can determine the interface element to be modified by reading the action definition information (such as operation type and target element identifier) ​​in the test script, and call the style modification interface before executing the action to modify its style attributes accordingly.

[0098] Custom styles can include border highlighting, background color changes, bold font, or animated blinking, used to visually highlight the target element in the interface before the action is executed. This allows the automation engine or visual recognition module to locate the element more quickly and accurately, avoiding element recognition delays or misjudgments.

[0099] In some embodiments, custom styles can be applied by modifying the style properties of page DOM nodes. For example, the system can dynamically modify the CSS stylesheet of an element via script injection before the action is executed, and immediately execute the corresponding click, input, or validation operations after the modification is complete. Compared with modifying the styles of all elements at once before the test begins, this approach only temporarily modifies the elements related to the current action, which can reduce the scope of interface style changes and reduce the risk of other interface elements being affected during test execution.

[0100] In one embodiment of the present invention, after the current test action is completed and the corresponding interface screenshot is taken, the system immediately triggers a style restoration operation to restore the custom style of the element to its initial style before the test execution. This restoration process can be achieved by reapplying the original stylesheet, undoing script-injected styles, or reading the style mapping cache to restore the original attribute values. The style restoration operation is typically performed after the screenshot is saved to ensure that the screenshot retains the visual highlighting effect of the action execution while not affecting the element recognition of subsequent actions. By restoring styles promptly after each action, it is ensured that the interface always remains close to its real state, avoiding interference or misjudgment caused by style residue between consecutive actions.

[0101] In some embodiments, the system can set a style restoration hook in the test engine, which automatically calls the restoration function when an action execution and screenshot completion event are detected. In this way, each action execution cycle can form a complete closed-loop process of "applying style - performing operation - screenshotting - restoring style", ensuring the readability of the test screen and the stability of the process.

[0102] Through the aforementioned dynamic style application and recovery mechanism, this embodiment can progressively highlight and identify the target elements of each test action without altering the overall page structure. This approach improves the recognition accuracy of single-step operations while reducing the impact on other interface elements, thereby enhancing the controllability of automated user interface testing and the accuracy of problem localization.

[0103] In one embodiment of the present invention, after each test action in the test script is executed in step S102 above, a screenshot of the user interface is taken, and annotation information is generated based on the operation type, operation element identifier, operation parameters, and expected result of the current test action, including:

[0104] For multiple consecutive test actions targeting the same operation element on the user interface, after the last test action in the series is executed, a screenshot of the user interface is taken, and unified annotation information is generated based on the operation type, operation element identifier, operation parameters, and expected results of the series of test actions.

[0105] like Figure 4 As shown, in one embodiment of the present invention, the system detects in real time whether consecutive actions are applied to the same operation element when executing the test script. When multiple consecutive actions are detected to be targeting the same interface element (e.g., the same input box or button), the system does not immediately take a screenshot of the first few actions, but waits until the last related action is completed before taking a screenshot. At this time, the element on the interface is in its final state, so the overall execution effect of this series of operations can be reflected by a single screenshot.

[0106] When generating unified annotation information, the system integrates the operation types (such as click, input, confirmation, etc.), operation parameters (such as input values ​​or selections), and corresponding expected results of these consecutive actions into a set of text descriptions. For example, when the three consecutive actions are "click input box A", "enter xx value", and "verify the input result is correct", the system will generate unified annotation information in the screenshot, such as "enter xx in input box A and verify the input result".

[0107] This method avoids repeatedly taking screenshots and annotating the same interface elements, reducing the number of generated video frames and improving the coherence and readability of test records. At the same time, consistent annotation information aligns better with user understanding, making the test process clearer and more intuitive during playback or problem localization.

[0108] In some embodiments, the system can determine whether a group of actions constitutes "multiple consecutive test actions" based on element identifiers and action time intervals. When the interval between consecutive actions is less than a preset threshold and the target element identifiers are the same, it is determined to be a group of consecutive operations. After the last action is completed and a screenshot is taken, the system will automatically clear the cache records of the operation group to continue monitoring the next group of actions.

[0109] Through the above processing, this embodiment reduces redundant screenshots and annotation operations while ensuring the integrity of test information, making the generated video recording more concise, while maintaining the complete expression of operation semantics, thereby further improving the efficiency of automated user interface testing and the accuracy of result analysis.

[0110] In one embodiment of the present invention, step S104, confirming the test result based on the video, includes:

[0111] The video is sent to the testers so that they can confirm whether the test was successful based on the custom style and annotation information displayed in the video, and identify the location of the problem if an issue occurs during the test.

[0112] In this embodiment, after the system generates the video, it automatically pushes the video file to the test management platform or the tester's terminal. Testers can visually view the entire execution process of the test script through the video playback interface. Each frame in the video corresponds to the execution result of a test action, and is overlaid with annotation information for that action, including the operation type, operation element identifier, operation parameters, and expected result. Simultaneously, the customizable style in the interface highlights the target element being operated on in the video frame, allowing testers to quickly identify the interface location corresponding to the test operation.

[0113] During manual verification, testers can compare the annotation information with the interface changes by playing back frame by frame or jumping by action number to determine whether each test action has achieved the expected results. When the display state of a certain frame is found to be inconsistent with expectations, the test can be determined to have failed, and the location of the problem can be accurately located based on the annotation information superimposed on the video, such as the specific operation element or action step.

[0114] In one embodiment of the invention, the system's video playback interface can also provide timeline markers and action indexes. Testers can directly jump to the corresponding action's screen location by clicking the corresponding index item for quick problem verification. This video-based manual verification method can intuitively present the operation steps, interface changes, and expected results during the testing process to the testers, making problem analysis more efficient and accurate.

[0115] In this way, no additional debugging tools or log analysis are required. Testers can use the generated video files to manually confirm the test results, which significantly improves the efficiency and visualization of problem localization in automated user interface testing.

[0116] In one embodiment of the present invention, the automation script consists of a series of actions, and each action is composed of elements, operation type, and operation data. Multiple actions constitute the automation script. Figure 3 As shown, in one embodiment of the present invention, the overall processing flow of the present invention includes the following steps:

[0117] 1. During automated execution, in order to quickly identify elements on the interface, add custom styles to the elements. These styles can be static CSS rules or dynamically generated according to the specific situation of each execution.

[0118] 2. When the automated script is executed, the custom style added in step 1 can be quickly recognized by the interface. The system will automatically generate a screenshot for each operation and add relevant descriptive information to the screenshot.

[0119] 3. Generate video file: Organize all screenshots into a video according to the operation sequence, and add corresponding descriptions and annotations to each frame. Each action and screenshot in the video can reflect the execution sequence and operation details of the automated script;

[0120] 4. Problem Localization and Analysis: Testers play the video and, by combining the descriptive information in the video with the state of UI elements, quickly locate the cause and location of the problem. By replaying the video and reviewing the annotations, the time cost of traditional debugging methods can be reduced.

[0121] The following sections will provide a detailed introduction to each of these four parts.

[0122] 1. Implementation of custom styles for user interface elements:

[0123] In traditional user interface automation testing, test scripts typically only execute commands such as clicks, inputs, and toggles, with results usually recorded as logs or single-frame screenshots. Due to the lack of sufficient visual context information, testers often need to repeatedly compare interface states when analyzing and locating problems, resulting in low efficiency.

[0124] To address this, the present invention introduces a custom style mechanism in the automated testing process. This mechanism dynamically applies special visual identifiers to the user interface elements during test execution, enabling each element to be quickly identified when performing its corresponding test action. In this way, clear visual markers can be provided for the test execution process without altering the interface's functional logic, improving the efficiency and accuracy of problem localization.

[0125] The custom style may include, but is not limited to, the following forms:

[0126] (1) Border highlighting: Add a distinct border color to the target operation element (such as button, input box, menu item, etc.), such as red, green or blue, so that the element is highlighted in the interface.

[0127] (2) Background color change: The background color of the operation element is adjusted to enhance the visual contrast. High saturation colors such as yellow, purple or orange are often used so that they can be intuitively identified during the test.

[0128] (3) Animation effects: Simple animation effects can be set for key operation elements, such as blinking, scaling or border color changing in a loop, to indicate the focus of the current action.

[0129] (4) Bold or change the color of the font: For operation elements with text content (such as button labels, prompts or input box titles), the font weight or color can be adjusted to make them more eye-catching in the test interface.

[0130] The aforementioned custom styles can be implemented through preset style configuration tables (CSS files), or dynamically generated by the system during test execution based on the test scenario. Before the test script runs, the system automatically loads or injects these custom styles and applies them to the corresponding target operation elements, so that the relevant interface elements are presented in a prominent form at each step of the test script execution, thereby achieving rapid identification and visual tracking.

[0131] By introducing this custom style mechanism, this invention not only improves the visualization of the testing system during execution, but also provides richer interface context information during the problem analysis phase, enabling testers to more efficiently identify abnormal elements and analyze interface state changes, thereby improving the efficiency and accuracy of problem localization in automated user interface testing.

[0132] 2. Generation and processing of screenshot and annotation information:

[0133] In one embodiment of the present invention, during the automated testing of the user interface, one or more screenshots are generated for each test action to record the state of the interface after the action is executed. These screenshots not only reflect the current state of the elements in the user interface but also overlay annotation information related to the action, facilitating the visualization of test results and rapid identification of subsequent problems.

[0134] (1) Screenshot generation:

[0135] During the execution of the test script, each test action corresponds to a specific interface operation, such as clicking, inputting, selecting, or scrolling. After each test action is completed, the system automatically takes a screenshot of the current interface to record the display state after the operation. This screenshot generation process is triggered by the test control module and requires no manual intervention. It can be executed after the interface rendering is complete or the operation callback event is triggered to ensure the integrity of the interface state in the screenshot.

[0136] (2) Generation and association of annotation information:

[0137] For each screenshot, the system generates corresponding annotation information based on the current test action. This annotation information includes, but is not limited to:

[0138] Operation element identifier: Used to uniquely identify the interface element being operated on, such as element ID, CSS selector, or element text attribute;

[0139] Operation type: Indicates the current operation being performed, such as click, input, scroll, hover, etc.

[0140] Operation parameters: When the operation involves parameter input, record the input content or selection information;

[0141] Expected result: This indicates the ideal state the interface should present after the operation is performed, such as button color change, input box content update, etc.

[0142] After a screenshot is generated, the system binds the above annotation information to the corresponding screenshot and overlays visual text labels on it. Annotations are typically displayed as short text boxes or labels on the edge of the screenshot or near the corresponding action element. The content should be concise and clear, such as: "Click button A," "Enter text 'Hello World'," "Verify input result is correct," etc. Through this annotation method, the screenshot not only records the interface state but also intuitively reflects the action intent and its expected effect.

[0143] (3) Screenshot storage and organization:

[0144] The generated screenshots and their corresponding annotations are stored uniformly in the test results database or file system. The system can automatically generate filenames and paths based on the execution order of the test scripts, timestamps, or test action numbers, ensuring the traceability and structured management of screenshots. Through this organization, testers can quickly retrieve screenshots and annotations corresponding to any action for problem analysis and result verification.

[0145] (4) Screenshot rules for continuous actions:

[0146] In some embodiments, when there are multiple consecutive test actions targeting the same operation element in the test script, the system can optimize the screenshot generation process according to rules. Specifically, when multiple consecutive test actions are detected to act on the same operation element, the system only takes a screenshot after the last test action in the consecutive action group has been executed. At this time, the state of the operation element in the interface has reached the final result, so the screenshot can reflect the overall effect of the entire consecutive operation process.

[0147] For the aforementioned continuous action group, the system will integrate the operation type, operation parameters, and expected results of all actions within the group into unified annotation information, and present it in a comprehensive description in the screenshot. For example, when the test script includes step 1 "Click input box A", step 2 "Enter text XX", and step 3 "Verify that the content of the input box is correct", the system will generate a screenshot after the last step is executed, and add unified annotation information to the screenshot: "Enter text XX in input box A and verify that the input result is correct", as shown in Figure 4.

[0148] By using this continuous action merging screenshot mechanism, the system can reduce the number of duplicate screenshots and redundant annotations generated, making the generated video or screenshot sequence more concise, while retaining complete operational semantics, thus improving the readability and analysis efficiency of test records.

[0149] 3. Video generation and display:

[0150] In one embodiment of the present invention, in order to more intuitively demonstrate the execution process of automated testing, the step S103 above, "stitching the screenshots corresponding to each test action into a video", specifically includes the process of video generation and display.

[0151] (1) Assembling the screenshots in sequence:

[0152] After the test script is executed, the system sorts the screenshots generated in step S102 according to the execution order of the test actions in the test script. Each screenshot corresponds to a test action, and the system arranges the screenshots chronologically based on the timestamp or execution sequence number of the test action. The inter-frame interval between adjacent screenshots can be calculated based on the actual execution time of the test script to accurately reconstruct the timing relationship of the test process in the generated video.

[0153] In some embodiments, the system can appropriately compress images during the screenshot assembly stage to reduce the size of the generated video while maintaining the visual clarity of the screenshots. In this way, each frame of the video corresponds to the execution result of a specific test action, enabling the entire testing process to be presented continuously and visually in the time dimension.

[0154] (2) Display of video footage and annotation information:

[0155] During video generation, the system synchronously embeds the annotation information from each screenshot into the video frames. Each frame displays the user interface state after the test action is executed, with text annotations superimposed above or below the screen to describe the test action content corresponding to the current frame. The annotation information includes operation type, operation element identifier, operation parameters, and expected results, enabling testers to intuitively understand the execution semantics of each action while watching the video.

[0156] While playing the video, testers can directly observe the execution result of each action and its specific manifestation on the interface. When an action malfunctions or the result does not meet expectations, the corresponding problematic steps and interface elements can be quickly located through the annotation information, thereby significantly improving the efficiency of problem analysis.

[0157] (3) Video format and display method:

[0158] The system generates video files in common formats such as MP4 and AVI, allowing them to be played in general-purpose media players or test management platforms. The videos retain a timeline of the test execution, and testers can use the playback controls to pause, fast forward, or rewind to view and analyze specific test actions.

[0159] In some embodiments, the system can also add a timeline and action number information to the video, enabling testers to quickly jump to the corresponding screen position of the target action using an index. The generated video file can be directly embedded in the test report or accessed remotely through the test management platform, thereby achieving the sharing and visualization of test results.

[0160] Through the aforementioned video generation and display process, this invention can transform originally discrete screenshot information into continuous video recordings, making the automated testing process more intuitive and easier to understand. Testers can quickly grasp the test execution sequence and interface changes through video playback, thereby significantly improving the efficiency of problem localization and analysis in automated user interface testing.

[0161] 4. Problem identification:

[0162] In one embodiment of the present invention, the video generated through the above steps allows testers to intuitively view the execution process of the test script and quickly locate the problem.

[0163] During the test results analysis phase, testers can directly play the generated video files and view the interface changes corresponding to each test action. Each frame of the video shows the user interface state after the test action is executed, along with overlaid annotation information related to that action, including the operation type, operation element identifier, operation parameters, and expected results. Testers can use this information to clearly understand whether the execution effect of each action matches the expectations.

[0164] When the interface display after a certain operation does not match the expected result in the annotation information, testers can pause the video and conduct a detailed analysis at that frame location. The annotation information provided by the system helps testers quickly identify the interface location where the problem occurred and the corresponding operation element. For example, after executing "click button A", if the button state on the interface does not change or the expected pop-up does not appear, testers can combine screenshots and text annotations in that frame to determine the source of the problem and further verify the relevant script logic.

[0165] In this way, the present invention transforms the traditional automated testing process, which relies on logs or discrete screenshots for problem analysis, into an intuitive problem localization process based on video visualization. Testers can quickly browse the entire testing process in a single video file and directly locate specific frames and actions when anomalies are found, thereby significantly improving the efficiency and accuracy of problem localization. Figure 5 illustrates a video-based test problem localization method.

[0166] As can be seen from the above embodiments, the present invention achieves full-process visualization of the testing process and rapid problem localization by introducing custom styles, screenshot annotations and video generation technology into the user interface automated testing process.

[0167] First, by applying custom styles to the elements being manipulated during the test execution phase, the target elements are highlighted in the interface. This allows the test system to quickly and accurately identify the target elements during execution, ensuring the accuracy of each test action and providing clear visual guidance for subsequent problem localization.

[0168] Secondly, after each test action is executed, the system automatically generates a screenshot containing annotation information. This annotation information records the operation type, operation element identifier, operation parameters, and expected results, ensuring that the execution process of each action is recorded in a structured manner. This screenshot not only demonstrates the interface changes but also supplements the semantic information of the operation with text annotations, making it easier for testers to directly understand the purpose and effect of each step during analysis.

[0169] Furthermore, the system stitches all screenshots together in the execution order of the test script to generate a video, presenting the entire automated testing process in a coherent and visual manner. Each frame in the video corresponds to a test action, and with the addition of annotation information and customizable display styles, testers can intuitively view the test execution order and interface changes without having to consult logs or compare screenshots one by one.

[0170] In summary, this invention achieves the visualization and intelligent presentation of automated test results through the synergistic effects of custom styles, screenshot annotations, and video generation. This solution significantly improves the efficiency of problem localization in automated user interface testing, enabling testers to determine the location and cause of problems in a shorter time. Simultaneously, it reduces manual intervention, lowers the workload of test analysis, and overall improves the execution efficiency of automated testing and the accuracy of result analysis.

[0171] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0172] Based on the same inventive concept, embodiments of the present invention also provide an automated user interface testing apparatus, which can be used to implement the automated user interface testing method described in the above embodiments, as described in the following embodiments. Since the principle by which the automated user interface testing apparatus solves the problem is similar to that of the automated user interface testing method, embodiments of the automated user interface testing apparatus can be found in embodiments of the automated user interface testing method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0173] Figure 6 This is a structural block diagram of the user interface automated testing device according to an embodiment of the present invention, such as... Figure 6 As shown, in one embodiment of the present invention, the user interface automated testing device of the present invention includes:

[0174] Style modification unit 1 is used to modify the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, wherein the custom style is used to highlight the operation elements for quick identification;

[0175] The screenshot processing unit 2 is used to take a screenshot of the user interface after each test action in the test script is executed, generate annotation information based on the operation type, operation element identifier, operation parameters and expected results of the current test action, and add the annotation information to the screenshot.

[0176] The video generation unit 3 is used to stitch together the screenshots corresponding to each test action into a video according to the order of each test action in the test script;

[0177] Test result determination unit 4 is used to confirm the test result based on the video, wherein the test result includes whether the test was successful and, if a problem occurs during the test, the location of the problem.

[0178] In one embodiment of the present invention, the test result determination unit 4 includes:

[0179] The large model processing module is used to input the video and the test rule set into the large artificial intelligence model with multimodal understanding capabilities. The large artificial intelligence model performs semantic reasoning based on the video frame content, the temporal relationship between frames and the test rule set, and outputs the execution effect judgment result of each test action. The execution effect judgment result includes the achievement status, achievement probability, anomaly category and corresponding anomaly frame position of each test action.

[0180] The test result determination module is used to determine that the test is successful when all test actions in the execution effect determination result are in the achieved state; otherwise, the test is determined to fail and an exception report is generated.

[0181] In one embodiment of the present invention, the style modification unit 1 is specifically used to modify the front-end style of the operation elements in the user interface involved in the test script from the initial style to the corresponding custom style before the test script is run.

[0182] In one embodiment of the present invention, the device further includes:

[0183] The callback unit is used to modify the front-end style of the user interface elements involved in the test script back to their corresponding initial styles after the test script has finished running.

[0184] In one embodiment of the present invention, the style modification unit 1 is specifically used to modify the front-end style of the operation element corresponding to the test action in the user interface from the initial style to the corresponding custom style before each test action in the test script is executed.

[0185] In one embodiment of the present invention, the device further includes:

[0186] The callback unit is used to modify the front-end style of the operation element corresponding to the test action in the user interface back to the corresponding initial style after the test action is completed and a screenshot is taken.

[0187] In one embodiment of the present invention, the screenshot processing unit 2 is specifically used to take a screenshot of the user interface after the last test action of a series of consecutive test actions for the same operation element on the user interface is executed, and generate unified annotation information based on the operation type, operation element identifier, operation parameters and expected results of the series of consecutive test actions.

[0188] In one embodiment of the present invention, the test result determination unit 4 includes:

[0189] The manual processing module is used to send the video to the testers so that the testers can confirm whether the test was successful based on the custom style and annotation information displayed in the video, and identify the location of the problem if a problem occurs during the test.

[0190] To achieve the above objectives, according to another aspect of this application, a computer device is also provided. For example... Figure 7 As shown, the computer device includes a memory, a processor, a communication interface, and a communication bus. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the method of the above embodiments.

[0191] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0192] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and units, such as the program units corresponding to the above-described method embodiments of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above-described method embodiments.

[0193] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0194] The one or more units are stored in the memory and, when executed by the processor, perform the methods described in the above embodiments.

[0195] The specific details of the aforementioned computer equipment can be understood by referring to the relevant descriptions and effects in the above embodiments, and will not be repeated here.

[0196] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed in a computer processor, implements the steps in the above-described user interface automated testing method. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0197] To achieve the above objectives, according to another aspect of this application, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described user interface automated testing method.

[0198] Obviously, those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.

[0199] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automated testing of user interfaces, characterized in that, include: Modify the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, wherein the custom style is used to highlight the operation elements for quick identification; After each test action in the test script is executed, a screenshot of the user interface is taken, and annotation information is generated based on the operation type, operation element identifier, operation parameters and expected results of the current test action, and the annotation information is added to the screenshot. According to the order of the test actions in the test script, the screenshots corresponding to each test action are stitched together into a video; The test results are confirmed based on the video, including whether the test was successful and, if any, the location of the problem when the test failed.

2. The user interface automated testing method according to claim 1, characterized in that, The step of confirming the test results based on the video includes: The video and the test rule set are input into a large artificial intelligence model with multimodal understanding capabilities. The large artificial intelligence model performs semantic reasoning based on the video frame content, inter-frame temporal relationship and test rule set, and outputs the execution effect judgment result of each test action. The execution effect judgment result includes the achievement status, achievement probability, anomaly category and corresponding anomaly frame position of each test action. The test is considered successful if all test actions in the execution effect judgment result are in the achieved state; otherwise, the test is considered to have failed and an exception report is generated.

3. The user interface automated testing method according to claim 1, characterized in that, The step of modifying the operational elements in the user interface involved in the test from the initial style to the corresponding custom style includes: Before the test script runs, the front-end style of the operation elements in the user interface involved in the test script is modified from the initial style to the corresponding custom style; The method further includes: After the test script finishes running, the front-end style of the user interface elements involved in the test script will be modified back to their corresponding initial style.

4. The user interface automated testing method according to claim 1, characterized in that, The step of modifying the operational elements in the user interface involved in the test from the initial style to the corresponding custom style includes: Before each test action in the test script is executed, the front-end style of the operation element corresponding to that test action in the user interface is changed from the initial style to the corresponding custom style. The method further includes: After the test action is completed and a screenshot is taken, the front-end style of the operation element corresponding to the test action in the user interface is changed back to the corresponding initial style.

5. The user interface automated testing method according to claim 1, characterized in that, After each test action in the test script is executed, a screenshot of the user interface is taken, and annotation information is generated based on the operation type, operation element identifier, operation parameters, and expected result of the current test action, including: For multiple consecutive test actions targeting the same operation element on the user interface, after the last test action in the series is executed, a screenshot of the user interface is taken, and unified annotation information is generated based on the operation type, operation element identifier, operation parameters, and expected results of the series of test actions.

6. The user interface automated testing method according to claim 1, characterized in that, The step of confirming the test results based on the video includes: The video is sent to the testers so that they can confirm whether the test was successful based on the custom style and annotation information displayed in the video, and identify the location of the problem if an issue occurs during the test.

7. The user interface automated testing method according to claim 1, characterized in that, The custom styles include at least one of the following: border highlighting, background color change, animation effect, font bolding, and font color change.

8. An automated user interface testing device, characterized in that, include: The style modification unit is used to modify the operation elements in the user interface involved in the test from the initial style to the corresponding custom style, wherein the custom style is used to highlight the operation elements for quick identification; The screenshot processing unit is used to take a screenshot of the user interface after each test action in the test script is executed, generate annotation information based on the operation type, operation element identifier, operation parameters and expected results of the current test action, and add the annotation information to the screenshot. The video generation unit is used to stitch together the screenshots corresponding to each test action into a video according to the order of each test action in the test script. The test result determination unit is used to confirm the test result based on the video, wherein the test result includes whether the test was successful and, if a problem occurs during the test, the location of the problem.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images