Data processing method, device and equipment and readable medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a readable medium. The method comprises the steps of determining whether a selector associated with a target interactive element in an interactive page can position the target interactive element or not in response to at least one test executed on the interactive page; in response to determining that the selector cannot locate the target interactive element, determining corresponding feature representations of a set of interactive elements in the interactive page; determining candidate interactive elements associated with the target interactive element from the group of interactive elements based on the historical feature representation of the target interactive element and the corresponding feature representation; and updating a selector associated with the target interactive element based on the candidate interactive element to continue testing the interactive page. In this way, the selector can be automatically repaired by the test case, and therefore the robustness and maintainability of the test case are improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to a method, apparatus, device, and computer-readable storage medium for data processing. Background Art

[0002] In interactive page design and development, testing interactive elements is crucial to user experience and satisfaction. However, in traditional end-to-end automated test cases, frequent updates to the user interface (UI), such as changes to element unique identifiers, hierarchy, or styles, often render hard-coded selectors ineffective, making testing difficult. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method for data processing is provided. The method includes: in response to at least one test having been performed on an interactive page, determining whether a selector associated with a target interactive element in the interactive page can locate the target interactive element; in response to determining that the selector cannot locate the target interactive element, determining corresponding feature representations of a set of interactive elements in the interactive page; based on the historical feature representations and the corresponding feature representations of the target interactive element, determining a candidate interactive element associated with the target interactive element from the set of interactive elements; and, based on the candidate interactive element, updating the selector associated with the target interactive element to continue testing the interactive page.

[0004] In a second aspect of the present disclosure, a device for data processing is provided. The device includes: a first determination module configured to determine whether a selector associated with a target interactive element in an interactive page can locate the target interactive element in response to at least one test having been performed on the interactive page; a second determination module configured to determine corresponding feature representations of a set of interactive elements in the interactive page in response to determining that the selector cannot locate the target interactive element; a third determination module configured to determine, from a set of interactive elements, a candidate interactive element associated with the target interactive element based on the historical feature representation and the corresponding feature representation of the target interactive element; and an update module configured to update the selector associated with the target interactive element based on the candidate interactive element to continue testing the interactive page.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method of the first aspect of the present disclosure.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium and can be executed by a processor to perform the method according to the first aspect of the present disclosure.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer-executable instructions, wherein the computer-executable instructions can be executed by a processor to perform the method according to the first aspect of the present disclosure.

[0008] It should be understood that the contents described in the Summary of the Invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent hereinafter with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A schematic diagram illustrating a process for data processing according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A block diagram showing an apparatus for data processing according to some embodiments of the present disclosure; and

[0013] Figure 4 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0014] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0015] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicit and implicit definitions may be included below.

[0016] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0017] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0018] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to the electronic device, application, server or storage medium and other software or hardware that performs the operation of the technical solution of the present disclosure based on the prompt message.

[0019] As an optional but non-limiting implementation, in response to receiving a user's active request, a prompt message may be sent to the user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0020] As used herein, the term "model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, corresponding outputs can be generated for given inputs. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0021] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs. It typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes the input from the previous layer.

[0022] Generally speaking, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association between input and output (also called input-to-output mapping) from the training data. The parameter values of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. The testing stage can sometimes be integrated into the training stage. In the application or inference stage, the trained model can be used to process the actual model input based on the parameter values obtained through training to determine the corresponding model output.

[0023] For the convenience of description, some concepts and terms are first defined here.

[0024] End-to-end automated test cases, World Wide Web (Web) automated test cases are automated scripts used to verify the functionality of web applications.

[0025] The Document Object Model (DOM) is a programming interface for processing and manipulating Hypertext Markup Language (HTML) documents. The DOM represents the structure of an HTML document, enabling programs to dynamically access and update its content, structure, and style. The DOM represents an HTML document as a tree, where each node represents a part of the document (e.g., an element, an attribute, text, etc.).

[0026] The core idea of the single-stage object detection algorithm (You Only Look Once, YOLO) is to directly predict multiple target categories and their locations in an image in a single forward propagation process, without the need for region proposals or multiple sliding window scans. This method uses a fully convolutional neural network (FCNN) structure, which divides the input image into a fixed number of grids, and directly predicts the target category, bounding box coordinates, and confidence level for each grid. Compared to two-stage detection methods (such as the R-CNN series), the YOLO method significantly improves inference speed while maintaining detection accuracy, making it suitable for real-time target detection tasks such as autonomous driving, security monitoring, and intelligent video analysis.

[0027] Residual Network (ResNet) is a deep convolutional neural network that introduces a residual learning mechanism and uses skip connections to alleviate the vanishing gradient problem in deep networks. Its core structure is the residual block, which allows the network to directly learn the residual mapping between input and output rather than a complete feature transformation, thereby reducing the difficulty of training deep networks. The ResNet architecture supports the construction of ultra-deep networks (such as ResNet-50, ResNet-101, and ResNet-152) and is widely used in computer vision tasks such as image classification, object detection, and semantic segmentation.

[0028] An agent benchmark is a performance evaluation framework for evaluating artificial intelligence (AI) agents in specific tasks or environments. This benchmark typically includes standardized datasets, task environments, and evaluation metrics to measure the agent's capabilities in perception, decision-making, reasoning, and interaction. Agent benchmarks can be used to evaluate reinforcement learning, large-scale agent models (LLM-based agents), and autonomous systems, and are widely used in robotic control, automated trading, game AI, and general artificial intelligence (AGI) research.

[0029] As briefly mentioned above, in traditional end-to-end automated test case practices, the frequent iterative updates of the user interface cause hard-coded selectors to become invalid, bringing difficulties to testing work.

[0030] In some solutions, the target element can be located by matching similar DOM paths. However, when the page is significantly modified, static rules are difficult to adapt, resulting in matching failures. In other solutions, the target element can be located by matching based on shallow visual features such as templates or pixel comparisons. However, this method is sensitive to changes in interface style and element status, and has poor generalization capabilities.

[0031] In order to at least partially solve the above-mentioned problems and other potential problems that may exist in traditional solutions, the present disclosure provides a data processing solution. According to various embodiments of the present disclosure, in response to at least one test having been performed on an interactive page, it is determined whether the selector associated with the target interactive element in the interactive page can locate the target interactive element. In the case where the selector cannot locate the target interactive element, the corresponding feature representations of a set of interactive elements in the interactive page are determined, and based on the historical feature representations and corresponding feature representations of the target interactive element, the candidate interactive element associated with the target interactive element is determined from a set of interactive elements. Subsequently, based on the candidate interactive element, the selector associated with the target interactive element is updated to continue testing the interactive page. In this way, the test case can automatically repair the selector, thereby improving the robustness and maintainability of the test case.

[0032] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0033] Figure 1 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, the example environment 100 generally includes an electronic device 110. The electronic device 110 can perform end-to-end automated testing on an interactive page 120 to be tested based on a test case 112. Such a test case can be deployed locally on the electronic device 110, or deployed on another device or a third-party platform. When the test case is deployed on another device or a third-party platform, the electronic device 110 can interact with the other device or the third-party platform to invoke the test case 112.

[0034] In some embodiments, the electronic device 110 can open the interactive page 120 based on the page link. Alternatively, it can also interact with other devices or third-party platforms. For example, the electronic device 110 receives the interactive page 120 or the page link of the interactive page 120 from other devices or third-party platforms.

[0035] In the example environment 100, the electronic device 110 can be any type of device with computing capabilities, including a terminal device or a server device. The terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof. The server device can include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like.

[0036] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0037] Figure 2 FIG2 shows a schematic diagram of a process 200 for data processing according to some embodiments of the present disclosure. The process 200 may be implemented in the electronic device 110. Figure 1 The process 200 is described in detail.

[0038] It should be understood that the process 200 can be implemented after at least one test has been performed on the interactive page 120. The at least one test on the interactive page 120 will be described below.

[0039] For any of the at least one test, after the test is successful, the electronic device 110 can build an element archive for the interactive page 120. In some embodiments, the element archive can store images of interactive elements in the interactive page 120, feature representations of the interactive elements, and selectors associated with the interactive elements. In some embodiments, the selectors can be generated by the test case 112, such as XPath selectors, CSS selectors, etc.

[0040] When building the element archive, the electronic device 110 can obtain the image of the interactive element in the interactive page and record the selector associated with the interactive element. Then, the ResNet network model is used to extract the feature representation of the interactive element based on the image of the interactive element. After obtaining the feature representation, the image, selector and feature representation of the interactive element are associated and stored in the element archive. For the purpose of discussion, the feature representation of the interactive element here may be referred to as "historical feature representation" hereinafter.

[0041] For example, the electronic device 110 may extract the feature representation of the interactive element in the following manner:

[0042]

[0043]

[0044] For testing of the interactive page 120 , when a selector fails (for example, the corresponding interactive element cannot be located), the element archive can be used to repair or update the failed selector.

[0045] Specifically, in block 210, the electronic device 110 determines whether the selector associated with the target interactive element in the interactive page 120 can locate the target interactive element in response to having performed at least one test on the interactive page 120. It should be understood that the target interactive element can be any interactive element in the interactive page 120, and the embodiments of the present disclosure are not limited in this regard.

[0046] For example, the electronic device 110 can use the test case 112 to generate a selector for a target interactive element in the interactive page 120, thereby locating the target interactive element through the selector to test the target interactive element. For example, if the test case 112 cannot select the target interactive element through the selector, it can be determined that the positioning of the target interactive element has failed.

[0047] In some embodiments, the electronic device 110 may also use a code-free testing tool to locate the target interactive element. For example, a large model may be used to locate the target interactive element in the interactive page 120, and the large model may be used to simulate user operations to implement testing of the interactive page 120. For example, an agent benchmark based on the large model may also be used to test the interactive page 120. For example, OS World, Screen Spot Pro, etc.

[0048] At block 220 , the electronic device 110 determines corresponding feature representations of a set of interactable elements in the interactive page 120 in response to determining that the selector cannot locate the target interactable element.

[0049] For example, a group of interactable elements may include all elements in the interactive page 120. For example, a group of interactable elements may include all elements in a designated area in the interactive page 120. For example, the designated area may be an area containing a target interactable element.

[0050] In some embodiments, the electronic device 110 may utilize a machine learning model to determine corresponding areas in the image of the interactive page 120 that indicate a set of interactable elements.

[0051] For example, the electronic device 110 may obtain an image of the interactive page 120 or an image of a specified area in the interactive page 120. For example, a screenshot of the interactive page 120 or the specified area in the interactive page 120. After obtaining the image, the electronic device 110 may use a trained machine learning model to identify the interactive elements in the interactive page 120, thereby obtaining a set of interactive elements. For example, a trained object detection model may be used to detect the bounding boxes of the interactive elements.

[0052] In some embodiments, the target detection model can be implemented based on the YOLO target detection framework. The training samples for training the target detection model can be open source interactive UI elements.

[0053] For example, the electronic device 110 may use the trained object detection model to detect the bounding box of the interactive element in the following manner:

[0054]

[0055] Furthermore, the electronic device 110 may utilize a trained deep learning model to extract corresponding feature representations based on the identified areas of a group of interactive elements.

[0056] In some embodiments, the deep learning model can be implemented based on a deep residual network ResNet. For example, the classification layer of the residual network can be removed, and the front network can be used as a feature extractor. During training, open source interactive UI elements can be used for training. Furthermore, after the training is completed, the trained residual network can be fine-tuned based on data from a specific field. For example, an interactive UI in the e-commerce field can be used to fine-tune it so that the residual network is more suitable for extracting feature representations of interactive elements in the e-commerce field.

[0057] For example, the electronic device 110 may utilize a deep learning model to extract corresponding feature representations in the following manner:

[0058]

[0059] In block 220 , the electronic device 110 determines a candidate interactable element associated with the target interactable element from a set of interactable elements based on the historical feature representation and the corresponding feature representation of the target interactable element.

[0060] In some embodiments, the electronic device 110 may respectively determine the similarity between the historical feature representation of the target interactive element and the corresponding feature representations of a set of interactive elements, and use the similarity to select a candidate interactive element from the set of interactive elements.

[0061] In some embodiments, the electronic device 110 may select at least one interactive element having a similarity greater than a similarity threshold (e.g., 0.8) from a group of interactive elements, and sort the at least one interactive element based on the similarity, and determine a candidate interactive element based on the sorted at least one interactive element. For example, the interactive element with the highest similarity may be selected from the sorted at least one interactive element as a candidate interactive element associated with the target interactive element.

[0062] For example, the electronic device 110 may determine the candidate interactive elements in the following manner:

[0063]

[0064]

[0065]

[0066] In block 230 , the electronic device 110 updates the selector associated with the target interactable element based on the candidate interactable elements to continue testing the interactive page.

[0067] In some embodiments, electronic device 110 may determine coordinates indicating the location of a candidate interactive element on an interactive page, generate a new selector based on the coordinates, and associate the new selector with the target interactive element to store the new selector in the element archive. In some embodiments, the coordinates indicating the location of a candidate interactive element on an interactive page may be the center coordinates of the candidate interactive element.

[0068] For example, the electronic device 110 may update the selector and continue to test the interactive page 120 in the following manner:

[0069]

[0070]

[0071]

[0072] In some embodiments, if a candidate interactive element cannot be determined from the interactive page 120, a backup strategy or manual intervention may be triggered. It should be understood that the backup strategy may be any method in the related art for repairing the inability to locate an interactive element, and the present disclosure is not limited thereto.

[0073] In summary, according to the embodiments of the present disclosure, during the testing of an interactive page using a test case, the test case can automatically repair a failed selector, thereby improving the robustness and maintainability of the test case.

[0074] Figure 3 1 shows a block diagram of an apparatus 300 for data processing according to some embodiments of the present disclosure. The apparatus 300 may be implemented in the electronic device 110. Each module / component in the apparatus 300 may be implemented by hardware, software, firmware, or any combination thereof.

[0075] The device 300 includes a first determination module 310, which is configured to determine whether a selector associated with a target interactive element in an interactive page can locate the target interactive element in response to at least one test having been performed on the interactive page. The device 300 also includes a second determination module 320, which is configured to determine a corresponding feature representation of a set of interactive elements in the interactive page in response to determining that the selector cannot locate the target interactive element. The device 300 also includes a third determination module 330, which is configured to determine a candidate interactive element associated with the target interactive element from a set of interactive elements based on the historical feature representation and the corresponding feature representation of the target interactive element. The device 300 also includes an update module 340, which is configured to update the selector associated with the target interactive element based on the candidate interactive element to continue testing the interactive page.

[0076] In some embodiments, the update module 340 is further configured to determine coordinates indicating a location of the candidate interactable element in the interactive page; based on the coordinates, generate a new selector; and associate the new selector with the target interactable element.

[0077] In some embodiments, the third determination module 330 is further configured to respectively determine the similarity between the historical feature representation and each feature representation in the corresponding feature representation; and select a candidate interactive element from a set of interactive elements using the similarity.

[0078] In some embodiments, the third determination module 330 is further configured to select at least one interactive element having a similarity greater than a similarity threshold from a group of interactive elements; and sort the at least one interactive element based on the similarity; and determine a candidate interactive element based on the sorted at least one interactive element.

[0079] In some embodiments, the historical feature representation is determined as follows: in at least one test, obtaining a first image of the target interactable element; and determining, using a first machine learning model, the historical feature representation of the target interactable element based on the first image.

[0080] In some embodiments, the apparatus 300 further includes an associating module configured to associate the first image, the historical feature representation, and a selector for locating a target interactable element.

[0081] In some embodiments, the second determination module 320 is further configured to determine, in the image of the interactive page, a corresponding area indicating a set of interactive elements using a second machine learning model; and to determine, based on the corresponding area, a corresponding feature representation of a set of interactive elements using a third machine learning model.

[0082] The units included in the device 300 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 300 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0083] Figure 4 4 shows a block diagram of an electronic device 400 in which one or more embodiments of the present disclosure may be implemented. Figure 4 The illustrated electronic device 400 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 4 The electronic device 400 shown can be used to implement Figure 1 electronic device 110.

[0084] like Figure 4 As shown, electronic device 400 is in the form of a general electronic device. Components of electronic device 400 may include, but are not limited to, one or more processors or processing units 410, memory 420, storage device 430, one or more communication units 440, one or more input devices 450, and one or more output devices 460. Processing unit 410 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 420. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 400.

[0085] The electronic device 400 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 400, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 420 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 430 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and can be accessed within the electronic device 400.

[0086] The electronic device 400 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 4 As shown in FIG, a magnetic disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. Memory 420 may include a computer program product 425 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0087] The communication unit 440 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 400 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 400 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0088] Input device 450 may be one or more input devices, such as a mouse, keyboard, or trackball. Output device 460 may be one or more output devices, such as a display, a speaker, or a printer. Electronic device 400 may also communicate with one or more external devices (not shown) via communication unit 440 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with electronic device 400, or with any device that allows electronic device 400 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0089] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0090] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0091] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0092] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0093] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0094] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A data processing method, comprising: In response to at least one test having been performed on an interactive page, determining whether a selector associated with a target interactable element in the interactive page can locate the target interactable element; In response to determining that the selector cannot locate the target interactable element, determining corresponding feature representations of a set of interactable elements in the interactive page; determining, from the set of interactable elements, a candidate interactable element associated with the target interactable element based on the historical feature representation of the target interactable element and the corresponding feature representation; as well as Based on the candidate interactable elements, the selector associated with the target interactable element is updated to continue testing the interactive page.

2. The method of claim 1 , wherein updating the selector comprises: determining coordinates indicating a position of the candidate interactable element in the interactive page; Based on the coordinates, a new selector is generated; as well as Associate the new selector with the target interactive element.

3. The method of claim 1 , wherein determining the candidate interactable element comprises: respectively determining a similarity between the historical feature representation and each feature representation in the corresponding feature representation; as well as The candidate interactive element is selected from the set of interactive elements using the similarity.

4. The method of claim 3 , wherein selecting the candidate interactable element from the set of interactable elements comprises: Selecting at least one interactive element from the group of interactive elements, the similarity of which is greater than a similarity threshold; as well as sorting the at least one interactable element based on the similarity; as well as The candidate interactable element is determined based on the ranked at least one interactable element.

5. The method of claim 1 , wherein the historical characterization is determined as follows: In the at least one test, obtaining a first image of the target interactive element; and The historical feature representation of the target interactable element is determined based on the first image using a first machine learning model.

6. The method according to claim 5, further comprising: The first image, the history feature representation, and a selector for locating the target interactable element are associated.

7. The method of claim 1 , wherein determining corresponding feature representations of a set of interactable elements in the interactive page comprises: determining, using a second machine learning model, corresponding regions indicative of the set of interactable elements in the image of the interactive page; as well as The corresponding feature representations of the set of interactable elements are determined based on the corresponding regions using a third machine learning model.

8. A device for data processing, comprising: a first determination module configured to, in response to at least one test having been performed on the interactive page, determine whether a selector associated with a target interactable element in the interactive page can locate the target interactable element; a second determining module configured to, in response to determining that the selector cannot locate the target interactable element, determine corresponding feature representations of a set of interactable elements in the interactive page; a third determining module configured to determine, from the set of interactable elements, a candidate interactable element associated with the target interactable element based on the historical feature representation of the target interactable element and the corresponding feature representation; as well as An updating module is configured to update the selector associated with the target interactive element based on the candidate interactive element to continue testing the interactive page.

9. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 7 when executed by the at least one processing unit.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 7.

Images