Real-time collaborative web test capturing method and system based on multi-feature scoring

Abstract

The application provides a real-time collaborative Web test capturing method and system based on multi-feature scoring, and relates to the technical field of Web automation testing.The application obtains interactive coordinates clicked by a user, performs cyclic penetration detection, and outputs a candidate element sequence;then, five-dimensional feature quantization scoring is performed to output a scoring result to select a target element, generate a corresponding unique identifier, and encapsulate a capturing event; a change observer interface is used to listen to changes, a browser page steady state is perceived in combination with a jitter prevention mechanism, so that when a page structure changes or is redrawn, a relevant listener is awakened, computing power is allocated to reselect a target element, a capturing event is encapsulated, and when the page is stable, the capturing event is sent to a backend server to be broadcast to other collaboration terminals in the same room.The application can effectively solve the problems of element positioning inaccuracy and low collaboration efficiency, reduce system resource consumption, and realize efficient real-time collaboration and financial-level data security protection in distributed crowdsourcing testing.

Description

Technical Field

[0001] This invention relates to the field of Web automated testing and computer collaborative processing technology, and more specifically, to a real-time collaborative Web testing capture method and system based on multi-feature scoring. Background Technology

[0002] In modern software engineering and quality assurance, crowdsourced testing and agile testing play a significant role in vulnerability discovery, functional verification, and user experience evaluation. However, in real-world projects, manual testing processes are cumbersome, error-prone, costly, and have poor reproducibility. Web automation testing capture tools based on browser extensions can automatically generate scripts, alleviating the bottleneck of manual testing. However, single-page applications and component-based frameworks lead to dynamic, deeply nested, and frequently asynchronous DOM structures, posing significant challenges to existing automated capture technologies.

[0003] Existing technologies exhibit several shortcomings when dealing with modern web applications. In element localization, traditional tools heavily rely on native shallow capture attributes. When faced with transparent debounce masks and deeply nested, semantically meaningless containers prevalent in modern web pages, target misalignment easily occurs, leading to inaccurate extraction of structured features and frequent element-not-found errors during automated playback. Regarding performance and state awareness, to capture dynamic components in single-page applications that do not refresh or load asynchronously, existing tools often employ high-frequency timed polling or static global event binding. This coarse-grained listening strategy not only fails to accurately perceive page stability but also causes significant computational waste and main thread blocking during interaction. Furthermore, in terms of collaboration and security mechanisms, existing test capture tools are mostly single-machine architectures, with test data stored only locally, failing to meet the collaborative needs of crowdsourced testing teams for real-time state synchronization across multiple platforms. Especially in enterprise intranet environments or testing scenarios involving sensitive inputs, the lack of application-layer end-to-end encryption protection exposes the system to significant risks of data leakage and man-in-the-middle tampering.

[0004] While some current solutions attempt to use recording tools to capture the entire DOM information of a web page and combine it with visual auxiliary models to correct location errors, these methods rely on full recording followed by offline correction, consuming significant front-end memory and storage resources. Furthermore, the complex visual reasoning calculations cannot be implemented in real-time on the front end for intent recognition. Other solutions based on heuristic algorithms focus on post-processing filtering of generated test reports, failing to address inaccurate location and missed operations at the data source. They also lack effective encryption protection during data upload, making it difficult to meet the requirements of high-concurrency collaboration and high security.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention aims to provide a real-time collaborative Web test capture method and system based on multi-feature scoring, in order to solve the technical defects in existing Web automated test capture and crowdsourced test collaboration, such as inaccurate element location, redundant system resource consumption, low efficiency of distributed collaboration, and risk of sensitive data leakage.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] A real-time collaborative web test capture method based on multi-feature scoring, applied to the browser front-end, includes:

[0009] S1, obtain the interaction coordinates of the user's click, perform a loop penetration probe on the interaction coordinates to obtain the top-level element, add the top-level element to the candidate element sequence, modify the interaction event response attribute of the top-level element to transparent, until the termination condition is met, restore the interaction event response attributes of all modified elements, and output the candidate element sequence.

[0010] S2, perform five-dimensional feature quantization scoring on the candidate element sequence and output the score result for each candidate element; wherein, the five-dimensional features include spatial dimension, depth dimension, interaction dimension, content dimension and context dimension;

[0011] S3. Select the target element based on the scoring result, generate the corresponding unique identifier, and encapsulate it into a capture event containing the target element's interaction type, identifier, timestamp, task ID, and client identifier;

[0012] S4. Utilize the change observer interface to monitor changes in the document object model tree where the target element is located, and combine this with a debouncing mechanism to perceive the steady state of the browser page. When the page undergoes structural changes or repainting, wake up the listeners related to the target element, allocate computing power to reselect the target element, encapsulate the capture event, and send the encapsulated capture event to the backend server when the page is stable, so that the backend server can broadcast it to other collaborative clients in the same room for testing.

[0013] Preferably, S1 specifically comprises:

[0014] Get the coordinates of user clicks ;

[0015] Initialize and define the candidate element sequence, the number of probed layers, the count of consecutive large containers, and the list of elements with modified styles;

[0016] The process of cyclic penetration detection is as follows:

[0017] Call Get the top-level element of the current interaction coordinates;

[0018] Add the top-level element to the candidate element sequence and determine whether the top-level element is a large container. If it is, increment the consecutive large container count by 1; otherwise, reset the consecutive large container count.

[0019] Record the original interactive event response attributes of the top-level element, and store the top-level element and its original interactive event response attributes in the list of elements with modified styles;

[0020] Modify the original interactive event response property of the top-level element to transparent, and increment the number of detected layers by 1 until the termination condition is triggered;

[0021] Iterate through the list of elements with modified styles, restore the original interactive event response properties for each element, and output the candidate element sequence.

[0022] Preferably, the termination condition is: the topmost element is the root node of the document, or the number of detected layers has reached a preset upper limit, or the continuous large container count value has reached a set large container count threshold.

[0023] The large container counting threshold is dynamically adjusted based on the proportion of container elements on the page and the density of interactive controls, using the following formula:

[0024]

[0025] in, This is the threshold for counting large containers; Basic threshold; , This is the weighting adjustment coefficient; This represents the total number of container elements on the current page. This represents the total number of DOM nodes on the current page. Density of interactive controls.

[0026] Preferably, the method further includes: using the interaction coordinates as the center, performing extended penetration detection within a preset range to obtain all candidate element sequences, and removing duplicates from all candidate element sequences to obtain deduplicated candidate element sequences.

[0027] Preferably, the five-dimensional feature quantification scoring is specifically as follows:

[0028] Iterate through each candidate element in the candidate element sequence and initialize the candidate element score to 0;

[0029] The candidate elements are scored in spatial dimension sequentially: the relative displacement between the interaction coordinates and the center point of the candidate element is calculated; if the interaction coordinates are inside the candidate element, points are added based on the size and position of the candidate element; otherwise, points are deducted.

[0030] Depth-dimensional scoring of candidate elements: Obtain the document object model depth and hierarchical index attribute of the candidate elements. That is, the deeper the document object model, the greater the penalty weight of the score, and the higher the hierarchical index attribute value, the greater the bonus weight.

[0031] Candidate elements are scored based on their interaction dimensions: the scores are weighted by combining the HTML tag type weights, native event attributes, and accessibility attributes of the candidate elements.

[0032] Scoring candidate elements based on content dimensions: adjusting scores according to the visible text length and business characteristic attributes of candidate elements;

[0033] Context-based scoring of candidate elements: By traversing upwards through the parent nodes of the candidate elements, the business container type of the candidate elements is determined. Candidate elements that are within form, navigation, or table containers are given bonus points, while those that are not are not.

[0034] The score of a candidate element is the result of a comprehensive evaluation of the spatial dimension, depth dimension, interaction dimension, content dimension, and context dimension.

[0035] Preferably, it also includes: during the scoring process, introducing a hard penalty mechanism to reduce the score of hidden input boxes, disabled state elements and accessible hidden nodes in the candidate element sequence to zero, and to reduce the score of empty elements and broad aggregation containers.

[0036] After scoring, the candidate elements are sorted in descending order of total score. If the ratio of the highest score to the second highest score exceeds the preset judgment threshold, the candidate element with the highest score is directly selected as the target element.

[0037] If the ratio does not exceed the judgment threshold, the relative distance and compactness attributes of the two candidate elements are compared together, and the final target element is output.

[0038] The determination threshold is dynamically adjusted based on the page DOM complexity, and the formula is as follows:

[0039] ;

[0040] in, This is the final judgment threshold; The preset base threshold; This is the adjustment coefficient; This represents the total number of DOM nodes on the current page. This represents the average number of DOM nodes for similar pages as statistically analyzed by the system. , These represent the maximum and minimum number of DOM nodes for the same type of page, respectively.

[0041] The compactness is used to measure whether an element's shape is a core interactive control, and the formula is as follows:

[0042] ;

[0043] in, For compactness; , These represent the width and height of the element, respectively.

[0044] Preferably, the anti-shake mechanism ensures that changes triggered consecutively within the time window are executed only after the last change is completed;

[0045] The time window is dynamically adjusted according to the current system load. When the page is in a high-frequency asynchronous rendering stage, the time window is automatically extended; when the page enters a steady state, the time window is shortened.

[0046] Preferably, it further includes: the backend server constructing logically isolated collaborative rooms based on the identifier in the captured event;

[0047] A distributed caching engine is used to perform secondary deduplication within a time window. The deduplicated captured events are then standardized to obtain standardized test content.

[0048] Based on the mechanism of excluding sender identifiers, the standardized test content is broadcast to other collaborative clients in the same room in real time;

[0049] The specific method of the second-level deduplication is as follows: the received identifier is compared with the historical records in the distributed cache to intercept redundant operations that are triggered repeatedly or across tabs.

[0050] The standardization process includes generating Chinese descriptions and assigning serial numbers in ascending order.

[0051] Preferably, it further includes: performing end-to-end hybrid encrypted communication between the browser front-end and the back-end server; the hybrid encrypted communication includes identity authentication, key negotiation and encrypted transmission;

[0052] During the authentication phase, the browser frontend sends a request to the backend server, which includes an authorization code and an extended identifier. After receiving the request, the backend server verifies the validity of the authorization code and the extended identifier. Once the verification is successful, the backend server generates a JWT token containing the user identifier and user permissions and returns it to the browser frontend. The browser frontend receives and stores the JWT token as an identity credential during the key negotiation phase.

[0053] During the key negotiation phase, the browser frontend initiates a WSS verification connection with the backend server based on the JWT token. After the connection is established, the browser frontend sends a handshake message to the backend server, requests and initiates ECDH key pair generation, and sends the generated ECDH key pair to the backend server. The backend server receives the ECDH key pair, generates an ECDH public key, and sends the ECDH public key to the browser frontend, thus completing the key negotiation.

[0054] During the encrypted transmission phase, the browser frontend encrypts the captured event based on the negotiated key, generates ciphertext and an integrity signature verification tag, and sends them to the backend server. After verifying the integrity signature verification tag, the backend server decrypts the ciphertext, processes the decrypted plaintext event, and then re-encrypts the processed event data before broadcasting it to other collaborating clients in the same room.

[0055] This invention also provides a real-time collaborative web test capture system based on multi-feature scoring, comprising:

[0056] Browser front-end and back-end server;

[0057] The browser front-end includes a candidate element unit, a five-dimensional feature quantization and scoring unit, a capture event encapsulation unit, and a resource scheduling and sending unit;

[0058] The candidate element unit is used to obtain the interaction coordinates of the user's click, to perform a cyclic penetration probe on the interaction coordinates to obtain the top-level element, and to add the top-level element to the candidate element sequence. The interaction event response attribute of the top-level element is modified to transparent until the termination condition is met, after which the interaction event response attributes of all modified elements are restored, and the candidate element sequence is output.

[0059] The five-dimensional feature quantization scoring unit is used to perform five-dimensional feature quantization scoring on the candidate element sequence and output the score result of each candidate element; wherein, the five-dimensional features include spatial dimension, depth dimension, interaction dimension, content dimension and context dimension;

[0060] The capture event encapsulation unit is used to select target elements according to the scoring results, generate corresponding unique identifiers, and encapsulate them into capture events containing target element interaction type, identifier, timestamp, task ID and client identifier;

[0061] The resource scheduling and sending unit is used to monitor changes in the document object model tree where the target element is located using the change observer interface, and to perceive the steady state of the browser page in conjunction with the debouncing mechanism. When the page undergoes structural changes or repainting, the listener related to the target element is awakened, computing power is allocated to reselect the target element, the capture event is encapsulated, and the encapsulated capture event is sent to the backend server when the page is stable.

[0062] The backend server includes an identity authentication unit and a routing distribution unit;

[0063] The identity authentication unit is used to verify the legitimacy of the browser front-end, and receives the capture event after the verification is valid;

[0064] The routing and distribution unit is used to construct logically isolated collaborative rooms based on the identifiers in the captured events, and then use a distributed caching engine to perform secondary deduplication within a time window. The deduplicated captured events are then standardized, and the standardized test content is broadcast to other collaborative clients in the same room.

[0065] The present invention also provides a real-time collaborative web test capture device based on multi-feature scoring, including a processor and a memory. The memory stores a computer program that can be executed by the processor to implement the real-time collaborative web test capture method based on multi-feature scoring as described above.

[0066] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device on which the computer-readable storage medium resides, implement the real-time collaborative web testing capture method based on multi-feature scoring as described above.

[0067] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0068] Firstly, it achieves highly accurate and lightweight positioning of complex web front-end elements. This invention introduces a loop-through technology based on interactive coordinate elements, combined with a five-dimensional feature weighted scoring algorithm covering space, depth, interaction, content, and context. Without relying on heavy computer vision models, it achieves effective identification of target elements, accurately locks onto real business controls, significantly reduces the positioning offset rate during automated playback, and improves the robustness of test scripts.

[0069] Secondly, it achieves efficient resource scheduling during dynamic page capture. This invention replaces traditional full recording or high-frequency polling modes by constructing an event-driven architecture based on a change observer and debouncing mechanism. It allocates computing power only when the page undergoes effective redrawing or structural changes, achieving accurate perception of the asynchronous rendering state of single-page applications. This adaptive scheduling method greatly reduces the impact of browser extensions on user page performance, avoids memory overflow and interface lag, and ensures a smooth and seamless testing and capture process.

[0070] Third, it enables efficient real-time collaboration in distributed crowdsourced testing. This invention establishes a long-connection-based isolated room allocation mechanism and a backend distributed caching deduplication mechanism to achieve millisecond-level synchronization of test steps between geographically dispersed teams. Combined with a time-window deduplication algorithm, it ensures the uniqueness and consistency of data within collaborative rooms, solving the problems of duplicate recording, redundant test cases, and state fragmentation in a distributed environment from an architectural perspective, significantly shortening the construction cycle of massive test cases.

[0071] Fourth, it achieves financial-grade security protection throughout the entire testing lifecycle. This invention provides end-to-end confidentiality protection for sensitive business data in crowdsourced testing environments by deeply integrating token authentication, dynamic key negotiation, and encrypted transmission systems. The dynamic key negotiation mechanism ensures forward security of the communication link, so even if a single session key is leaked, it will not threaten the security of historical data; the encrypted transmission algorithm, while providing data encryption, performs strict integrity checks, effectively resisting man-in-the-middle attacks and malicious instruction tampering, meeting enterprise-level security compliance requirements.

[0072] In summary, this invention constructs a precise, low-consumption, efficient, and secure Web test capture system through a series of collaborative technical means, including front-end penetration positioning, dynamic computing power adaptive scheduling, back-end real-time collaborative distribution, and end-to-end hybrid encryption. This not only improves the quality and efficiency of automated test script generation but also provides solid technical support for large-scale distributed collaborative testing. Attached Figure Description

[0073] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0074] Figure 1 This is a schematic diagram of a real-time collaborative web test capture method based on multi-feature scoring, as provided in Example 1.

[0075] Figure 2 The flowchart for penetration detection and multi-feature scoring provided in Example 1.

[0076] Figure 3 The overall architecture diagram of the real-time collaborative web test capture process provided in Example 1.

[0077] Figure 4 The timing diagram for end-to-end hybrid encrypted collaborative interaction provided in Example 1.

[0078] Figure 5This is a schematic diagram of a real-time collaborative web test capture system based on multi-feature scoring, provided in Embodiment 2.

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0081] Example 1

[0082] Embodiment 1 of the present invention provides a real-time collaborative web test capture method based on multi-feature scoring, which can be implemented by a real-time collaborative web test capture device based on multi-feature scoring (hereinafter referred to as the test capture device), and in particular, executed by one or more processors within the test capture device.

[0083] In this embodiment, the test capture device may be an electronic device equipped with a processor, the processor having a computer program for the real-time collaborative Web test capture method based on multi-feature scoring, and the computer program being executable, such as a computer, smartphone, smart tablet, workstation, etc., without limitation.

[0084] The core architecture of this invention adopts a distributed collaborative model, mainly consisting of a browser front-end running in the user's browser environment and a back-end server deployed in the cloud. The browser front-end senses the user's interactive intent on the webpage in real time and establishes a secure full-duplex communication link with the back-end server based on the WebSocket protocol. The back-end server, through a logical isolation mechanism of collaborative rooms, connects engineers with different testing tasks on a unified real-time synchronization plane, ensuring accurate capture, efficient scheduling, and secure transmission of test steps.

[0085] like Figure 1 As shown, a real-time collaborative web test capture method based on multi-feature scoring includes steps S1 to S4.

[0086] S1, obtain the interaction coordinates of the user's click, perform a loop penetration probe on the interaction coordinates to obtain the top-level element, add the top-level element to the candidate element sequence, modify the interaction event response attribute of the top-level element to transparent, until the termination condition is met, restore the interaction event response attributes of all modified elements, and output the candidate element sequence.

[0087] This step aims to resolve element positioning offset issues caused by deep nesting, transparent debounce overlays, or dynamic overlays in modern web front-end frameworks. For example... Figure 2 As shown, when a user triggers an interactive action such as clicking, typing, or scrolling on a browser page, the physical interaction coordinates of the screen at the time of the interaction are first obtained. To penetrate any potential visual obscurations, the system calls the browser's native coordinate element detection interface, elementsFromPoint, which returns a sequence of all overlapping DOM elements arranged from top to bottom along the Z-axis.

[0088] Specifically:

[0089] Get the coordinates of user clicks ;

[0090] Initialize and define the candidate element sequence, the number of probed layers, the count of consecutive large containers, and the list of elements with modified styles;

[0091] The process of cyclic penetration detection is as follows:

[0092] Call Get the top-level element E of the current interaction coordinates;

[0093] Add the top-level element to the candidate element sequence and determine whether the top-level element E is a large container (such as a div whose width or height exceeds 80% of the screen). If it is, increment the consecutive large container count by 1; otherwise, reset the consecutive large container count to 0.

[0094] Record the original interactive event response attributes of the top-level element, and store the top-level element and its original interactive event response attributes in the list of elements with modified styles;

[0095] Modify the original interactive event response property of the top-level element to transparent (this can be done by dynamically modifying the element's CSS style property using JavaScript, setting its pointer-events property to none), and increment the number of detected layers by 1 until the termination condition is triggered.

[0096] This operation makes the element no longer respond to any coordinate positioning events during subsequent probing, thus achieving physical transparency.

[0097] Iterate through the list of elements with modified styles, restore the original interactive event response attributes of each element to ensure that normal page interaction is not affected; output the candidate element sequence.

[0098] Furthermore, the termination condition may be: the topmost element is the document root node (document.body), or the number of detected layers has reached a preset upper limit (e.g., 50 layers), or the continuous large container count value has reached a set large container count threshold, such as 3.

[0099] Of course, the large container counting threshold can also be dynamically adjusted based on the proportion of container elements on the page and the density of interactive controls to avoid the fixed threshold from mistakenly terminating the detection on simple / complex pages. The formula is as follows:

[0100]

[0101] in, This is the threshold for counting large containers; Set the base threshold, for example, to 3; , This is the weighting adjustment coefficient; This represents the total number of container elements on the current page. This represents the total number of DOM nodes on the current page. Interactive control density (number of interactive controls / number of pixels in the visible area of ​​the page).

[0102] In this embodiment, the large container can refer to a container element whose offsetWidth occupies a viewport ratio greater than a preset ratio and does not contain business interaction attributes. The consecutive large container count is set to 3 to filter out multiple nested background layout layers.

[0103] To further improve the tolerance of positioning, the method also includes: taking the interaction coordinates as the center, performing extended penetration detection within a preset range (such as a range of ±5 pixels) to obtain all candidate element sequences, and removing duplicates from all candidate element sequences to obtain a deduplicated candidate element sequence.

[0104] S2, perform five-dimensional feature quantization scoring on the candidate element sequence and output the score result for each candidate element; wherein, the five-dimensional features include spatial dimension, depth dimension, interaction dimension, content dimension and context dimension.

[0105] Furthermore, the five-dimensional feature quantification scoring is specifically as follows:

[0106] Iterate through each candidate element in the candidate element sequence and initialize the candidate element score to 0;

[0107] The candidate elements are scored in spatial dimension in turn: the relative displacement (such as Euclidean distance) between the interaction coordinates and the center point of the candidate element is calculated. If the interaction coordinates are inside the candidate element, the score is added according to the size and position of the candidate element. For example, a base score of 20 to 80 points is assigned according to the area of ​​the element. The smaller the area of ​​the element, the more specific the interaction goal usually is, and the higher the score is; otherwise, the score is deducted (such as subtracting 50 points).

[0108] For example, if the click is inside an element: Spatial dimension score = 20 + 60 (1 - relative distance);

[0109] If the click occurs outside the element: Spatial dimension score = -50.

[0110] The candidate elements are scored based on their depth dimension: the document object model depth (DOM depth value) and the z-index value are obtained. The deeper the document object model, the greater the penalty weight in the score, and the higher the z-index value, the greater the bonus weight.

[0111] For example, depth dimension score = (z-index normalized value) ) - (DOM depth normalization value) ).

[0112] The candidate elements are scored based on their interaction dimensions: the scores are weighted by combining the HTML tag type weight of the candidate elements (for example, button, input, and a tags have very high base scores), native event attributes (such as onclick, onmousedown), and accessibility attributes (such as role equal to button).

[0113] Basic tag weights: such as button / input=15, a(href)=12, div=5, span=2, etc.

[0114] Bonus points for native events: +8 if attributes such as onclick / onchange exist, +0 otherwise;

[0115] ARIA attribute bonus: +5 points if attributes such as role / aria-label exist, otherwise +0;

[0116] Interaction dimension score = tag base weight + native event bonus + ARIA attribute bonus.

[0117] Content-based scoring of candidate elements: Scores are adjusted based on the visible text length and business characteristic attributes of the candidate elements. For example, longer visible text lengths result in higher weightings, and the presence of business placeholders or other specific business attributes adds points. Business characteristic attributes include standard HTML placeholder attributes and custom attributes containing prefixes such as "data-test-" or "data-qa-".

[0118] Visible text length score: for example, using min(visible text characters × 1, 10);

[0119] Business characteristic attributes bonus: +5 points for the presence of attributes such as placeholder / title, otherwise +0;

[0120] Content dimension score = visible text length score + business feature attribute bonus.

[0121] Context-based scoring of candidate elements: By traversing upwards through the parent nodes of the candidate elements, the business container type of the candidate elements is determined. Candidate elements that are within a form, navigation, or table container are given bonus points, while those that are not are not.

[0122] For example, candidate elements located inside a form receive +15, those inside a nav receive +10, those inside a table receive +8, and those in other containers receive +0.

[0123] The score for a candidate element is the result of combining scores from spatial, depth, interaction, content, and context dimensions. For example, weights can be assigned to each scoring item and then summed to obtain the score.

[0124] To filter out distracting elements, a hard penalty mechanism is introduced during the scoring process. Hidden input boxes, disabled elements, and accessible hidden nodes in the candidate element sequence have their scores reduced to zero. Empty elements and broad aggregation containers have their scores reduced by 90%.

[0125] Output the final score after penalty for each candidate element.

[0126] S3. Select the target element based on the scoring result, generate the corresponding unique identifier, and encapsulate it into a capture event containing the target element's interaction type, identifier, timestamp, task ID, and client identifier.

[0127] After scoring, the candidate elements are sorted in descending order of total score. If the ratio of the highest score to the second highest score exceeds a preset threshold (e.g., 1.5 times), the candidate element with the highest score is directly selected as the target element.

[0128] If the ratio does not exceed the judgment threshold, the relative distance and compactness attributes (such as compactness / aspect ratio) of the two candidate elements are compared together, and the final target element is output.

[0129] In this embodiment, the judgment threshold can be dynamically adjusted based on the complexity of the page's DOM (Document Object Model) to avoid misjudgment on simple / complex pages with a fixed threshold. The formula is as follows:

[0130] ;

[0131] in, This is the final judgment threshold; The preset base threshold; This is the adjustment coefficient; This represents the total number of DOM nodes on the current page. This represents the average number of DOM nodes for similar pages as statistically analyzed by the system. , These represent the maximum and minimum number of DOM nodes for pages of the same type.

[0132] The more complex the page (the larger D), the higher the threshold. The higher the value, the less likely it is to cause misjudgment due to dense elements.

[0133] The relative distance can be calculated by normalizing the fit between the centers of two candidate elements to eliminate the influence of element size, or by using Manhattan distance or L2 norm distance, which is not limited here.

[0134] Compactness measures whether an element's shape is a core interactive control (e.g., buttons / input boxes are often compact rectangles, and large containers are often elongated shapes), as shown in the formula below:

[0135] ;

[0136] in, For compactness; , These represent the width and height of the element, respectively.

[0137] A unique identifier is a unique structured identifier generated for the target element (such as an XPath or CSS selector).

[0138] S4. Utilize the change observer interface to monitor changes in the document object model tree where the target element is located, and combine this with a debouncing mechanism to perceive the steady state of the browser page. When the page undergoes structural changes or repainting, wake up the listeners related to the target element, allocate computing power to reselect the target element, encapsulate the capture event, and send the encapsulated capture event to the backend server when the page is stable, so that the backend server can broadcast it to other collaborative clients in the same room for testing.

[0139] To avoid the CPU waste caused by the high-frequency periodic polling of the DOM state in traditional tools, this embodiment utilizes the MutationObserver interface to monitor changes in the page's child node list, attributes, and subtree structure. The system incorporates a debouncing mechanism to detect stable page rendering. Specifically, when a DOM change is detected, the system does not immediately trigger feature extraction logic; instead, it initiates a dynamic time window.

[0140] The debouncing mechanism ensures that continuous changes triggered within the time window (e.g., 300 milliseconds) only execute subsequent logic after the last change is completed. The time window can be dynamically adjusted according to the current system load. When the page is in a high-frequency asynchronous rendering phase, the time window is automatically extended (e.g., extended to 800 milliseconds) to aggregate multiple trivial changes. When the page enters a steady state, the time window is shortened.

[0141] For example, if there are no DOM changes for 300 consecutive ms, the browser page is considered to be in a steady state. If the page undergoes structural changes / repainting, the listener for the target element is activated, front-end computing power is allocated to re-execute S1-S3, and the target element and capture events are updated. If the page is in a steady state: additional computing power allocation stops, and the browser waits for the event to be sent.

[0142] This event-driven, asynchronous, non-blocking capture method allows the browser frontend to consume resources only when necessary, effectively avoiding memory leaks and main thread blocking, and ensuring the smoothness of the interface for users when performing complex testing operations.

[0143] When allocating computing power, the scoring algorithm can be executed asynchronously through Web Worker, or the element's validation logic can be re-explored during browser idle time via requestIdleCallback.

[0144] Once a valid interaction event is captured and a unique identifier is generated, the event is sent to the backend server. In a preferred embodiment, the backend server further includes: constructing logically isolated collaboration rooms based on the task identifier carried in the request, and attaching the current connection to the corresponding collaboration room.

[0145] To address the repetitive work issues common in crowdsourced testing, the backend server integrates the Redis distributed caching engine to perform two-level deduplication logic. The distributed caching engine performs two-level deduplication within a time window, and the deduplicated captured events are then standardized to obtain standardized test content.

[0146] Based on the sender identification exclusion mechanism, the standardized test content is broadcast to other collaborative clients in the same room in real time; the specific method of the secondary deduplication is to compare the received identifier with the historical records in the distributed cache and intercept redundant operations that are triggered repeatedly across tabs; the standardization process includes Chinese description generation and serial number increment allocation.

[0147] Specifically, a sequence of operations within a sliding time window is maintained in Redis for each collaboration room. When a new event arrives, the system compares its identifier, interaction type, and timestamp with the historical records in the cache. If other test engineers in the same room have already performed the exact same operation within a very short time, it is considered a redundant operation and is intercepted. After deduplication verification, natural language processing is automatically invoked to generate a Chinese business description for the step, and an incrementing sequence number is assigned. Subsequently, standardized test case data is stored in a PostgreSQL database. Simultaneously, using the ExcludeID mechanism, the cleaned standardized test steps are broadcast in real-time to all other active clients in the collaboration room via an encrypted full-duplex communication link. After receiving the event, other clients decrypt it and reconstruct the operation based on the target element identifier, achieving real-time collaborative testing.

[0148] This mechanism enables remote testers to synchronize the operation trajectories of others in real time on their browser interfaces, eliminating the generation of duplicate test cases from the data source and significantly improving team collaboration efficiency.

[0149] Considering that crowdsourced testing often involves enterprise intranet addresses and sensitive business data, such as Figure 3 As shown, another preferred embodiment of the present invention further includes: in the browser front-end (e.g., Figure 3 Browser extension A) and backend server (such as Figure 3 End-to-end hybrid encrypted communication is performed between the Go backend server and the Go server; the hybrid encrypted communication includes identity authentication, key negotiation and encrypted transmission, and a three-stage security protection system is constructed.

[0150] like Figure 3 As shown, the event capture module generates the original captured event, then encrypts the event through the ECDH key negotiation of the encrypted event module (CryptoHelper), and then sends it to the backend through the network communication and connection management module (WebSocketManager).

[0151] Among them, such as Figure 4 As shown, during the authentication phase, the browser frontend sends an authentication request based on the HTTPS protocol to the backend server (such as HTTP POST / api / auth / plugin). The request includes an authorization code (auth_code) and an extension identifier (browser_extension_id). After receiving the request, the backend server verifies the validity of the authorization code and extension identifier, and generates a JWT token containing the user identifier (user_id) and user permissions (role) (which can be set to have a validity period of 30 days) and returns it to the browser frontend. The browser frontend receives and stores the JWT token as an identity credential during the key negotiation phase, ensuring the legitimacy of access.

[0152] During the key negotiation phase, the Elliptic Curve Diffie-Hellman Algorithm (ECDH) was employed. Specifically, the browser frontend initiated a WSS verification connection with the backend server based on a JWT token (URL parameters included: token, projectId, testcaseId, and session_id). After the connection was established, the browser frontend sent a handshake message to the backend server, requesting and initiating ECDH key pair generation, and then sent the generated ECDH key pair (publicKey) to the backend server. The backend server received the ECDH key pair, generated an ECDH public key, calculated the shared secret using the ECDH algorithm, and derived the underlying AES-256-GCM symmetric encryption key (deriveKey). This derived key was then sent to the browser frontend, completing the key negotiation. Once the key negotiation was complete, the user officially joined the corresponding test case room.

[0153] This process eliminates the need to transmit private keys over the channel, achieving forward security for a single session. Even if the key for a single session is compromised, the security of historical data will not be threatened.

[0154] During the encrypted transmission phase, when the test engineer triggers an interactive event, the browser frontend encrypts the captured event based on the negotiated key. It can use the AES-256-GCM symmetric encryption algorithm with authentication to process the event, generating ciphertext and an integrity signature verification tag (such as generating a 12-byte random initialization vector IV and a 16-byte authentication tag) and sending it to the backend server.

[0155] After the backend server verifies the integrity signature verification tag, it decrypts the ciphertext, processes the decrypted plaintext event, and then re-encrypts the processed event data before broadcasting it to other collaborating clients in the same room (e.g., ...). Figure 3 Browser extension B).

[0156] If data is found to have been tampered with or signature mismatched during transmission, the connection will be immediately blocked and a security audit alert will be triggered during the authentication phase. This authentication-enabled encryption mechanism not only ensures the confidentiality of test data, but also effectively resists man-in-the-middle attacks and malicious command replay through the additional authentication data AAD verification provided by the GCM mode, providing a financial-grade security barrier for distributed collaborative testing.

[0157] In practical applications, such as when a large financial enterprise's intranet system is undergoing crowdsourced stress testing, multiple test engineers located in different regions simultaneously log into the system. Using this invention, when Engineer A performs a complex money transfer operation, the browser frontend can accurately extract the XPath of the transfer button from a page filled with dynamic pop-ups and transparent overlays using a five-dimensional feature scoring algorithm. Simultaneously, through adaptive scheduling, Engineer A's browser does not experience any lag during the recording process. The captured transfer steps are encrypted using ECDH / AES-GCM and transmitted to the backend server. The backend server identifies the operation as belonging to a transfer task room, performs Redis deduplication, and pushes the step to Engineer B's browser in real time. Engineer B can immediately see Engineer A's progress on their screen, allowing them to switch to testing other functionalities. The entire process not only ensures absolute data security but also, through real-time deduplication and synchronization, automates the manual recording and summarizing work that would otherwise take hours, completing it in milliseconds.

[0158] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0159] This invention replaces the traditional shallow capture method that relies on event.target with front-end penetration positioning technology, thus solving the problem of positioning offset under complex UI components.

[0160] This invention addresses the issue of excessive intrusion of automated tools on user page performance through adaptive resource scheduling based on the MutationObserver interface. Furthermore, it resolves the problems of state fragmentation and data redundancy in distributed testing through backend distributed caching and a two-level deduplication mechanism.

[0161] This invention addresses the security risks of sensitive information leakage in crowdsourcing environments through a three-stage hybrid encryption system.

[0162] In summary, this invention constructs a complete technical closed loop from accurate capture to efficient collaboration and secure transmission, providing reliable technical support for the automated testing of modern web applications.

[0163] Example 2

[0164] like Figure 5As shown, the second embodiment of the present invention also provides a real-time collaborative web test capture system based on multi-feature scoring, including a browser front-end and a back-end server;

[0165] The browser front-end includes a candidate element unit, a five-dimensional feature quantization and scoring unit, a capture event encapsulation unit, and a resource scheduling and sending unit;

[0166] The candidate element unit is used to obtain the interaction coordinates of the user's click, to perform a cyclic penetration probe on the interaction coordinates to obtain the top-level element, and to add the top-level element to the candidate element sequence. The interaction event response attribute of the top-level element is modified to transparent until the termination condition is met, after which the interaction event response attributes of all modified elements are restored, and the candidate element sequence is output.

[0167] The five-dimensional feature quantization scoring unit is used to perform five-dimensional feature quantization scoring on the candidate element sequence and output the score result of each candidate element; wherein, the five-dimensional features include spatial dimension, depth dimension, interaction dimension, content dimension and context dimension;

[0168] The capture event encapsulation unit is used to select target elements according to the scoring results, generate corresponding unique identifiers, and encapsulate them into capture events containing target element interaction type, identifier, timestamp, task ID and client identifier;

[0169] The resource scheduling and sending unit is used to monitor changes in the document object model tree where the target element is located using the change observer interface, and to perceive the steady state of the browser page in conjunction with the debouncing mechanism. When the page undergoes structural changes or repainting, the listener related to the target element is awakened, computing power is allocated to reselect the target element, the capture event is encapsulated, and the encapsulated capture event is sent to the backend server when the page is stable.

[0170] The backend server includes an identity authentication unit and a routing distribution unit;

[0171] The identity authentication unit is used to verify the legitimacy of the browser front-end, and receives the capture event after the verification is valid;

[0172] The routing and distribution unit is used to construct logically isolated collaborative rooms based on the identifiers in the captured events, and then use a distributed caching engine to perform secondary deduplication within a time window. The deduplicated captured events are then standardized, and the standardized test content is broadcast to other collaborative clients in the same room.

[0173] Example 3

[0174] The third embodiment of the present invention also provides a real-time collaborative web test capture device based on multi-feature scoring, which includes a memory and a processor. The memory stores a computer program, which can be executed by the processor to implement the real-time collaborative web test capture method based on multi-feature scoring as described above.

[0175] Example 4

[0176] The fourth embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by the processor of the device where the computer-readable storage medium is located, implement the real-time collaborative web testing capture method based on multi-feature scoring as described above.

[0177] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time collaborative Web test capturing method based on multi-feature score, applied to a browser front end, characterized in that, include: S1, obtain the interaction coordinates of the user's click, perform a loop penetration probe on the interaction coordinates to obtain the top-level element, add the top-level element to the candidate element sequence, modify the interaction event response attribute of the top-level element to transparent, until the termination condition is met, restore the interaction event response attributes of all modified elements, and output the candidate element sequence. S2, perform five-dimensional feature quantization scoring on the candidate element sequence and output the score result for each candidate element; wherein, the five-dimensional features include spatial dimension, depth dimension, interaction dimension, content dimension and context dimension; The five-dimensional feature quantification scoring is specifically as follows: Iterate through each candidate element in the candidate element sequence and initialize the candidate element score to 0; The candidate elements are scored in spatial dimension sequentially: the relative displacement between the interaction coordinates and the center point of the candidate element is calculated; if the interaction coordinates are inside the candidate element, points are added based on the size and position of the candidate element; otherwise, points are deducted. Depth-dimensional scoring of candidate elements: Obtain the document object model depth and hierarchical index attribute of the candidate elements. The deeper the document object model, the greater the penalty weight of the score, and the higher the hierarchical index attribute value, the greater the bonus weight. Candidate elements are scored based on their interaction dimensions: the scores are weighted by combining the HTML tag type weights, native event attributes, and accessibility attributes of the candidate elements. Scoring candidate elements based on content dimensions: adjusting scores according to the visible text length and business characteristic attributes of candidate elements; Context-based scoring of candidate elements: By traversing upwards through the parent nodes of the candidate elements, the business container type of the candidate elements is determined. Candidate elements that are within form, navigation, or table containers are given bonus points, while those that are not are not. The score of a candidate element is the result of a comprehensive evaluation of the spatial dimension, depth dimension, interaction dimension, content dimension, and context dimension. During the scoring process, a hard penalty mechanism is introduced, which resets the score to zero for hidden input boxes, disabled elements and accessible hidden nodes in the candidate element sequence, and reduces the score for empty elements and broad aggregation containers. S3. Select the target element based on the scoring result, generate the corresponding unique identifier, and encapsulate it into a capture event containing the target element's interaction type, identifier, timestamp, task ID, and client identifier; S4. Utilize the change observer interface to monitor changes in the document object model tree where the target element is located, and combine this with a debouncing mechanism to perceive the steady state of the browser page. When the page undergoes structural changes or repainting, wake up the listeners related to the target element, allocate computing power to reselect the target element, encapsulate the capture event, and send the encapsulated capture event to the backend server when the page is stable, so that the backend server can broadcast it to other collaborative clients in the same room for testing.

2. The method of claim 1, wherein the method further comprises: S1 specifically refers to: Obtaining interaction coordinates of a user click ; Initialize and define the candidate element sequence, the number of probed layers, the count of consecutive large containers, and the list of elements with modified styles; The process of cyclic penetration detection is as follows: Call Get the topmost element of the current interaction coordinates; Add the top-level element to the candidate element sequence and determine whether the top-level element is a large container. If it is, increment the consecutive large container count by 1; otherwise, reset the consecutive large container count. Record the original interactive event response attributes of the top-level element, and store the top-level element and its original interactive event response attributes in the list of elements with modified styles; Modify the original interactive event response property of the top-level element to transparent, and increment the number of detected layers by 1 until the termination condition is triggered; Iterate through the list of elements with modified styles, restore the original interactive event response properties for each element, and output the candidate element sequence.

3. The method of claim 2, wherein the method further comprises: The termination conditions are: the topmost element is the root node of the document, or the number of detected layers has reached the preset upper limit, or the continuous large container count value has reached the set large container count threshold. The large container counting threshold is dynamically adjusted based on the proportion of container elements on the page and the density of interactive controls, using the following formula: wherein, is a large container count threshold value; is a base threshold value; , is a weight adjustment coefficient; is a total number of current page container class elements; is a total number of all DOM nodes of the current page; is an interaction control density.

4. The real-time collaborative web test capture method based on multi-feature scoring according to claim 2, characterized in that... It also includes: taking the interaction coordinates as the center, performing extended penetration detection within a preset range to obtain all candidate element sequences, and removing duplicates from all candidate element sequences to obtain deduplicated candidate element sequences.

5. A real-time collaborative web test capture method based on multi-feature scoring according to claim 2, characterized in that... It also includes: After scoring, the candidate elements are sorted in descending order of total score. If the ratio of the highest score to the second highest score exceeds the preset judgment threshold, the candidate element with the highest score is directly selected as the target element. If the ratio does not exceed the judgment threshold, the relative distance and compactness attributes of the two candidate elements are compared together, and the final target element is output. The determination threshold is dynamically adjusted based on the page DOM complexity, and the formula is as follows: ； in, This is the final judgment threshold; The preset base threshold; This is the adjustment coefficient; This represents the total number of DOM nodes on the current page. This represents the average number of DOM nodes for similar pages as statistically analyzed by the system. , These represent the maximum and minimum number of DOM nodes for the same type of page, respectively. The compactness is used to measure whether an element's shape is a core interactive control, and the formula is as follows: ； in, For compactness; , These represent the width and height of the element, respectively.

6. The real-time collaborative web test capture method based on multi-feature scoring according to claim 1, characterized in that... The anti-shake mechanism ensures that, by setting a time window, subsequent logic is only executed after the last change is completed when a change event is triggered consecutively within the time window. The time window is dynamically adjusted according to the current system load. When the page is in a high-frequency asynchronous rendering stage, the time window is automatically extended; when the page enters a steady state, the time window is shortened.

7. The real-time collaborative web test capture method based on multi-feature scoring according to claim 1, characterized in that... It also includes: the backend server constructing logically isolated collaborative rooms based on the identifier in the captured event; A distributed caching engine is used to perform secondary deduplication within a time window. The deduplicated captured events are then standardized to obtain standardized test content. Based on the mechanism of excluding sender identifiers, the standardized test content is broadcast to other collaborative clients in the same room in real time; The specific method of the second-level deduplication is as follows: the received identifier is compared with the historical records in the distributed cache to intercept redundant operations that are triggered repeatedly or across tabs. The standardization process includes generating Chinese descriptions and assigning serial numbers in ascending order.

8. The real-time collaborative web test capture method based on multi-feature scoring according to claim 1, characterized in that... It also includes: performing end-to-end hybrid encrypted communication between the browser front-end and the back-end server; the hybrid encrypted communication includes identity authentication, key negotiation and encrypted transmission; During the authentication phase, the browser frontend sends a request to the backend server, which includes an authorization code and an extended identifier. After receiving the request, the backend server verifies the validity of the authorization code and the extended identifier. Once the verification is successful, the backend server generates a JWT token containing the user identifier and user permissions and returns it to the browser frontend. The browser frontend receives and stores the JWT token as an identity credential during the key negotiation phase. During the key negotiation phase, the browser frontend initiates a WSS verification connection with the backend server based on the JWT token. After the connection is established, the browser frontend sends a handshake message to the backend server, requests and initiates ECDH key pair generation, and sends the generated ECDH key pair to the backend server. The backend server receives the ECDH key pair, generates an ECDH public key, and sends the ECDH public key to the browser frontend, thus completing the key negotiation. During the encrypted transmission phase, the browser frontend encrypts the captured event based on the negotiated key, generates ciphertext and an integrity signature verification tag, and sends them to the backend server. After verifying the integrity signature verification tag, the backend server decrypts the ciphertext, processes the decrypted plaintext event, and then re-encrypts the processed event data before broadcasting it to other collaborating clients in the same room.

9. A real-time collaborative web test capture system based on multi-feature scoring, used to implement the real-time collaborative web test capture method based on multi-feature scoring as described in any one of claims 1-8, characterized in that, Including browser front-end and back-end server; The browser front-end includes a candidate element unit, a five-dimensional feature quantization and scoring unit, a capture event encapsulation unit, and a resource scheduling and sending unit; The candidate element unit is used to obtain the interaction coordinates of the user's click, to perform a cyclic penetration probe on the interaction coordinates to obtain the top-level element, and to add the top-level element to the candidate element sequence. The interaction event response attribute of the top-level element is modified to transparent until the termination condition is met, after which the interaction event response attributes of all modified elements are restored, and the candidate element sequence is output. The five-dimensional feature quantization scoring unit is used to perform five-dimensional feature quantization scoring on the candidate element sequence and output the score result of each candidate element; wherein, the five-dimensional features include spatial dimension, depth dimension, interaction dimension, content dimension and context dimension; The capture event encapsulation unit is used to select target elements according to the scoring results, generate corresponding unique identifiers, and encapsulate them into capture events containing target element interaction type, identifier, timestamp, task ID and client identifier; The resource scheduling and sending unit is used to monitor changes in the document object model tree where the target element is located using the change observer interface, and to perceive the steady state of the browser page in conjunction with the debouncing mechanism. When the page undergoes structural changes or repainting, the listener related to the target element is awakened, computing power is allocated to reselect the target element, the capture event is encapsulated, and the encapsulated capture event is sent to the backend server when the page is stable. The backend server includes an identity authentication unit and a routing distribution unit; The identity authentication unit is used to verify the legitimacy of the browser front-end, and receives the capture event after the verification is valid; The routing and distribution unit is used to construct logically isolated collaborative rooms based on the identifiers in the captured events, and then use a distributed caching engine to perform secondary deduplication within a time window. The deduplicated captured events are then standardized, and the standardized test content is broadcast to other collaborative clients in the same room.

Images