AI-based automatic test case dynamic generation method and system, electronic device and storage medium

Abstract

The application provides an AI-based automatic test case dynamic generation method and system, electronic equipment and storage medium, relates to the field of augmented reality and virtual reality technology, and obtains a gaze track and a gesture sequence through user behavior collection and processing to integrate and generate user interaction behavior information; the interaction behavior information is subjected to AI-driven behavior correlation construction processing to generate behavior interaction feature information; subpixel response delay information is generated through phase shift capture processing of a display screen polarization state, and abnormal rendering area information is identified accordingly; potential interaction intention information is generated through AI backtracking processing by combining the behavior interaction feature information and the abnormal rendering area information; and the intention information is taken as a dynamic generation target to dynamically construct an automatic test case and generate a dynamic test case set. Unknown interaction defect scenarios can be actively covered by fusing user interaction behavior and physical-level rendering anomaly detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality and virtual reality technology, in particular to an AI-based automatic test case dynamic generation method and system, an electronic device and a storage medium. BACKGROUND

[0002] In the field of augmented reality and virtual reality application testing, with the diversification of interactive modes, scenarios in which users trigger non-standard operation paths through gaze tracks and gesture sequences are increasing. Such unknown interaction paths often cause logical conflicts due to physical layer abnormalities such as interface rendering ghosting obstruction and operation delay mutation. Traditional testing methods based on function coverage cannot actively discover such defects.

[0003] The current mainstream solution uses user operation log playback technology to construct a test script library by recording real user operation sequences, and uses trajectory clustering algorithms to filter high-frequency paths for automatic regression testing. Although this solution can reuse historical interaction data, it has the following defects:

[0004] First, log playback is completely limited to existing operations and cannot infer potential paths that users intended to execute but were interrupted due to physical abnormalities. Second, it lacks the ability to perceive hardware-level defects such as rendering ghosting, resulting in a mismatch between test scripts and real physical environments. Third, the clustering process ignores high-attention but non-triggered intent areas in gaze tracks, leading to missed key defect scenarios. SUMMARY

[0005] The present application aims to provide an AI-based automatic test case dynamic generation method and system, an electronic device and a storage medium to solve the problem that passive playback mode in the prior art cannot cover potential interaction paths interrupted by physical abnormalities.

[0006] To solve the above technical problems, in a first aspect, the present application provides an AI-based automatic test case dynamic generation method, comprising:

[0007] Based on user behavior collection and processing, gaze tracks and gesture sequences in augmented reality and virtual reality applications are obtained, and user interaction behavior information is generated by integrating the gaze tracks and the gesture sequences;

[0008] The user interaction behavior information is subjected to AI-driven behavior correlation construction processing to generate behavior interaction feature information;

[0009] The display screen polarization state of the AI is subjected to phase shift capture processing to generate sub-pixel response delay information;

[0010] The rendering abnormal area of the sub-pixel response delay information is identified to generate rendering abnormal area information;

[0011] The AI back reasoning processing is performed in combination with the behavior interaction feature information and the rendering abnormal area information, and potential interaction intention information is generated.

[0012] The dynamic test case set is generated dynamically by taking the potential interaction intention information as a dynamic generation target.

[0013] Optionally, the AI back reasoning processing is performed in combination with the behavior interaction feature information and the rendering abnormal area information, and potential interaction intention information is generated, including:

[0014] The implicit causal relationship between the user operation delay of the behavior interaction feature information and the visual attention degree of the behavior interaction feature information is constructed and processed, and behavior causal structure information is generated;

[0015] The virtual object display state in the behavior causal structure information is intervened and injected with conditions in combination with the rendering abnormal area information, and anti-intervention parameters are generated;

[0016] The operation result of the gesture operation not performed under the rendering abnormal area is deduced and processed by using the anti-intervention parameters, and anti-operation result information is generated;

[0017] Based on the anti-operation result information, the potential interaction intention of the user is described and reconstructed, and potential interaction intention information is generated.

[0018] Optionally, the AI-driven behavior association construction processing is performed on the user interaction behavior information, and behavior interaction feature information is generated, including:

[0019] The asynchronous coupling processing is performed on the pause interval in the gaze trajectory of the user interaction behavior information and the operation conflict in the gesture sequence of the user interaction behavior information, and behavior contradiction focus information is generated;

[0020] Based on the behavior contradiction focus information, the visual attention accumulation area in the gaze trajectory and the operation delay burst period in the gesture sequence are intention hesitation quantification processed, and interaction hesitation feature information is generated;

[0021] In combination with the interaction hesitation feature information, the hotspot area in the gaze trajectory that does not trigger the gesture is marked and processed with intention residue, and potential intention marking information is generated;

[0022] The potential intention marking information and the behavior contradiction focus information are used to perform topological reconstruction processing on the interaction intention differentiation path of the user, and behavior interaction feature information is generated.

[0023] Optionally, the dynamic test case set is generated dynamically by taking the potential interaction intention information as a dynamic generation target, including:

[0024] performing spatial coordinate conversion processing on a virtual operation region to which the potential interaction intention information is directed, to generate intention spatial positioning information;

[0025] performing spatial variation processing on an operation path in the intention spatial positioning information in combination with the rendering abnormal region information, to generate variation operation path information;

[0026] performing conflict injection processing on a gesture action sequence in the potential interaction intention information by using the variation operation path information, to generate a conflict-enhanced gesture sequence;

[0027] dynamically synthesizing test case execution logic based on the conflict-enhanced gesture sequence and the variation operation path information, to generate a dynamic test case set.

[0028] Optionally, in combination with the rendering abnormal region information, performing intervention condition injection processing on a virtual object display state in the behavior causal structure information, to generate an anti-intervention parameter, including:

[0029] performing physical interference quantification processing on an optical distortion degree of a ghost region in the rendering abnormal region information, to generate a ghost interference intensity parameter;

[0030] performing state reconstruction processing on an operation delay in the behavior causal structure information and a causal link of visual attention degree in the behavior causal structure information by using the ghost interference intensity parameter, to generate distortion causal link information;

[0031] performing dynamic simulation processing on a visibility decay feature of the virtual object display state in the ghost region in combination with the distortion causal link information, to generate a display state decay model;

[0032] performing intervention correction processing on operation feasibility of the unexecuted gesture operation under distortion causal constraints based on the display state decay model, to generate an anti-intervention parameter.

[0033] Optionally, performing topological reconstruction processing on an interaction intention differentiation path of the user by using the potential intention mark information and the behavior contradiction focus information, to generate behavior interaction feature information, including:

[0034] performing intention splitting processing on an untriggered hotspot region in the potential intention mark information and an operation conflict region in the behavior contradiction focus information, to generate split intention node information;

[0035] performing repulsion strength calculation processing on an operation detour path around the conflict region in the gaze trajectory based on the split intention node information, to generate a path repulsion parameter;

[0036] Utilize the path rejection parameter, the connection relationship between the split intention node information is dynamically attenuated, and intention connection attenuation information is generated.

[0037] In combination with the intention connection attenuation information, the spatial topological structure of the interaction intention differentiation path is evolutionarily simulated, and reconstruction topological path information is generated.

[0038] Based on the reconstruction topological path information, the differentiation characteristics of the user's behavior mode are structurally described, and behavior interaction feature information is generated.

[0039] Optionally, utilize the anti-intervention parameter, the operation result of the unexecuted gesture operation under the rendering abnormal area is deduced, and anti-operation result information is generated, including:

[0040] Based on the anti-intervention parameter, the interaction response rule of the virtual object in the rendering abnormal area is constrained and reconstructed, and abnormal interaction constraint information is generated;

[0041] Utilize the abnormal interaction constraint information, the deformation process of the trajectory form of the unexecuted gesture operation in the rendering abnormal area is dynamically simulated, and gesture deformation trajectory information is generated;

[0042] In combination with the gesture deformation trajectory information, the state transition path of the virtual object under abnormal constraint is evolutionarily deduced, and object state evolution information is generated;

[0043] The critical conditions of operation interruption and error triggering in the object state evolution information are boundary positioned, and operation failure boundary information is generated;

[0044] Utilize the operation failure boundary information, the final interaction result of the unexecuted gesture operation in the rendering abnormal area is completely deduced, and anti-operation result information is generated.

[0045] Secondly, the application provides an AI-based automatic test case dynamic generation system, comprising:

[0046] The behavior acquisition module is used for acquiring the gaze trajectory and gesture sequence in the augmented reality and virtual reality application based on the user's behavior acquisition processing, integrating the gaze trajectory and the gesture sequence, and generating user interaction behavior information;

[0047] The association construction module is used for AI-driven behavior association construction processing of the user interaction behavior information, and generates behavior interaction feature information;

[0048] The polarization capture module is used for phase shift capture processing of the display screen polarization state of AI, and generates sub-pixel response delay information;

[0049] an abnormality identification module, configured to identify a rendering abnormal area of the sub-pixel response delay information, and generate rendering abnormal area information;

[0050] an intention deduction module, configured to combine the behavior interaction feature information and the rendering abnormal area information, and perform AI reverse deduction processing to generate potential interaction intention information;

[0051] a use case generation module, configured to take the potential interaction intention information as a dynamic generation target, construct an automatic test case dynamically, and generate a dynamic test case set.

[0052] In a third aspect, the present application provides an electronic device, comprising:

[0053] a memory, configured to store a computer program;

[0054] a processor, configured to implement the steps of the AI-based automatic test case dynamic generation method according to the first aspect when the computer program is executed.

[0055] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program can implement the steps of the AI-based automatic test case dynamic generation method according to the first aspect when the computer program is executed by a processor.

[0056] The AI-based automatic test case dynamic generation method provided by the present application can obtain a gaze trajectory and a gesture sequence in an augmented reality and virtual reality application through behavior collection and processing based on a user, integrate the gaze trajectory and the gesture sequence to generate user interaction behavior information, perform AI-driven behavior correlation construction processing on the user interaction behavior information to generate behavior interaction feature information, perform phase shift capture processing on a display screen polarization state of AI to generate sub-pixel response delay information, identify a rendering abnormal area of the sub-pixel response delay information to generate rendering abnormal area information, combine the behavior interaction feature information and the rendering abnormal area information to perform AI reverse deduction processing to generate potential interaction intention information, and take the potential interaction intention information as a dynamic generation target to construct an automatic test case dynamically and generate a dynamic test case set. The technical solution of the present application has the following beneficial effects:

[0057] The user interaction behavior information is generated by integrating the gaze track and the gesture sequence, and a multi-modal interaction data base is constructed. The AI-driven processing is used to actively construct the implicit correlation between visual attention and operation delay from the interaction behavior, and form quantifiable behavior characteristics. The sub-pixel level response delay information is captured and generated based on the display screen polarization phase shift, and the physical layer perception of the hardware rendering defects is realized. The abnormal area in the response delay is identified, and the rendering ghosting defect of the virtual object is accurately located. The potential interaction path of the user intention execution but due to the physical interruption is revealed by the back deduction combining the behavior characteristics and the rendering abnormality. Finally, the unconventional operation test case is dynamically constructed using the potential intention, and the defect scenarios that cannot be reached by the traditional method are actively covered.

[0058] Further, the behavior causal structure is constructed based on the implicit causal relationship between the operation delay and the visual attention in the behavior interaction characteristic information, the anti-parameters are generated by injecting intervention conditions to the display state of the virtual object combining the rendering abnormal area, the operation result of the unexecuted gesture in the abnormal area is deduced using the parameters, and finally the potential interaction intention information is reconstructed. The dependence on the occurred operation in the traditional log playback is broken, the operation path corresponding to the interruption intention is actively generated through the back deduction under the physical abnormal intervention, and the test environment mismatch problem caused by the rendering defect is solved; the hardware level abnormality is converted into a causal constraint condition, the operation failure boundary is located in the state evolution of the virtual object, and the precise coverage of the unknown defect scene of the augmented reality and the virtual reality is realized. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0060] Figure 1 A flowchart of an AI-based automatic test case dynamic generation method provided by an embodiment of the present application;

[0061] Figure 2 A specific implementation schematic diagram of an AI-based automatic test case dynamic generation method provided by an embodiment of the present application;

[0062] Figure 3 A specific implementation schematic diagram of an AI-based automatic test case dynamic generation method provided by an embodiment of the present application;

[0063] Figure 4 A structure schematic diagram of an AI-based automatic test case dynamic generation system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0064] Research found that in augmented reality and virtual reality application testing, when users trigger non-standard interaction paths through gaze track and gesture sequence, logical conflicts are often caused by physical layer abnormalities such as interface rendering ghost blocking, operation delay mutation, etc. Traditional testing methods based on function coverage cannot actively capture such defects, while existing automation solutions use user operation log playback technology to generate test scripts by clustering high-frequency paths, but there are fundamental limitations: they completely rely on existing operation records and cannot infer potential user intentions interrupted by physical abnormalities; they lack the ability to perceive hardware-level defects such as display screen rendering ghost, causing the test environment to deviate from the real physical state; and they ignore high-attention but non-triggered intention areas in gaze track, resulting in missed detection of key defect scenarios.

[0065] To solve the above problems, the present application proposes an AI-based automated test case dynamic generation method. Specifically, the optical property changes of the display screen are captured by polarization imaging technology to identify the rendering ghost area, an action correlation model is constructed based on user gaze and gesture data, and the potential interaction intention interrupted by physical abnormalities is actively reconstructed using back-inference technology. Finally, a test case set containing irregular operation paths is dynamically generated based on this intention, fundamentally solving the potential path missing problem caused by physical abnormalities: achieving collaborative perception of hardware rendering defects and software behavior, breaking through the limitations of passive playback of historical operations, and significantly improving the active defect discovery capability of augmented reality and virtual reality applications in dynamic virtual-real fusion environments.

[0066] To better enable those skilled in the art to understand the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0067] The core of the present application is to provide an AI-based automated test case dynamic generation method, and a specific embodiment of the method is shown in the flowchart as Figure 1 The method comprises:

[0068] S101, based on user behavior collection and processing, obtaining gaze track and gesture sequence in augmented reality and virtual reality applications, integrating the gaze track and the gesture sequence, and generating user interaction behavior information;

[0069] In this step, the gaze track refers to the path data of the user's eyeball focus moving on the screen;

[0070] The gesture sequence refers to the record formed by the user's finger operation action in space in time sequence;

[0071] The user interaction behavior information refers to a complete data set describing the user operation intention formed by spatiotemporal correlation and integration of the gaze track and the gesture sequence.

[0072] In the embodiment of the application, first, the infrared eye tracker captures the time points and coordinate positions of the user's gaze on different areas of the screen to form the original gaze path; second, the depth camera collects the moving direction and touch action of the user's finger in the three-dimensional space to form the basic gesture record; then, the time synchronization mechanism is established to align the timestamps of the gaze path with the time axis of the gesture action; finally, the two types of data are fused in the space-time dimension to generate the user interaction behavior information stream containing the operation focus and action correlation.

[0073] In an actual case, in a virtual reality education application, the user continuously gazes at the chemical experiment equipment icon but does not perform the grabbing action, and the system integrates the gaze track and the missing gesture sequence to generate the interaction behavior information reflecting the operation contradiction.

[0074] S102, AI-driven behavior correlation construction processing is performed on the user interaction behavior information to generate behavior interaction feature information;

[0075] In this step, the behavior interaction feature information refers to an implicit correlation model between the visual attention intensity and the operation response delay mined from the user interaction behavior information by artificial intelligence technology.

[0076] In the embodiment of the application, first, the user interaction behavior information is scanned to extract the visual focus area in the gaze track with a stay duration exceeding a regular threshold; second, the delay period of the operation instruction before the burst in the gesture sequence is detected; then, the overlap ratio of the high visual attention area and the operation delay period in the spatial position is calculated; finally, the neural network is used to analyze the internal correlation between the visual attention accumulation and the operation delay burst to construct a quantifiable behavior interaction feature model.

[0077] Continuing the above case, the system detects the high-intensity gaze of the user on the dangerous chemical equipment icon and the significant delay of the grabbing action to generate the behavior interaction feature information that the recognition of the danger of the equipment causes the operation hesitation.

[0078] S103, phase shift capture processing is performed on the polarization state of the display screen of the AI to generate sub-pixel response delay information;

[0079] In this step, the sub-pixel response delay information refers to the pixel unit rendering response time delay data caused by the abnormal polarization state of the liquid crystal molecules of the display screen.

[0080] In the embodiment of the application, first, a linearly polarized light source of a specific wavelength is projected onto the surface of the display screen; second, the polarization phase shift of the reflected light is received through the photonic crystal sensor array; third, the instantaneous rate of change of the polarization angle of each sub-pixel unit is calculated; and finally, a sub-pixel level heat map reflecting the rendering delay degree of the local area of the screen is generated.

[0081] Continuing the above case, in the augmented reality navigation interface, the polarization imaging device detects that the crossroad sign area has abnormal phase shift, and generates sub-pixel response delay information of the area.

[0082] S104, identify the rendering abnormal area of the sub-pixel response delay information, and generate rendering abnormal area information;

[0083] In this step, the rendering abnormal area information refers to a spatial position data set of a virtual object display ghost caused by sub-pixel response delay exceeding a safe threshold.

[0084] In the embodiment of the application, first, a physically tolerable response delay threshold is set; second, a continuous over-limit pixel block in the sub-pixel response delay information is scanned; third, the virtual object contour boundary covered by the delay block is marked; and finally, a rendering abnormal area coordinate set describing the abnormal position and range of the ghost is generated.

[0085] Continuing the above case, the system of the previous example generates rendering abnormal area information of the crossroad sign ghost blocking the turning arrow of the crossroad sign area.

[0086] S105, in combination with the behavior interaction feature information and the rendering abnormal area information, AI backtracking processing is performed to generate potential interaction intent information;

[0087] In this step, the potential interaction intent information refers to an operation target description that the user may perform but is interrupted, which is reconstructed based on the physical abnormal constraint condition through backtracking technology.

[0088] In the embodiment of the application, first, the behavior interaction feature information is analyzed to establish a causal association chain of visual attention intensity and operation response delay; second, the spatial coordinates of the rendering abnormal area are injected into the causal chain as external intervention factors; third, the complete path of the gesture operation after eliminating the rendering abnormal area is simulated; and finally, an operation intent description framework that the user may complete in a barrier-free environment is deduced.

[0089] Continuing the above case, the system of the previous example deduces that the potential interaction intent information of the user to view traffic information by performing a sliding operation to view the road conditions of the turning intersection if there is no crossroad sign ghost.

[0090] S106, taking the potential interaction intent information as a dynamic generation target, constructing an automated test case dynamically, and generating a dynamic test case set.

[0091] In this step, the dynamic test case set refers to an executable test script set containing unconventional operation instruction sequences constructed with potential interaction intent as the generation target.

[0092] In the embodiments of the present application, first, the potential interaction intent information is parsed into a spatial coordinate operation sequence; second, a path distortion variable is injected within the rendering abnormal area coordinate range; then, an operation instruction stream that fuses abnormal gesture combinations and unconventional trajectories is generated; and finally, it is compiled into an automated test case set that can trigger the target scene.

[0093] Continuing the above case, the system constructs a test instruction set for triggering the road condition panel by quickly sliding three times in the intersection identification residual image area to verify the interface logic collapse defect caused by residual image obstruction.

[0094] To sum up, S101 to S106 break through the path dependence of traditional testing on historical operations through the deep collaboration of multi-source behavior data collection and physical signal perception, fuse gaze focus movement and gesture action sequence at the data collection layer to build a user intent model, use polarization phase shift capture technology to identify hardware rendering defects at the physical perception layer, and use the anti-intervention mechanism to reconstruct the potential operation path that is physically interrupted at the analysis and deduction layer. Finally, a test case set covering unknown defects in virtual-real fusion scenes is dynamically generated. This method fundamentally solves the problem of missed detection of interaction logic caused by rendering abnormalities in augmented reality and virtual reality applications, and significantly improves the active discovery capability of defects in dynamic interaction scenes.

[0095] In order to solve the problem of potential intent recognition in the physical interruption scene, in some embodiments, according to S105, the AI back-deduction processing is performed in combination with the behavior interaction feature information and the rendering abnormal area information to generate potential interaction intent information, including:

[0096] S201, the implicit causal relationship between the user operation delay of the behavior interaction feature information and the visual attention degree of the behavior interaction feature information is constructed and processed to generate behavior causal structure information;

[0097] In S201, the behavior causal structure information refers to a directional dependence network established by analyzing the spatial distribution of the visual attention accumulation area and the time sequence association of the operation delay burst period.

[0098] In the embodiments of the present application, first, the gaze trajectory in the behavior interaction feature information is scanned to extract the visual focus block coordinates that exceed the conventional stay time; second, the delay period and its spatial positioning before the operation instruction is issued in the gesture sequence are detected; then, the spatial overlap degree of the visual focus block and the operation delay area is calculated; finally, a directional dependence network in which the visual attention intensity change drives the operation delay response is constructed through a causal discovery algorithm.

[0099] S202, in combination with the rendering exception area information, intervene condition injection processing is performed on the virtual object display state in the behavior causal structure information, and anti-intervention parameters are generated;

[0100] In S202, the anti-intervention parameters refer to converting the physical distortion characteristics of the rendering exception area into state intervention variables of the causal network nodes.

[0101] In the embodiments of the present application, first, the virtual object display nodes in the behavior causal structure information that overlap with the rendering exception area are located; second, the attenuation coefficient of the optical distortion degree of the exception area on the object visibility is analyzed; then the attenuation coefficient is mapped to a correction factor of the causal dependence strength; finally, the state transfer function of the target node is reconstructed to generate the intervention parameter set.

[0102] S203, using the anti-intervention parameters, the operation result under the rendering exception area without executing the gesture operation is deduced to generate anti-operation result information;

[0103] In S203, the anti-operation result information refers to simulating the virtual object state transition path triggered by the missing gesture under the physical intervention constraint.

[0104] In the embodiments of the present application, first, the object state evolution rule under the constraint of the anti-intervention parameters is loaded; second, the virtual gesture starting coordinates are set on the boundary of the rendering exception area; then the gradual change trajectory of the object display attribute in the gesture movement process is deduced according to the state transfer rule; finally, the complete path characteristics when the object state reaches the critical failure threshold are recorded.

[0105] S204, based on the anti-operation result information, a description reconstruction processing is performed on the potential interaction intention of the user to generate potential interaction intention information.

[0106] In S204, the potential interaction intention information refers to the user expected operation target description framework deduced according to the object state transition path.

[0107] In the embodiments of the present application, first, the termination form characteristics of the object state transition in the anti-operation result information are analyzed; second, the intention description template in the pre-defined interaction scene is matched; then the intention execution strength parameter is corrected according to the path turning point; finally, the structured intention description is generated by fusing the operation object identifier and the target action.

[0108] The following is a specific example:

[0109] In the VR education experiment scene, the user continuously gazes at the corrosive chemical container (visual attention accumulation area) but does not perform the pouring operation (operation delay burst period), and the system establishes a causal dependence network of "attention to the container causing operation hesitation"; after detecting that there is a rendering residual image in the container label area, the decay coefficient of the residual image causing a 60% decrease in label visibility is calculated, which is converted into a causal intervention parameter; the pouring gesture is simulated at the boundary of the residual image, and the path of the dose exceeding the standard caused by the invisible label when the liquid is injected into the test tube is deduced; according to the flow rate mutation characteristics before the liquid overflow, the intention template of "accurately adding 50ml solution" is matched, and the potential interactive intention of "pouring corrosive solution into standard scale" is reconstructed.

[0110] In summary, S201 to S204 first convert the visual operation contradiction into a calculable dependence network by establishing the causal intervention mechanism of the behavior characteristics and the physical defects, secondly quantify the hardware rendering anomaly as a network node intervention variable, then deduce the object failure path of the unperformed operation under the physical constraint, and finally reconstruct the operation target interrupted by the user. The process realizes the active mining of potential intention in the physical anomaly scene, and provides a core basis for generating test cases of virtual-real fusion defects.

[0111] In order to deeply analyze the implicit decision conflict in the user interaction behavior, in some embodiments, according to S102, the AI-driven behavior association construction processing is performed on the user interaction behavior information to generate behavior interaction feature information, including:

[0112] S301, asynchronously coupling the pause interval in the gaze trajectory of the user interaction behavior information and the operation conflict in the gesture sequence of the user interaction behavior information to generate behavior contradiction focus information;

[0113] In S301, the behavior contradiction focus information refers to a set of associated abnormal points formed in the time and space dimensions by the visual focus stay period exceeding the regular duration in the gaze trajectory and the sudden operation interruption action in the gesture sequence.

[0114] In the embodiments of the present application, first, the gaze trajectory data stream in the user interaction behavior information is analyzed frame by frame, the visual focus stay period formed by continuous multiple frames of line of sight locking the same space coordinate is detected, and the start and end time stamps and the space coordinate range of the period are marked; secondly, the gesture sequence data stream is traversed, the operation interruption area formed by the sudden stop or direction mutation of the operation instruction execution process is identified, and the space coordinates and duration of the interruption action are extracted; then, a space-time mapping coordinate system is established, the time window of the visual focus stay period and the time window of the operation interruption area are aligned, and the Euclidean distance of the space coordinates and the time sequence offset are calculated; finally, the associated points with the spatial distance exceeding the preset threshold and the time sequence offset lower than the critical value are screened, and a set of behavior contradiction focus information with strong space-time correlation is aggregated and generated.

[0115] S302, based on the behavior contradiction focus information, the visual attention accumulation area in the gaze trajectory and the operation delay burst period in the gesture sequence are subjected to intention hesitation quantification processing, and interaction hesitation feature information is generated;

[0116] In S302, the interaction hesitation feature information refers to the operation decision uncertainty quantification atlas generated by coupling calculation of the spatial heat distribution of the visual attention accumulation area and the duration of the operation delay burst period.

[0117] In the embodiments of the present application, first, the gaze density distribution graph corresponding to each visual focus stay period in the behavior contradiction focus information is extracted, and a visual attention intensity heat map based on spatial coordinates is generated. Secondly, the operation delay burst period overlapping with the focus stay period in the gesture sequence is located, and the delay start and end time and the delay action space range are accurately calibrated. Then, in the spatio-temporal fusion coordinate system, the spatial overlap coverage of the visual attention intensity heat map and the operation delay area is calculated, and the hesitation degree basic value is calculated by weighting the duration of the delay burst period. Finally, the multi-layer perceptron model is used to perform nonlinear transformation on the basic value, and the interaction decision uncertainty quantification atlas with spatial distribution characteristics is output.

[0118] S303, in combination with the interaction hesitation feature information, the hotspot area in the gaze trajectory that does not trigger the gesture is subjected to intention residual marking processing, and potential intention marking information is generated;

[0119] In S303, the potential intention marking information refers to the visual focus spatial coordinate set in the gaze trajectory that is not associated with the effective gesture operation filtered based on the high hesitation quantification atlas.

[0120] In the embodiments of the present application, first, the quantification atlas in the interaction hesitation feature information is scanned, the peak area exceeding the decision uncertainty threshold is identified and its spatial boundary is extracted. Secondly, the original line of sight coordinate sequence in the corresponding time window in the gaze trajectory data stream is traced back, and the spatial movement path of the line of sight focus in this period is reconstructed. Then, a dynamic time window is set on the reconstructed path, and it is detected whether each focus area produces an effective gesture trigger signal within the window period. Finally, the high-attention focus area that does not detect the gesture trigger is marked with spatial coordinates, and a set of spatial marker points of residual intention is formed.

[0121] S304, using the potential intention marking information and the behavior contradiction focus information, the interaction intention differentiation path of the user is subjected to topological reconstruction processing, and behavior interaction feature information is generated.

[0122] In S304, the behavior interaction feature information refers to the interaction decision differentiation network containing the main operation path and the potential branch path reconstructed by fusing the intention residual marker points and the behavior contradiction focus points.

[0123] In the embodiments of the present application, firstly, the spatial coordinate point set of potential intention marking information is mapped into a decision node in a topological network; secondly, the time-space offset parameters in the behavior contradiction focus information are extracted, and the strength weight value of the connection edge between nodes is calculated; then, the network bifurcation point corresponding to the hesitation degree peak value area in the interactive hesitation characteristic information is identified; finally, according to the distribution of the decision node, the weight of the connection edge and the position of the bifurcation point, an interactive intention differentiation network model containing the main operation path and the potential alternative path is reconstructed.

[0124] The following is a specific example:

[0125] In the augmented reality industrial maintenance scene, the maintenance personnel continuously gaze at the equipment fault indicator light area to form an ultra-long visual focus stay period, and the gesture operation suddenly stops when executing the disassembly instruction; the system detects the time-space abnormal point to generate the behavior contradiction focus information; the spatial overlap rate of the visual attention heat map of the fault light area and the delay period of the disassembly action reaches 85%, and a high hesitation quantization atlas is output; it is found through backtracking that no gesture operation is triggered in this area, which is marked as a potential intention coordinate point; finally, a differentiation decision network of the main path to continue disassembly and the potential path of the fault light is reconstructed, forming the behavior interaction characteristic information reflecting the operation contradiction of the maintenance personnel.

[0126] In summary, S301 to S304 realize the deep mining of implicit behavior characteristics through a four-stage progressive processing mechanism. Firstly, the time-space contradiction points of gaze and gesture are accurately captured; secondly, a coupling quantization model of visual attention and operation delay is established; then, the intention residual area that is not triggered by operation is located; finally, a decision differentiation network is reconstructed; a complete conversion link from original behavior data to computable feature structure is formed, breaking through the dependence of traditional methods on explicit behavior correlation, and providing high-value behavior pattern input for backtracking.

[0127] In order to construct a test case set that can trigger a physical defect scene, in some embodiments, according to S106, the potential interactive intention information is taken as a dynamic generation target to construct an automated test case dynamically, and a dynamic test case set is generated, including:

[0128] S401, performing spatial coordinate conversion processing on a virtual operation area pointed by an intention in the potential interactive intention information, to generate intention spatial positioning information;

[0129] In S401, the intention spatial positioning information refers to a three-dimensional space key point sequence formed by mapping an operation target object identifier in the potential interactive intention description framework to a virtual scene coordinate system, which includes an accurate coordinate set of a starting point, a path turning point and an end point.

[0130] In the embodiments of the present application, first, the operation target object feature identifier in the potential interaction intention information is analyzed; second, the three-dimensional model surface topology of the target object is obtained by searching the virtual scene space database; then, the geometric center point and boundary feature point coordinates of the model interactive region are calculated; and finally, the coordinate sequence containing the spatial position and orientation vector is generated in the order of the operation path.

[0131] In S402, the variant operation path information refers to the non-continuous spatial trajectory after the original path is subjected to deformation disturbance based on the optical distortion parameters of the rendering abnormal area, which retains the original path topology but introduces path distortion caused by physical defects.

[0132] In S402, the variant operation path information refers to the non-continuous spatial trajectory after the original path is subjected to deformation disturbance based on the optical distortion parameters of the rendering abnormal area, which retains the original path topology but introduces path distortion caused by physical defects.

[0133] In the embodiments of the present application, first, the operation path of the intention space positioning information is detected in the spatial overlapping section with the rendering abnormal area; second, the distortion gradient distribution and direction vector field of the abnormal area are extracted; then, the deformation disturbance intensity and direction offset angle are calculated in the path overlapping section; and finally, the deformation disturbance is applied along the path normal direction to generate a twisted trajectory with a fracture zone and a curvature mutation.

[0134] In S403, the conflict-enhanced gesture sequence refers to the antagonistic adjustment instruction stream of the spatial twist features of the variant path converted into gesture action parameters, which contains the speed mutation, action force reversal and trajectory oscillation features.

[0135] In S403, the conflict-enhanced gesture sequence refers to the antagonistic adjustment instruction stream of the spatial twist features of the variant path converted into gesture action parameters, which contains the speed mutation, action force reversal and trajectory oscillation features.

[0136] In the embodiments of the present application, first, the deformation parameters of the variant operation path information are decomposed into the curvature change rate and the fracture gap value; second, the basic gesture force, speed and direction parameters in the potential interaction intention information are analyzed; then, the reverse action force vector and the speed attenuation coefficient are inserted at the path deformation peak point; and finally, the gesture action control sequence containing multiple conflict instructions is reconstructed.

[0137] In S404, the dynamic test case set refers to the executable test script set fused with the conflict gesture instructions and the variant spatial path, which encapsulates the environment initial state, operation instruction sequence and expected failure condition.

[0138] In S404, the dynamic test case set refers to the executable test script set fused with the conflict gesture instructions and the variant spatial path, which encapsulates the environment initial state, operation instruction sequence and expected failure condition.

[0139] In the embodiments of the present application, firstly, a synchronization mapping relationship between the timestamps of the conflict-enhanced gesture sequence and the spatial coordinates of the variant operation path is established; secondly, a spatiotemporal coupled operation event stream is generated when the gesture conflict instruction is bound at the path turning point; then, the event stream is compiled into a binary instruction sequence that can be parsed by the device; finally, the initial state parameters of the virtual scene are packaged to generate a complete test case set.

[0140] The following is a specific example:

[0141] In the AR device maintenance training scene, the system parses the potential interactive intent of "disassembling the overheated engine bolt", maps the bolt model to a three-dimensional space operation coordinate sequence; after detecting that there is a rendering residual image on the surface of the bolt, the disassembly path is distorted into an irregular trajectory with a broken belt according to the residual image distortion gradient; the reverse screwing force and operation pause instruction are injected at the path breaking point to generate a conflict gesture sequence containing sudden reverse rotation; finally, the test case of "executing the conflict gesture along the broken path to trigger the tool jam" is synthesized, and the virtual tool system crash defect caused by rendering abnormalities is verified.

[0142] In summary, S401 to S404 realize the active construction of test cases through a four-stage dynamic transformation mechanism. First, the intent description is mapped to an operable space path; second, physical deformation disturbance is applied to the path based on hardware defect characteristics; third, the spatial deformation is converted into antagonistic conflict of gesture action; finally, the executable test script is generated by fusing space-time parameters; a complete closed loop from user intent to defect triggering is formed, breaking the dependence of traditional test case generation methods on preset rules, and significantly improving the defect detection rate of physical abnormal scenes.

[0143] In order to accurately model the intervention mechanism of hardware defects on interactive logic, in some embodiments, according to S202, the display state of the virtual object in the behavior causal structure information is intervened and injected with conditions in combination with the rendering abnormal area information, to generate anti-intervention parameters, including:

[0144] S501, quantifying the optical distortion degree of the residual image area in the rendering abnormal area information to generate a residual image interference intensity parameter;

[0145] In S501, the residual image interference intensity parameter refers to the visibility attenuation coefficient of the virtual object calculated based on the phase shift amplitude and distribution density of polarized light, which comprehensively reflects the contour blurring and color distortion degree caused by light distortion.

[0146] In the embodiments of the present application, first, the polarization phase offset heat map of the residual image block in the rendering abnormal area information is extracted; second, the maximum deviation value of the phase offset angle relative to the normal display reference is measured; then, the spatial gradient distribution of the offset angle is analyzed to calculate the contour edge blur radius; at the same time, the color deviation amplitude caused by the phase offset is detected; finally, the edge blur radius and the color deviation amplitude are fused to generate the normalized optical distortion interference strength value.

[0147] In S502, the distortion causal link information refers to a modified version of the state-dependent rule set formed after the residual image interference strength is injected as an external intervention variable into the original causal transfer function.

[0148] In S502, the distortion causal link information refers to a modified version of the state-dependent rule set formed after the residual image interference strength is injected as an external intervention variable into the original causal transfer function.

[0149] In the embodiments of the present application, first, the causal nodes in the behavior causal structure information that overlap with the residual image area in space are located; second, the state transition equation of the change of visual attention degree to operation delay response in the original causal link is analyzed; then, the residual image interference strength parameter is mapped as an attenuation weight factor in the transition equation; finally, the causal state transition rule library containing the optical distortion constraint is reconstructed.

[0150] In S503, the display state attenuation model refers to a mathematical description framework of the visibility gradual change process of the virtual object moving with the spatial position under the constraint of the distortion causal rule.

[0151] In S503, the display state attenuation model refers to a mathematical description framework of the visibility gradual change process of the virtual object moving with the spatial position under the constraint of the distortion causal rule.

[0152] In the embodiments of the present application, first, the state transition rule library in the distortion causal link information is loaded; second, the object display integrity parameter is initialized at the boundary of the residual image area; then, the visibility attenuation curve of the object moving to the center of the residual image is deduced according to the transition rule; finally, the dynamic mathematical model containing the attenuation rate and the critical disappearance threshold is established.

[0153] In S504, the anti-intervention parameter refers to a set of feasibility constraint variables that fuse the visibility attenuation characteristics of the object and the spatial trajectory of the gesture operation.

[0154] In S504, the anti-intervention parameter refers to a set of feasibility constraint variables that fuse the visibility attenuation characteristics of the object and the spatial trajectory of the gesture operation.

[0155] In the embodiments of the present application, first, the three-dimensional coordinates of the critical disappearance point of the object in the display state decay model are analyzed; second, the shortest spatial distance between the gesture operation path trajectory and the critical point is calculated; then, the maximum allowed time consumption of the gesture operation is dynamically adjusted according to the decay rate; finally, an intervention parameter set containing the spatial distance tolerance and the time window threshold is generated.

[0156] The following is a specific example:

[0157] In the AR museum tour scene, the system detects that there is a high-density polarization phase offset in the artifact display label area, and calculates to generate a strong interference intensity parameter; the parameter is injected into the causal link of "visual attention artifact label triggered operation delay", and the distortion causal rule of "operation triggered when residual image exists needs to be delayed" is reconstructed; based on the rule, the visibility decay process of the label in the residual image area is deduced, and a dynamic model of accelerated decline of label visibility when moving close to the center of the residual image is established; according to the model, it is determined that the gesture operation needs to be completed within a certain distance from the critical disappearance point and the operation speed needs to be increased to a multiple of the baseline value, and an anti-intervention parameter set containing the spatial distance limit value and the operation speed threshold value is generated.

[0158] In summary, S501 to S504 realize accurate modeling of physical defects through a four-level cross-layer intervention mechanism. First, the optical distortion is quantified as a calculable parameter; second, the state transmission rule of the causal network is reconstructed; third, a dynamic decay model of object visibility is established; finally, operation feasibility constraint parameters are generated; a closed-loop intervention chain from hardware signals to software logic is formed, providing high-fidelity physical constraints for backtracking and breaking through the limitations of traditional methods for simplified modeling of hardware defects.

[0159] In order to finely reconstruct the differentiation features of the user decision path, in some embodiments, according to S304, the potential intention marking information and the behavior contradiction focus information are used to perform topological reconstruction processing on the user's interaction intention differentiation path, and behavior interaction feature information is generated, including:

[0160] S601, performing intention splitting processing on the untriggered hot spot area in the potential intention marking information and the operation conflict area in the behavior contradiction focus information, to generate split intention node information;

[0161] In S601, the split intention node information refers to an independent decision unit set formed by dividing the intersection part of the untriggered hot spot area and the operation conflict area through spatial overlap analysis, and each unit contains spatial boundary coordinates and an initial intention intensity value.

[0162] In the embodiment of the application, firstly, a three-dimensional mapping relationship between an untriggered hotspot area space coordinate set of potential intention marking information and an operation conflict area space coordinate set of behavior contradiction focus information is established; secondly, a spatial intersection volume and a continuous distribution characteristic of the two coordinate sets are calculated; then, the overlapping area is divided into minimum decision unit blocks according to the spatial continuity principle; finally, a basic intention intensity value is assigned to each unit block to generate a split node set with a weight attribute.

[0163] In S602, the path repulsion parameter refers to a combined measurement index of the path curvature mutation amplitude and the avoidance distance threshold value generated when the gaze trajectory circumvents the split node boundary.

[0164] In S602, the path repulsion parameter refers to a combined measurement index of the path curvature mutation amplitude and the avoidance distance threshold value generated when the gaze trajectory circumvents the split node boundary.

[0165] In the embodiment of the application, firstly, a three-dimensional mapping relationship between an untriggered hotspot area space coordinate set of potential intention marking information and an operation conflict area space coordinate set of behavior contradiction focus information is established; secondly, a spatial intersection volume and a continuous distribution characteristic of the two coordinate sets are calculated; then, the overlapping area is divided into minimum decision unit blocks according to the spatial continuity principle; finally, a basic intention intensity value is assigned to each unit block to generate a split node set with a weight attribute.

[0166] In S603, the intention connection attenuation information refers to a topology relationship correction data set obtained by implementing nonlinear attenuation on the initial connection weight of adjacent split nodes according to the spatial repulsion strength.

[0167] In S603, the intention connection attenuation information refers to a topology relationship correction data set obtained by implementing nonlinear attenuation on the initial connection weight of adjacent split nodes according to the spatial repulsion strength.

[0168] In the embodiment of the application, firstly, a three-dimensional mapping relationship between an untriggered hotspot area space coordinate set of potential intention marking information and an operation conflict area space coordinate set of behavior contradiction focus information is established; secondly, a spatial intersection volume and a continuous distribution characteristic of the two coordinate sets are calculated; then, the overlapping area is divided into minimum decision unit blocks according to the spatial continuity principle; finally, a basic intention intensity value is assigned to each unit block to generate a split node set with a weight attribute.

[0169] In S604, the reconstructed topology path information refers to a spatial trajectory network model containing a main path and a potential branch path generated by a path optimization algorithm based on the attenuation weight distribution graph.

[0170] In S604, the reconstructed topology path information refers to a spatial trajectory network model containing a main path and a potential branch path generated by a path optimization algorithm based on the attenuation weight distribution graph.

[0171] In the embodiments of the present application, first, the dynamic weight distribution atlas in the intention connection attenuation information is loaded; second, the core path skeleton network formed by the high weight connection edges is identified; then, the spatial distribution of the branch point of the low weight connection region is detected; finally, the multi-level branch network is generated by simulating the evolution trajectory of the path extending from the core skeleton to the branch point through the minimum energy path algorithm.

[0172] In S605, the differentiation characteristics of the user's behavior mode are structurally described based on the reconstructed topological path information to generate behavior interaction feature information.

[0173] In S605, the behavior interaction feature information refers to converting the topological path network into a computable decision framework containing the priority of the main operation flow and the trigger condition of the alternative operation flow.

[0174] In the embodiments of the present application, first, the core backbone path coordinate sequence in the reconstructed topological path information is parsed; second, the starting node and the terminal node spatial coordinates of each level branch path are extracted; then, the spatial deviation angle and the path length ratio of the branch path and the backbone path are quantified; finally, a structured feature description model containing the path selection probability distribution and the execution condition constraint is constructed.

[0175] The following is a specific example:

[0176] In the VR space training scene, the user forms a high attention untriggered hot spot for the fuel valve icon, and there is a valve operation conflict in this region; the system divides the overlapping region into five split nodes and assigns an initial intensity; it is detected that the gaze trajectory bypasses the node three boundary with high curvature, and the repulsion parameter is calculated; the connection edge between node two and node three is exponentially attenuated according to the repulsion value; the differentiated topological network of the main path executing routine detection and the branch path emergency valve closing is reconstructed; and finally, the structured behavior feature model of the main path selection probability and the branch path temperature threshold is generated.

[0177] In summary, S601 to S605 realize the fine modeling of the decision path through a five-level topological evolution mechanism. First, the intention conflict is converted into a computable node unit through spatial segmentation; second, the physical repulsion effect produced by the gaze bypass is quantified; third, the node connection weight is dynamically reconstructed; fourth, the multi-level path network evolution is simulated; and finally, the structured behavior decision model is generated; forming a complete conversion chain from the original behavior contradiction to the computable differentiated feature, breaking through the limitations of traditional linear behavior modeling, and providing a high-fidelity decision path input for backtracking.

[0178] In order to accurately simulate the interaction failure process caused by physical defects, in some embodiments, according to S203, the anti-intervention parameter is used to perform backtracking processing on the operation result of the unexecuted gesture operation under the rendering abnormal area to generate anti-operation result information, including:

[0179] S701, based on the anti-intervention parameter, the interaction response rule of the virtual object in the rendering abnormal area is constrained and reconstructed to generate abnormal interaction constraint information;

[0180] In S701, the abnormal interaction constraint information refers to converting the spatial distance tolerance of the anti-intervention parameter and the time window threshold into the set of spatial sensitivity decay function and state change acceleration rule of the virtual object responding to the gesture operation.

[0181] In the embodiment of the application, first, the minimum safe spatial distance limit parameter in the anti-intervention parameter is analyzed; second, the maximum allowed time threshold parameter is extracted; then, the spatial distance limit is mapped to a non-linear function of the sensitivity of the object responding to the gesture approaching action decreasing with distance; at the same time, the time window threshold is converted into an acceleration factor function of the object state change rate increasing with operation time; finally, the two types of functions are fused to generate an interaction constraint rule library containing spatial sensitivity curve and time acceleration factor.

[0182] S702, using the abnormal interaction constraint information, the deformation process of the trajectory form of the gesture operation not performed in the rendering abnormal area is dynamically simulated to generate gesture deformation trajectory information;

[0183] In S702, the gesture deformation trajectory information refers to a non-continuous spatial distortion path of the original gesture path under the constraint of spatial sensitivity decay; the path contains a fracture zone and a curvature mutation feature.

[0184] In the embodiment of the application, first, the spatial sensitivity decay function in the abnormal interaction constraint information is loaded; second, the initial spatial coordinates and direction vector of the gesture operation are set at the boundary of the rendering abnormal area; then, the deformation gradient distribution of the normal direction of each point of the path is calculated according to the spatial sensitivity function; finally, the fracture zone and the curvature mutation are generated by applying displacement disturbance along the gradient direction to generate a distorted spatial trajectory.

[0185] S703, combining the gesture deformation trajectory information, the state migration path of the virtual object under abnormal constraint is evolved to generate object state evolution information;

[0186] In S703, the object state evolution information refers to the complete migration path record of the change of the key attribute value of the virtual object with the operation process under the action of the gesture deformation trajectory combined with the time acceleration factor.

[0187] In the embodiment of the application, first, the accurate correspondence between the spatial coordinate points and the time stamps of the gesture deformation trajectory is established; second, the time acceleration factor function in the abnormal interaction constraint information is loaded; then, the object state change rate is improved according to the acceleration factor at the curvature mutation point of the deformation trajectory; finally, the whole path attribute value change sequence of the object from the initial state to the critical failure state is continuously recorded.

[0188] S704, boundary positioning processing is performed on critical conditions of operation interruption and error triggering in the object state evolution information, and operation failure boundary information is generated;

[0189] In S704, the operation failure boundary information refers to a set of accurate spatial positions and time nodes that trigger functional abnormalities or operation interruptions in the object state transition path.

[0190] In the embodiments of the present application, first, the first derivative mutation points of the attribute values in the object state evolution information are scanned; second, the critical position coordinates of the attribute values exceeding the preset safety threshold are detected; then, the turning points of the irreversible state changes are located; finally, the three-dimensional spatial coordinates and accurate time stamp data pairs of the failure triggering points are marked.

[0191] S705, using the operation failure boundary information, integrity deduction processing is performed on the final interaction result of the unexecuted gesture operation in the rendering abnormal area, and anti-operation result information is generated.

[0192] In S705, the anti-operation result information refers to a visual deduction report framework that integrates failure boundary features and abnormal terminal state reconstruction.

[0193] In the embodiments of the present application, first, the spatial position coordinates and time node data in the operation failure boundary information are analyzed; second, the motion direction and speed parameters of the gesture deformation trajectory at the corresponding time are traced back; then, the gradual change process curve of the object state within the time window before and after the failure point is reconstructed; finally, a three-dimensional visual deduction report containing the error triggering position, failure mechanism diagram and state change curve is generated.

[0194] The following is a specific example:

[0195] In virtual airplane maintenance training, the system generates a proximity sensitivity decay rule for the wrench operation based on the anti-interference parameters; the wrench trajectory deformation process is deduced in the engine component residual area; the bolt stress value accelerates under the action of the simulated deformation trajectory; the critical point of thread gliding triggered by stress exceeding the yield strength is detected; finally, the complete operation result of "wrench trajectory deviation leading to bolt failure" is reconstructed, and a visual three-dimensional report containing the stress curve mutation and gliding position is generated.

[0196] In summary, S701 to S705 realize high-fidelity simulation of physical defect scenarios through a five-level precise deduction mechanism. First, the intervention parameters are converted into spatial sensitivity and time acceleration double-constraint rules; second, the dynamic deformation process of the gesture path is deduced; then, the object state transition is simulated in combination with the time acceleration factor; then, the failure critical point is accurately located; finally, the visual operation result is reconstructed; a closed-loop deduction chain from hardware defects to operation failure is formed, breaking through the limitations of traditional methods for simplified modeling of physical abnormal scenarios.

[0197] Figure 4 A structural schematic diagram of one specific embodiment of an AI-based automatic test case dynamic generation system provided by an embodiment of the present application is shown in FIG. 1. Figure 4 The system can include:

[0198] A behavior collection module 41 is configured to collect and process user behaviors to obtain gaze tracks and gesture sequences in an augmented reality and virtual reality application, integrate the gaze tracks and the gesture sequences, and generate user interaction behavior information.

[0199] An association construction module 42 is configured to perform AI-driven behavior association construction processing on the user interaction behavior information to generate behavior interaction feature information.

[0200] A polarization capture module 43 is configured to perform phase shift capture processing on a display screen polarization state of AI to generate sub-pixel response delay information.

[0201] An abnormality identification module 44 is configured to identify a rendering abnormal area of the sub-pixel response delay information to generate rendering abnormal area information.

[0202] An intention deduction module 45 is configured to perform AI reverse deduction processing in combination with the behavior interaction feature information and the rendering abnormal area information to generate potential interaction intention information.

[0203] A test case generation module 46 is configured to use the potential interaction intention information as a dynamic generation target to construct an automatic test case dynamic to generate a dynamic test case set.

[0204] The AI-based automatic test case dynamic generation system of the embodiment of the present application is used to implement the AI-based automatic test case dynamic generation method described above, and thus the specific embodiments of the AI-based automatic test case dynamic generation system can refer to the embodiment part of the AI-based automatic test case dynamic generation method described above, and the specific embodiments can refer to the description of the respective embodiment parts, which will not be described herein again.

[0205] The present application also provides an electronic device, which includes a memory for storing a computer program and a processor for executing the computer program to implement the steps of the AI-based automatic test case dynamic generation method described above.

[0206] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the AI-based automatic test case dynamic generation method described above.

[0207] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a read-only memory, a random access memory, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0208] Embodiments of the present application also provide a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the steps in any of the above-described AI-based automatic test case dynamic generation method embodiments.

[0209] The skilled person can further realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0210] The above provides a detailed description of an AI-based automatic test case dynamic generation method, system, electronic device and storage medium provided by the present application. The principles and implementation modes of the present application are described herein by applying specific examples. The above example descriptions are only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary skilled persons in the technical field, without departing from the principles of the present application, the present application can be improved and modified in several ways. These improvements and modifications also fall within the scope of protection of the present application.

Claims

1. An AI-based automated test case dynamic generation method, characterized in that, The method comprises the following steps: Based on the behavior collection processing of the user, the gaze track and the gesture sequence in the augmented reality and virtual reality application are obtained, the gaze track and the gesture sequence are integrated, and the user interaction behavior information is generated; The AI-driven behavior association construction processing is performed on the user interaction behavior information to generate behavior interaction feature information, wherein the behavior interaction feature information refers to the implicit association model between visual attention intensity and operation response delay mined from the user interaction behavior information by artificial intelligence technology; The phase shift capture processing is performed on the display screen polarization state of AI to generate sub-pixel response delay information; The rendering abnormal area of the sub-pixel response delay information is identified to generate rendering abnormal area information; The AI backtracking processing is performed in combination with the behavior interaction feature information and the rendering abnormal area information to generate potential interaction intention information; The dynamic test case set is generated by taking the potential interaction intention information as a dynamic generation target to construct an automatic test case dynamically.

2. The method of claim 1, wherein, The AI backtracking processing is performed in combination with the behavior interaction feature information and the rendering abnormal area information to generate potential interaction intention information, including: The implicit causal relationship between the user operation delay of the behavior interaction feature information and the visual attention degree of the behavior interaction feature information is constructed to generate behavior causal structure information; The virtual object display state in the behavior causal structure information is intervened by the rendering abnormal area information to generate anti-intervention parameters; The anti-operation result information is generated by using the anti-intervention parameters to deduce the operation result of the gesture operation not executed in the rendering abnormal area. Based on the anti-operation result information, the potential interaction intention of the user is described and reconstructed to generate potential interaction intention information.

3. The method of claim 1, wherein, The AI-driven behavior association construction processing is performed on the user interaction behavior information to generate behavior interaction feature information, including: The asynchronous coupling processing is performed on the pause interval in the gaze track of the user interaction behavior information and the operation conflict in the gesture sequence of the user interaction behavior information to generate behavior contradiction focus information; Based on the behavior contradiction focus information, the visual attention accumulation area in the gaze track and the operation delay burst period in the gesture sequence are quantified to generate interaction hesitation feature information; In combination with the interaction hesitation feature information, the hotspot area in the gaze track which is not triggered by the gesture is marked for intention residue to generate potential intention marking information; The topological reconstruction processing is performed on the interaction intention differentiation path of the user by using the potential intention marking information and the behavior contradiction focus information to generate behavior interaction feature information.

4. The method of claim 1, wherein, The dynamic test case set is generated by taking the potential interaction intention information as a dynamic generation target to construct an automatic test case dynamically, including: The spatial coordinate conversion processing is performed on the virtual operation area pointed by the intention in the potential interaction intention information to generate intention space positioning information; In combination with the rendering abnormal area information, the spatial variation processing is performed on the operation path in the intention space positioning information to generate variation operation path information; The gesture action sequence in the potential interaction intention information is subjected to conflict injection processing by using the variation operation path information, and a conflict-enhanced gesture sequence is generated. Based on the conflict-enhanced gesture sequence and the variation operation path information, a test case execution logic is dynamically synthesized to generate a dynamic test case set.

5. The method of claim 2, wherein, Based on the rendering abnormal area information, intervention condition injection processing is performed on the virtual object display state in the behavior cause-effect structure information to generate anti-intervention parameters, including: Physical disturbance quantization processing is performed on the optical distortion degree of the residual image area in the rendering abnormal area information to generate a residual image disturbance intensity parameter; By using the residual image disturbance intensity parameter, state reconstruction processing is performed on the operation delay in the behavior cause-effect structure information and the causal link between the visual attention degree in the behavior cause-effect structure information to generate distortion causal link information; Based on the display state decay model, intervention correction processing is performed on the operation feasibility of the unexecuted gesture operation under the distortion causal constraint to generate anti-intervention parameters. By using the potential intention marker information and the behavior contradiction focus information, topological reconstruction processing is performed on the interaction intention differentiation path of the user to generate behavior interaction feature information, including:

6. The method of claim 3, wherein, Intention splitting processing is performed on the untriggered hotspot area in the potential intention marker information and the operation conflict area in the behavior contradiction focus information to generate split intention node information; Based on the split intention node information, operation detour path exclusion strength calculation processing is performed on the surrounding conflict area in the gaze trajectory to generate path exclusion parameters; By using the path exclusion parameters, dynamic attenuation processing is performed on the connection relationship between the split intention node information to generate intention connection attenuation information; Based on the intention connection attenuation information, evolution simulation processing is performed on the spatial topological structure of the interaction intention differentiation path to generate reconstructed topological path information; Based on the reconstructed topological path information, structured description processing is performed on the differentiation features of the user's behavior mode to generate behavior interaction feature information. By using the anti-intervention parameters, the operation result of the unexecuted gesture operation under the rendering abnormal area is deduced to generate anti-operation result information, including:

7. The method of claim 2, wherein, Based on the anti-intervention parameters, constraint reconstruction processing is performed on the interaction response rules of the virtual object in the rendering abnormal area to generate abnormal interaction constraint information; By using the abnormal interaction constraint information, dynamic simulation processing is performed on the deformation process of the trajectory form of the unexecuted gesture operation in the rendering abnormal area to generate gesture deformation trajectory information; Based on the gesture deformation trajectory information, evolution deduction processing is performed on the state transition path of the virtual object under abnormal constraint to generate object state evolution information; Boundary positioning processing is performed on the critical conditions of operation interruption and error triggering in the object state evolution information to generate operation failure boundary information; ​ The operation failure boundary information is used to perform integrity deduction processing on a final interaction result of the non-performed gesture operation in the rendering abnormal area, and generate reverse operation result information.

8. An AI-based automated test case dynamic generation system, characterized by, The method comprises the steps of: An action collection module is configured to collect user behavior to obtain a gaze track and a gesture sequence in an augmented reality and virtual reality application, integrate the gaze track and the gesture sequence, and generate user interaction behavior information. An association construction module is configured to perform AI-driven behavior association construction processing on the user interaction behavior information to generate behavior interaction feature information, wherein the behavior interaction feature information refers to an implicit association model between visual attention intensity and operation response delay that is mined from the user interaction behavior information by using artificial intelligence technology. A polarization capture module is configured to perform phase shift capture processing on a display screen polarization state of AI to generate sub-pixel response delay information. An abnormality identification module is configured to identify a rendering abnormal area of the sub-pixel response delay information to generate rendering abnormal area information. An intention deduction module is configured to perform AI reverse deduction processing in combination with the behavior interaction feature information and the rendering abnormal area information to generate potential interaction intention information. A use case generation module is configured to use the potential interaction intention information as a dynamic generation target to dynamically construct an automated test use case and generate a dynamic test use case set.

9. An electronic device, comprising: The method comprises the steps of: A memory is configured to store a computer program. A processor is configured to execute the computer program to implement the steps of the AI-based automated test use case dynamic generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium and can be executed by the processor to implement the AI-based automated test use case dynamic generation method according to any one of claims 1 to 7.

Images