Extended reality system failure detection and handling method and system

Abstract

Embodiments of the present disclosure disclose an extended reality system fault detection and processing method and system. A specific embodiment of the method includes: monitoring and processing running test case information to obtain monitoring result information; in response to determining that the monitoring result information meets a preset abnormal condition, determining whether the detected running data detection information meets a preset fault condition; and in response to determining that the running data detection information meets the preset fault condition, performing a fault processing operation according to fault detection report information corresponding to the running data detection information. This embodiment can narrow down the scope of investigation, shorten the time spent on investigation, and reduce the computing resources consumed.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of extended reality systems, and specifically to methods and systems for fault detection and handling in extended reality systems. Background Technology

[0002] Testing extended reality systems allows for faster detection and repair of faults, and assigning fault-handling tasks to highly suitable personnel can improve the effectiveness of fault repair. Currently, the main method for testing extended reality systems is to use traditional single-point automated testing methods to test each end of the XR system.

[0003] However, in practice, it has been found that when using traditional single-point automated testing methods, the following technical problems often arise: Performing single-point testing on each end requires checking the logs of multiple heterogeneous systems, resulting in a large scope of investigation and long testing time. Single-point testing also causes the detection data of each end to be fragmented, making it difficult to correlate the detection data of each end, resulting in long investigation and error location time and high consumption of computing resources.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not form prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose a method and system for fault detection and handling in extended reality systems to solve one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide an extended reality system fault detection and handling method, which monitors and processes running test case information to obtain monitoring result information; in response to determining that the above monitoring result information meets preset abnormal conditions, it determines whether the detected running data detection information meets preset fault conditions; in response to determining that the above running data detection information meets the above preset fault conditions, it performs fault handling operations according to the fault detection report information corresponding to the above running data detection information.

[0008] Optionally, the method further includes: in response to determining that the above monitoring result information meets preset abnormal conditions, sending an operation data acquisition instruction to the head-mounted display device and the terminal device; receiving various sub-operation data detection information sent by the head-mounted display device and the terminal device; and determining the execution log information and the received various sub-operation data detection information as the detected operation data detection information.

[0009] Optionally, the method further includes: aligning the above-mentioned running data detection information according to the above-mentioned running test case information to obtain aligned data information; generating a multi-terminal analysis data packet according to the above-mentioned aligned data packet; and determining whether the detected running data detection information meets the preset fault conditions according to the above-mentioned multi-terminal analysis data packet.

[0010] Optionally, the method further includes: sending the aforementioned multi-terminal analysis data packet and fault identification instruction to the cloud; and determining whether the detected operational data detection information meets the preset fault conditions based on the received fault identification information sent by the cloud.

[0011] Optionally, the method further includes: in response to determining that the detected operational data detection information meets the above-mentioned preset fault conditions, sending a fault detection report information generation instruction to the cloud; and receiving the fault detection report information corresponding to the above-mentioned operational data detection information sent by the cloud.

[0012] Optionally, the method further includes: determining fault execution task information based on the fault detection report information; and performing fault handling operations based on the fault execution task information.

[0013] Secondly, some embodiments of this disclosure provide an extended reality system fault detection and processing system, wherein the extended reality system fault detection and processing system includes a main controller, a head-mounted display device, a terminal device, and a cloud. The main controller is used to execute the extended reality system fault detection and processing method as described in the first aspect above. The head-mounted display device and the terminal device are used to collect various operation logs in response to the detection of an operation data collection instruction; and send the collected operation logs as sub-operation data detection information to the main controller. The cloud is used to generate fault identification information based on the received multi-terminal analysis data packet and fault identification instruction; send the fault identification information to the main controller; generate fault detection report information in response to receiving a fault detection report information generation instruction sent by the main controller; and send the fault detection report information to the main controller.

[0014] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0015] The above embodiments of this disclosure have the following beneficial effects: Through the extended reality system fault detection and handling method of some embodiments of this disclosure, the scope of fault investigation can be narrowed, the time spent on inspection and location can be shortened, and the time spent troubleshooting can be reduced. The reason for the large scope of fault investigation, long inspection and location time, and long troubleshooting time is that: single-point testing of each end requires detection of logs from multiple heterogeneous systems, resulting in a large investigation scope and long testing time; single-point testing causes fragmentation of detection data from each end, making it difficult to correlate the detection data from each end, leading to long troubleshooting and location time and high computational resource consumption. Based on this, some embodiments of this disclosure provide an extended reality system fault detection and handling method, including: first, monitoring and processing the running test case information to obtain monitoring result information. Thus, the monitoring results of the running test case information can be obtained to determine whether a fault has occurred. Then, in response to determining that the above monitoring result information meets preset abnormal conditions, determining whether the detected running data detection information meets preset fault conditions. Thus, it can be further determined whether the received running data detection information from each end has a fault. In response to the determination that the aforementioned operational data detection information meets the aforementioned preset fault conditions, fault handling operations are performed based on the fault detection report information corresponding to the aforementioned operational data detection information. Thus, fault handling can be performed on the operational data detection information that has malfunctioned. Because the detected operational data detection information includes logs from the head-mounted display device, terminal device, and executed test cases, it integrates the operational data from each end into operational data detection information. Fault handling based on this integrated operational data detection information allows for the integration of operational information from each end for fault diagnosis, rather than performing single-point testing on each end. This narrows the scope of the investigation, further shortens the detection time, and reduces computational resource consumption. Furthermore, fault handling operations can be performed based on the generated fault detection report information, allowing for the identification of the responsible party and further shortening the fault handling time. Attached Figure Description

[0016] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0017] Figure 1 This is an exemplary system architecture diagram of an extended reality system fault detection and handling method and system according to some embodiments of the present disclosure; Figure 2 These are flowcharts of some embodiments of the extended reality system fault detection and handling method and system according to this disclosure; Figure 3 This is a flowchart of some other embodiments of the extended reality system fault detection and handling method and system according to the present disclosure; Figure 4 This is a schematic diagram of the system structure of the extended reality system fault detection and handling method and system disclosed herein; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0021] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0023] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1An exemplary system architecture 100 is shown that can be applied to an embodiment of the extended reality system fault detection and handling method disclosed herein.

[0025] like Figure 1 As shown, the exemplary system architecture 100 may include a head-mounted display device 11 and a terminal device 12.

[0026] The head-mounted display device 11 may include one or two optomechanical and optical elements 111. These optomechanical and optical elements are used to display a virtual interface. Furthermore, the head-mounted display device 11 also includes a frame 112. In some embodiments, the sensors, processing unit, memory, and battery of the head-mounted display device 11 may be housed within the frame 112. In some alternative implementations, one or more components of the sensors, processing unit, memory, and battery may be integrated into another separate accessory (not shown), connected to the frame 112 via a data cable 13. In some alternative implementations, the head-mounted display device 11 may only have display functionality and some sensors, while providing data processing, data storage, power supply, and other capabilities through a terminal device 12.

[0027] The terminal device 12 includes a touch component 121, which can be a touch screen or a touchpad. The touch screen can be used to display content and control window dragging, and the touchpad can be used to control window dragging. In some embodiments, the head-mounted display device 11 and the terminal device 12 can be communicatively connected. The communication connection can be a wired connection or a Bluetooth connection.

[0028] It should be understood that Figure 1 The number of head-mounted display devices and terminal devices shown is merely illustrative. Any suitable number of head-mounted display devices and terminal devices can be included depending on the implementation requirements.

[0029] Further reference Figure 2 The diagram illustrates a flow 200 of some embodiments of the extended reality system fault detection and handling method according to the present disclosure. The extended reality system fault detection and handling method includes the following steps: Step 201: Monitor and process the running test case information to obtain monitoring results.

[0030] In some embodiments, the aforementioned execution entity can monitor and process the running test case information to obtain monitoring result information. The running test case information can be test cases used to test the extended reality system. Here, the specific type of the test cases is not limited; for example, the test cases can be test cases for performing gesture interactions. The monitoring result information can be the result of executing the running test case information and the monitored performance index data of the connected head-mounted display device and terminal device. Here, the specific type of the head-mounted display device is not limited and can be set according to actual needs. For example, the head-mounted display device can be AR glasses, and the terminal device can be a host connected to the head-mounted display device. As described above... Figure 1 As shown, Figure 1 The head-mounted display device 11 shown above can be a head-mounted display device. Figure 1 The terminal device 12 shown can be any of the aforementioned terminal devices. The result of executing the aforementioned test case information can be either successful or unsuccessful. The aforementioned performance metrics data can be the CPU value, GPU value, remaining memory, and frame rate (FPS) of the head-mounted display device and the terminal device. In practice, the executing entity can connect to the head-mounted display device and the terminal device via their IP addresses. It should be noted that the head-mounted display device and the terminal device must have USB debugging mode enabled, and the head-mounted display device and the terminal device must be on the same local area network as the executing entity. Subsequently, the executing entity can use the ADB toolkit to obtain the CPU value, GPU value, remaining memory, and frame rate (FPS) of the head-mounted display device and the terminal device when executing the aforementioned test case information. Next, the obtained CPU value, GPU value, remaining memory, and frame rate (FPS) of the connected head-mounted display device and the terminal device are determined as performance metrics data. Finally, the performance metrics data and the result of executing the aforementioned test case information are determined as monitoring result information.

[0031] Optionally, firstly, the executing entity can also call a performance metric data acquisition interface to obtain the CPU value, GPU value, remaining memory, and frame rate (FPS) of the aforementioned head-mounted display device and terminal device. This performance metric data acquisition interface can be an API interface used to obtain the CPU value, GPU value, remaining memory, and frame rate (FPS) of the aforementioned head-mounted display device and terminal device. Then, the executing entity can call an execution result acquisition interface, which can be an API interface used to obtain the results of executing the aforementioned test case information.

[0032] Step 202: In response to determining that the monitoring result information meets the preset abnormal conditions, determine whether the detected operating data detection information meets the preset fault conditions.

[0033] In some embodiments, in response to determining that the monitoring result information meets preset abnormal conditions, the execution entity may determine whether the detected operational data detection information meets preset fault conditions. The preset abnormal conditions may be that the monitoring result information indicates that the execution of the operational test case information has failed or that the CPU value, GPU value, remaining memory, or frame rate (FPS) included in the performance indicator data is not within the normal range. There are no specific limitations on the normal ranges for CPU value, GPU value, remaining memory, and frame rate (FPS), which can be set according to actual needs. For example, the normal range for CPU value may be 18% to 42%, the normal range for GPU value may be 15% to 48%, the normal range for remaining memory may be ≥1.7GB, and the normal range for frame rate (FPS) may be 50 to 60 FPS. The operational data detection information may include logs from the head-mounted display device executing the operational test case information, logs from the terminal device executing the operational test case information, and execution log information. The execution log information mentioned above can be a time-series log obtained by the execution entity executing the script corresponding to the above-mentioned test case information, or it can be a time-series log obtained by calling the interface to execute the script corresponding to the above-mentioned test case information.

[0034] Optionally, the aforementioned execution entity may also call the log retrieval interface to obtain the time-series logs obtained from the script running the aforementioned test case information. The aforementioned log retrieval interface can be an API interface used to retrieve logs obtained from the script running the aforementioned test case information.

[0035] In an optional implementation of one embodiment, in response to determining that the above monitoring result information meets preset abnormal conditions, the execution entity may determine whether the detected operational data detection information meets preset fault conditions through the following steps: The first step is to align the runtime data detection information based on the aforementioned test case information to obtain aligned data information. This aligned data information can be the aligned runtime test data. In practice, the executing entity first obtains the ID of the aforementioned test case information. Then, the obtained ID is added to each sub-running data detection information and execution log information included in the aforementioned runtime data detection information to obtain updated runtime data detection information. Afterwards, a fixed-time-slice alignment algorithm is used to perform timestamp alignment on the updated runtime data detection information to obtain aligned runtime data detection information as the aligned data information.

[0036] The second step is to generate a multi-terminal analysis data packet based on the alignment data information described above. This multi-terminal analysis data packet can be the packaged alignment data information. In practice, the executing entity can compress the alignment data information into a zip format, and then use this compressed zip file as the multi-terminal analysis data packet.

[0037] The third step is to determine whether the detected operational data detection information meets the preset fault conditions based on the above multi-terminal analysis data packets.

[0038] Optionally, the aforementioned execution entity may also call the multi-terminal analysis data packet acquisition interface to obtain the multi-terminal analysis data packet. The aforementioned multi-terminal analysis data packet acquisition interface can be an API interface used to obtain the generated multi-terminal analysis data packet.

[0039] In an optional implementation of one embodiment, the execution entity can determine whether the detected runtime data detection information meets the preset fault conditions based on the multi-terminal analysis data packet through the following steps: The first step is to send the aforementioned multi-terminal analysis data packet and fault identification instruction to the cloud. The fault identification instruction can be a command sent to the cloud to generate fault identification information corresponding to the aforementioned multi-terminal analysis data packet. In practice, the executing entity can send the aforementioned multi-terminal analysis data packet and fault identification instruction to the cloud.

[0040] Optionally, the aforementioned executing entity may also call the data upload interface to send the multi-terminal analysis data packet and fault identification instructions to the cloud. The data upload interface can be an API interface used to send the multi-terminal analysis data packet and fault identification instructions to the cloud.

[0041] The second step involves determining, based on the received fault identification information sent from the cloud, whether the detected operational data meets the preset fault conditions. The fault identification information can indicate that the multi-terminal analysis data packet contains errors or that the multi-terminal analysis data packet does not contain errors. In practice, if the fault identification information indicates that the multi-terminal analysis data packet does not contain errors, it is determined that the detected operational data does not meet the preset fault conditions. Conversely, if the fault identification information indicates that the multi-terminal analysis data packet contains errors, it is determined that the detected operational data meets the preset fault conditions.

[0042] Optionally, before determining whether the detected operational data meets the preset fault conditions, the aforementioned execution entity may also perform the following steps: The first step involves sending a data acquisition command to the head-mounted display device and the terminal device in response to the determination that the monitoring results meet preset abnormal conditions. This data acquisition command can represent a command to collect logs from the head-mounted display device and the terminal device. For example, the data acquisition command could be a command to collect logs from the past 30 seconds.

[0043] The second step is to receive various sub-operational data detection information sent by the aforementioned head-mounted display device and the aforementioned terminal device. These sub-operational data detection information can be data collected by the aforementioned head-mounted display device and the aforementioned terminal device. The sub-operational data detection information can be logs collected by the aforementioned head-mounted display device or logs collected by the aforementioned terminal device.

[0044] The third step is to identify the received sub-run data detection information and execution log information as run data detection information.

[0045] Step 203: In response to determining that the operation data detection information meets the preset fault conditions, perform fault handling operations based on the fault detection report information of the corresponding operation data detection information.

[0046] In some embodiments, in response to determining that the above-mentioned operational data detection information meets the above-mentioned preset fault conditions, the above-mentioned execution entity may perform fault handling operations based on the fault detection report information corresponding to the above-mentioned operational data detection information.

[0047] The above embodiments of this disclosure have the following beneficial effects: Through the extended reality system fault detection and handling method of some embodiments of this disclosure, the scope of fault investigation can be narrowed, the time spent on inspection and location can be shortened, and the time spent troubleshooting can be reduced. The reason for the large scope of fault investigation, long inspection and location time, and long troubleshooting time is that: single-point testing of each end requires detection of logs from multiple heterogeneous systems, resulting in a large investigation scope and long testing time; single-point testing causes fragmentation of detection data from each end, making it difficult to correlate the detection data from each end, leading to long troubleshooting and location time and high computational resource consumption. Based on this, some embodiments of this disclosure provide an extended reality system fault detection and handling method, including: first, monitoring and processing the running test case information to obtain monitoring result information. Thus, the monitoring results of the running test case information can be obtained to determine whether a fault has occurred. Then, in response to determining that the above monitoring result information meets preset abnormal conditions, determining whether the detected running data detection information meets preset fault conditions. Thus, it can be further determined whether the received running data detection information from each end has a fault. In response to the determination that the aforementioned operational data detection information meets the aforementioned preset fault conditions, fault handling operations are performed based on the fault detection report information corresponding to the aforementioned operational data detection information. Thus, fault handling can be performed on the operational data detection information that has malfunctioned. Because the detected operational data detection information includes logs from the head-mounted display device, terminal device, and executed test cases, it integrates the operational data from each end into operational data detection information. Fault handling based on this integrated operational data detection information allows for the integration of operational information from each end for fault diagnosis, rather than performing single-point testing on each end. This narrows the scope of the investigation, further shortens the detection time, and reduces computational resource consumption. Furthermore, fault handling operations can be performed based on the generated fault detection report information, allowing for the identification of the responsible party and further shortening the fault handling time.

[0048] Further reference Figure 3 The diagram illustrates a flow 300 of another embodiment of the extended reality system fault detection and handling method. Flow 300 of this extended reality system fault detection and handling method includes the following steps: Step 301: Monitor and process the running test case information to obtain monitoring result information.

[0049] Step 302: In response to determining that the monitoring result information meets the preset abnormal conditions, determine whether the detected operating data detection information meets the preset fault conditions.

[0050] In some embodiments, the specific implementation of steps 301-302 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-202 in the corresponding embodiments will not be repeated here.

[0051] Step 303: In response to determining that the operation data detection information meets the preset fault conditions, a fault detection report information generation instruction is sent to the cloud.

[0052] In some embodiments, in response to determining that the aforementioned operational data detection information meets the aforementioned preset fault conditions, the executing entity may send a fault detection report information generation instruction to the aforementioned cloud. The fault detection report information generation instruction may be an instruction sent to the aforementioned cloud to generate fault detection report information corresponding to the aforementioned operational data detection information.

[0053] Step 304: Receive the fault detection report information corresponding to the operation data detection information sent by the cloud.

[0054] In some embodiments, the aforementioned execution entity may receive fault detection report information corresponding to the aforementioned operational data detection information sent from the cloud.

[0055] Step 305: Determine the fault execution task information based on the fault detection report information.

[0056] In some embodiments, the aforementioned executing entity can determine fault execution task information based on the aforementioned fault detection report information. The aforementioned fault detection report information can be fault details of the detected operational data detection information. The fault details can include at least one of the following: the location of each fault code, the fault phenomenon, and the ID of the suggested handler. The aforementioned fault phenomenon can be the fault type of the aforementioned operational data detection information; for example, the fault phenomenon could be that the terminal device GestureService is stuck in a loop while processing low-confidence data. The aforementioned fault execution task information can be a created work order for handling the fault. The aforementioned suggested handler ID can be the handler's number; for example, the suggested handler ID could be 01. In practice, the aforementioned executing entity can call JIRA's REST API to create a work order corresponding to the aforementioned fault detection report information. Then, the created work order is identified as fault execution task information. For example, the title of the work order could be "[Auto] Gesture Recognition Failure - Low Confidence Processing Anomaly, Handler ID: 01".

[0057] Optionally, the aforementioned executing entity may also call the fault execution task information acquisition interface to obtain the fault execution task information corresponding to the fault detection report information. The aforementioned fault execution task information acquisition interface can be an API interface used to obtain the fault execution task information corresponding to the aforementioned fault detection report information.

[0058] In addressing the technical problems mentioned above by adopting technical solutions, and considering the application scenario of unattended automated regression testing, the following technical issues often arise: fixed configuration of the handler, without considering the handler's past experience in handling faults, results in low accuracy in matching the handler, leading to poor processing effectiveness and time consumption. Given the following requirements for this application scenario: automatic determination of the handler is necessary, and the accuracy of the determined handler is crucial, we have decided to adopt the following solution: Optionally, the aforementioned execution entity can also perform the following steps: Optionally, the ID of the person handling the above-mentioned recommendations can also be determined through the following steps: The first step is to denoise the acquired historical fault handling information set, resulting in a denoised historical fault handling information set. The historical fault handling information in this set can be a mapping between the location of the fault code and the ID of the person handling the fault.

[0059] The second step involves generating a knowledge graph based on the aforementioned historical fault handling information set and the preset project source code library. The preset project source code library can be the source code being examined that generated the historical fault handling information set. In practice, firstly, the executing entity can use the BiLSTM-CRF algorithm to extract entities from the historical fault handling information set. Then, the BERT relation classification algorithm can be used to create initial triples between the historical fault handling information set and the preset project source code library. Next, the TransE embedding algorithm is used to fuse the subgraphs of the historical fault path information set and the subgraphs of the preset project source code library to obtain the initial knowledge graph. Finally, the path ranking PRA algorithm is used to update the initial knowledge graph, resulting in the updated initial knowledge graph.

[0060] The third step involves parsing the aforementioned multi-endpoint analysis data packets to determine the location of the code error. This location can be the line and path where the erroneous code is located. In practice, the executing entity can use a regular expression matching algorithm to parse the multi-endpoint analysis data packets and obtain the code error location.

[0061] Fourth, based on the knowledge graph described above, determine the information of the first person to handle the error at the corresponding code error location. This first person to handle the error can be the ID of the person who handled the error code at the aforementioned code error location.

[0062] Fifth, based on the preset set of personnel information, determine the second personnel information corresponding to the aforementioned code error location. The preset personnel information in the preset personnel information set can be pre-defined personnel responsible for handling the error code corresponding to the aforementioned code error location. Furthermore, the preset personnel information in the preset personnel information set can represent a correspondence between code error locations and personnel. In practice, the executing entity can determine the preset personnel information corresponding to the aforementioned code error location in the preset personnel information set as the target preset personnel information. Then, the ID of the personnel included in the target preset personnel information is determined as the second personnel information.

[0063] Step 6: In response to the determination that the first processing personnel information and the second processing personnel information are different, based on the historical details of the first processing personnel information and the second processing personnel information, determine the matching score information corresponding to the first processing personnel information and the second processing personnel information. The historical details information can be the error codes previously processed by the first processing personnel information and the error codes previously processed by the second processing personnel information. The matching score information can be the similarity between the historical details information of the first processing personnel information and the location of the error code. In practice, the executing entity can determine the similarity between the historical details information of the first processing personnel information and the location of the error code as the matching score information corresponding to the first processing personnel information based on a cosine similarity algorithm, and determine the similarity between the location of the error code and the historical details information of the second processing personnel information as the matching score information corresponding to the second processing personnel information.

[0064] Step 7: Based on the matching scores of the first and second processing personnel information, determine the processing personnel information that meets the preset matching conditions from the first and second processing personnel information as the target processing personnel information. The preset matching condition may be that the corresponding matching score in the first and second processing personnel information is the highest.

[0065] Step 8: Add the ID corresponding to the above target handler information as the ID of the suggested handler to the above fault detection report information.

[0066] The above-described technical solution, as an inventive point of this disclosure, solves the technical problem of "low accuracy in matching processors, resulting in poor processing effectiveness and time consumption." The reasons for this low accuracy and poor processing effectiveness and time consumption are as follows: The processor is fixed in its configuration, without considering the past fault handling experience of the assigned processor, leading to low accuracy in matching processors and consequently poor processing effectiveness and time consumption. Therefore, this disclosure matches processors based on their past processing experience and a knowledge graph. Simultaneously, the knowledge graph can be continuously optimized and updated with business operation data, reducing the workload of manually maintaining the configuration table, adapting to long-term business operation scenarios with iterative updates to program code, and improving the accuracy of personnel allocation. This allows processors to quickly handle faults, improving fault handling effectiveness and shortening processing time.

[0067] In addressing the technical problems mentioned above by adopting technical solutions, the following technical issues often arise in the application scenario: unattended automated regression testing. Directly determining the handler based on a set of unprocessed historical fault handling information without considering preprocessing this information leads to low accuracy in matching handlers, resulting in poor processing effectiveness and time consumption. Considering the following requirements for this application scenario: automatic handler determination and high accuracy of the determined handler, we have decided to adopt the following solution: In an optional implementation of one embodiment, the execution entity may perform noise reduction processing on the acquired historical fault handling information set through the following steps to obtain the noise-reduced historical fault handling information set as the historical fault handling information set: The first step involves clustering the acquired historical fault handling information set based on threshold neighborhood parameter information to obtain various clusters. The threshold neighborhood parameter information can be defined as a time threshold, the location of the fault code, and the ID of the handler. In practice, the executing entity can use the weighted DBSCAN algorithm to cluster the acquired historical fault handling information set based on the threshold neighborhood parameter information to obtain various clusters.

[0068] The second step involves updating the existing clusters based on the number of clusters that meet a preset quantity condition. This preset quantity condition can be defined as the number of data points in a cluster exceeding a target value. The specific value of this target value is not limited and can be set according to actual needs. In practice, the executing entity can use the hierarchical agglomerative clustering (AGNES) algorithm to split the clusters that meet the preset quantity condition, obtaining cluster groups. Then, these split cluster groups are added to the existing clusters to obtain the updated clusters.

[0069] The third step involves determining that there exists a cluster among the aforementioned clusters that satisfies a preset single-point condition. This cluster is then removed from the list of clusters, resulting in updated clusters. The preset single-point condition can be that the number of data points included in a cluster is less than two.

[0070] The fourth step involves determining, in response to the discovery that there are clusters among the aforementioned clusters that satisfy a preset similarity condition, identifying each cluster satisfying the preset similarity condition as a target cluster. The preset similarity condition can be that the Euclidean distance between the centroids of the clusters is greater than 0.9. The centroid can be the centroid of the cluster. In practice, the executing entity can determine the Euclidean distance between the centroids of every two clusters, obtaining the Euclidean distance of each cluster centroid. Then, the two clusters satisfying the preset similarity condition are identified as target cluster groups, resulting in target cluster groups.

[0071] The fifth step involves filtering the target cluster groups to obtain updated clusters. In practice, for each target cluster group, the executing entity can remove one target cluster from the existing clusters to obtain updated clusters.

[0072] The sixth step involves updating the acquired historical fault handling information set based on the updated clusters, resulting in the updated historical fault handling information set. In practice, the executing entity can replace the acquired historical fault handling information set with the updated historical fault handling information corresponding to each cluster to perform noise reduction on the acquired historical fault handling information set, obtaining the noise-reduced historical fault handling information set as the final historical fault handling information set.

[0073] The above technical solution, as an inventive point of this disclosure, solves the technical problem of "low accuracy in matching handlers, resulting in poor processing effectiveness and time consumption." The reasons for this low accuracy and poor processing effectiveness and time consumption are as follows: directly determining the handler based on the unprocessed historical fault processing information set without considering preprocessing the historical fault processing information set leads to low accuracy in matching handlers, resulting in poor processing effectiveness and time consumption. Therefore, this disclosure performs clustering updates on the acquired historical fault processing information set based on threshold neighborhood parameter information to achieve noise reduction. Then, based on the noise-reduced historical fault processing information set, a handler with higher accuracy can be determined. This improves the accuracy of personnel allocation, allowing handlers to quickly process faults, improving fault processing effectiveness and shortening processing time.

[0074] Step 306: Perform fault handling operations based on the fault execution task information.

[0075] In some embodiments, the executing entity can perform a fault handling operation based on the fault execution task information. This fault handling operation can be sending the fault execution task information to the handler corresponding to the fault execution task information. In practice, the executing entity can send the fault execution task information to the handler corresponding to the fault execution task information to perform the fault handling operation. It should be noted that the mobile device that sends the fault execution task information to the handler can be a mobile phone, for example.

[0076] Optionally, the aforementioned execution entity may also invoke a fault handling operation interface to perform fault handling operations. This fault handling operation interface can be an API interface used to perform fault handling operations.

[0077] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 The process 300 of the test scenario recovery method in some corresponding embodiments demonstrates that it can locate fault phenomena and improve the accuracy of locating the person handling the fault error. Therefore, it can improve the speed of locating fault errors and the person handling them, thereby shortening the time spent on fault handling and handling the fault in a timely manner.

[0078] Further reference Figure 4 , Figure 4 A schematic diagram 400 illustrates a system architecture that can be applied to some embodiments of the extended reality system fault detection and handling system disclosed herein. For example... Figure 4As shown, the extended reality system fault detection and handling system disclosed herein may include: a head-mounted display device and terminal device 401, a main controller 402, and a cloud 403. The head-mounted display device and terminal device 401 can be used to collect various operation logs in response to the detection of an operation data acquisition command; and to send the collected operation logs as sub-operation data detection information to the main controller. The main controller 402 can be used to perform the above-described... Figure 2 and the above Figure 3 The extended reality system fault detection and handling method described herein. The cloud-based 403 can be used to generate fault identification information based on the received multi-terminal analysis data packet and fault identification instruction; send the fault identification information to the main controller; generate fault detection report information in response to receiving the fault detection report information generation instruction sent by the main controller; and send the fault detection report information to the main controller.

[0079] The following steps illustrate the flow of some embodiments of the extended reality system fault detection and handling system according to this disclosure. The aforementioned extended reality system fault detection and handling system includes the following steps: The first step involves the head-mounted display device and terminal device 401, which collect various operation logs in response to the detection of an operation data collection command.

[0080] In some embodiments, the head-mounted display device and the terminal device 401 are configured to collect various operation logs in response to detecting an operation data collection command. These operation logs can be logs collected by the head-mounted display device according to the operation data collection command and logs collected by the terminal device according to the operation data collection command. In practice, the head-mounted display device and the terminal device use the logs collected corresponding to the operation data collection command as individual operation logs. Here, the specific type of the main controller is not limited and can be set according to actual needs; for example, the main controller can be... Figure 5 The electronic device 500 in the system. The main controller can be connected to the head-mounted display device and the terminal device, as well as to the cloud.

[0081] The second step involves a head-mounted display device and a terminal device 401, which are used to send the collected operation logs as sub-operation data detection information to the main controller.

[0082] In some embodiments, the head-mounted display device and the terminal device are used to send the collected operation logs as sub-operation data detection information to the main controller.

[0083] The third step, cloud-based 403, is used to generate fault identification information based on the received multi-terminal analysis data packets and fault identification instructions.

[0084] In some embodiments, the cloud-based 403 is configured to generate fault identification information based on the received multi-terminal analysis data packet and fault identification instruction. In practice, firstly, the cloud-based 403 can decompress the multi-terminal analysis data packet according to the received fault identification instruction to obtain a decompressed multi-terminal analysis data packet. Then, for each log entry in the decompressed multi-terminal analysis data packet, the cloud-based 403 can determine whether an error has occurred in the log entry based on a preset fault identification information set. The preset fault identification information in the preset fault identification information set can be a correspondence between log entries and fault results. The fault result can be either an error has occurred or no error has occurred. Finally, in response to determining that the decompressed multi-terminal analysis data packet includes log entries with errors, the presence of errors in the multi-terminal analysis data packet is determined as fault identification information. In response to determining that the decompressed multi-terminal analysis data packet does not include log entries with errors, the absence of errors in the multi-terminal analysis data packet is determined as fault identification information.

[0085] The fourth step is a 403 error in the cloud, used to send fault identification information to the main controller.

[0086] In some embodiments, the cloud 403 is used to send fault identification information to the main controller.

[0087] Step 5: Cloud 403 response, in response to receiving the fault detection report information generation instruction sent by the main controller, generates fault detection report information.

[0088] In some embodiments, the cloud-based 403 is configured to generate fault detection report information in response to receiving a fault detection report information generation instruction sent by the main controller. In practice, the cloud-based 403 can determine the location of each fault code in the multi-terminal analysis data packet using a regular expression matching algorithm. Then, it determines the corresponding preset fault handling information for the location of the fault code from a preset fault handling information set, obtaining the determined preset fault handling information. Then, it determines the determined preset fault handling information as fault detection report information. The preset fault handling information in the preset fault handling information set can be a correspondence between the location of the fault code, the ID of the handler, and the fault phenomenon. For example, the preset fault handling information could be the location of the fault code: line 156 of the suspicious source file gesture_processor.cpp, the fault phenomenon: the terminal device GestureService gets stuck in a loop while processing low-confidence data, and the handler's ID: 09. The location of the fault code can also be the location of the code where the error occurred.

[0089] Step 6: Cloud 403, used to send fault detection report information to the main controller.

[0090] In some embodiments, the cloud 403 is used to send the fault detection report information to the main controller.

[0091] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device 500 (e.g., a head-mounted display device, a terminal device, or the main controller 402 included in the above-described extended reality system fault detection and processing system) suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0092] like Figure 5 As shown, the electronic device 500 may include a processing unit 501 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0093] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0094] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of some embodiments of this disclosure.

[0095] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0096] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0097] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: monitor and process running test case information to obtain monitoring result information; in response to determining that the monitoring result information meets preset abnormal conditions, determine whether the detected running data detection information meets preset fault conditions; and in response to determining that the running data detection information meets the preset fault conditions, perform fault handling operations based on fault detection report information corresponding to the running data detection information.

[0098] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0100] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0101] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for fault detection and handling in an extended reality system, comprising: The information of running test cases is monitored and processed to obtain monitoring results. In response to determining that the monitoring result information meets preset abnormal conditions, determine whether the detected operating data detection information meets preset fault conditions; In response to determining that the operation data detection information meets the preset fault conditions, a fault handling operation is performed based on the fault detection report information corresponding to the operation data detection information.

2. The method according to claim 1, wherein, The method further includes: In response to determining that the monitoring result information meets preset abnormal conditions, an operation data acquisition command is sent to the head-mounted display device and the terminal device; Receive various sub-operation data detection information sent by the head-mounted display device and the terminal device; The execution log information and the received sub-run data detection information are identified as the run data detection information.

3. The method according to claim 1, wherein, The process of determining whether the detected operational data meets the preset fault conditions includes: Based on the running test case information, the running data detection information is aligned to obtain aligned data information; Based on the alignment data information, a multi-terminal analysis data packet is generated; Based on the multi-terminal analysis data packet, determine whether the detected operational data detection information meets the preset fault conditions.

4. The method according to claim 3, wherein, The step of determining whether the detected operational data detection information meets the preset fault conditions based on the multi-terminal analysis data packet includes: Send the multi-terminal analysis data packet and fault identification command to the cloud; Based on the fault identification information received from the cloud, determine whether the detected operational data detection information meets the preset fault conditions.

5. The method according to claim 4, wherein, The method further includes: In response to determining that the detected operational data detection information meets the preset fault conditions, a fault detection report information generation instruction is sent to the cloud. Receive fault detection report information corresponding to the operation data detection information sent by the cloud.

6. The method according to claim 1, wherein, The step of performing fault handling operations based on the fault detection report information corresponding to the operational data detection information includes: Based on the fault detection report information, determine the fault execution task information; Based on the fault execution task information, perform fault handling operations.

7. A fault detection and handling system for extended reality systems, wherein, The extended reality system fault detection and handling system includes a main controller, a head-mounted display device, a terminal device, and a cloud platform. The main controller is used to execute the extended reality system fault detection and handling method as described in any one of claims 1-6, wherein... The head-mounted display device and the terminal device are used to collect various operation logs in response to the detection of an operation data collection command; and send the collected operation logs as sub-operation data detection information to the main controller. The cloud platform is used to generate fault identification information based on the received multi-terminal analysis data packets and fault identification instructions; send the fault identification information to the main controller; generate fault detection report information in response to receiving a fault detection report information generation instruction sent by the main controller; and send the fault detection report information to the main controller.

8. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

Images