Fault detection method and related product

Abstract

The invention provides a fault detection method and a related product, and relates to the technical field of fault detection. The fault detection method comprises the steps of obtaining current state information of a to-be-detected system in a current time period, wherein the current state information comprises at least one of a current operation condition, configuration, performance and a health state of the to-be-detected system in the current time period; predicting a fault injection action of the to-be-detected system according to the current state information of the to-be-detected system so as to generate a fault scene through the fault injection action; and determining that the to-be-detected system has a fault in the fault scene. According to the embodiment of the invention, the system fault detection speed and accuracy can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of fault detection, and in particular to a fault detection method and electronic equipment. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] Detecting potential software system failures is crucial for software system development and testing. For example, detecting vulnerabilities in drone trajectory planning in the Unmanned Aircraft System Traffic Management (UTM) system is crucial for airspace safety.

[0004] The technical problem to be solved by this application is how to accelerate the speed of software system fault detection and improve the accuracy of fault detection. Summary of the Invention

[0005] The present disclosure aims to provide a fault detection method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the fault detection speed and fault detection accuracy of a system to be detected.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0007] An embodiment of the present disclosure provides a fault detection method, comprising: obtaining current status information of a system to be detected within a current time period, the current status information including at least one of the current operating status, configuration, performance, and health status of the system to be detected within the current time period; predicting a fault injection action of the system to be detected based on the current status information of the system to be detected, so as to generate a fault scenario through the fault injection action; and determining that the system to be detected has failed under the fault scenario.

[0008] An embodiment of the present disclosure provides a fault detection device, including: a current state information acquisition module, a fault injection action prediction module, and a fault determination module.

[0009] Among them, the current status information acquisition module is used to obtain the current status information of the system to be detected in the current time period, and the current status information includes at least one of the current operating status, configuration, performance and health status of the system to be detected in the current time period; the fault injection action prediction module can be used to predict the fault injection action of the system to be detected based on the current status information of the system to be detected, so as to generate a fault scenario through the fault injection action; the fault determination module is used to determine whether the system to be detected has a fault under the fault scenario.

[0010] An embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor; the memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement any of the above-mentioned fault detection methods.

[0011] An embodiment of the present disclosure provides a computer-readable storage medium having computer program instructions stored thereon for implementing any of the above-mentioned fault detection methods.

[0012] The present disclosure provides a computer program product or computer program, which includes computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium and executed by a processor to implement the above-mentioned fault detection method.

[0013] The fault detection method, apparatus, electronic device, computer-readable storage medium, and computer program product provided by the embodiments of the present disclosure can determine the fault injection action of the system to be detected based on the current state information of the system to be detected in the current time period, so as to determine the fault of the system to be detected under the fault injection action. The embodiments provided by the present application can generate the fault injection action of the system to be detected in a targeted manner based on the current state information of the system to be detected, so as to test the fault of the system to be tested under the fault injection action in a targeted manner, thereby accelerating the fault detection process of the system to be detected and improving the accuracy of system fault detection.

[0014] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0016] Figure 1 A schematic diagram of a scenario that can be applied to the fault detection method or fault detection device of the embodiment of the present disclosure is shown.

[0017] Figure 2 The figure is a flowchart showing a fault detection method according to an exemplary embodiment.

[0018] Figure 3 The present invention is a test system operating environment shown according to an exemplary embodiment.

[0019] Figure 4 The figure is a schematic diagram showing a fault detection architecture according to an exemplary embodiment.

[0020] Figure 5 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0021] Figure 6 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0022] Figure 7 The figure is an architectural diagram of a reinforcement learning strategy model according to an exemplary embodiment.

[0023] Figure 8 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0024] Figure 9 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0025] Figure 10 The figure is a schematic diagram showing a fault detection architecture according to an exemplary embodiment.

[0026] Figure 11 The following is a working process of an action sampler according to an exemplary embodiment.

[0027] Figure 12 The figure is a flow chart of a method for training a target neural network according to an exemplary embodiment.

[0028] Figure 13 The figure is a block diagram of a fault detection device according to an exemplary embodiment.

[0029] Figure 14 A schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0031] The features, structures or characteristics described in the present disclosure may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0032] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all content and steps, nor must they be executed in the order described. For example, some steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0033] In the description of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different; the terms "comprising", "including" and "having" are used to express the meaning of open inclusion and mean that in addition to the listed elements / components / etc., there may be other elements / components / etc.

[0034] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other unless there is a conflict.

[0035] The technical contents involved in this application will be explained and illustrated below.

[0036] Detecting potential failures is crucial for the development and testing of software systems such as mission-critical planning systems, such as detecting drone trajectory planning failures in UAV air traffic management (UTM), which is crucial for airspace safety.

[0037] Inevitably, there exists a small number of rare, unpredictable scenarios in which the system under test (e.g., UTM) may suffer catastrophic failures. Most traditional fault detection methods focus on achieving high coverage through random scenario injection. However, random methods are unacceptably time-consuming and have low accuracy when detecting long-tail vulnerabilities. To address this challenge, this application proposes a scenario-oriented approach to search for long-tail conditions, accelerating the fault detection process and improving fault detection accuracy.

[0038] Unmanned Aerial Vehicle Traffic Management (UTM) is a mission-critical system used to ensure the safety and coordination of low-altitude flight operations. With the increasing use of drones in civil and commercial missions such as logistics, disaster relief, and environmental monitoring, vulnerability detection in UTM systems is crucial to preventing accidents in complex real-world scenarios, necessitating rigorous testing during the validation phase.

[0039] Below, this application will explain and illustrate this application by taking the system to be tested as the traffic management of the drone system as an example, but the system to be tested in this application is not limited to this. Any system that needs to be tested (including software systems or hardware systems, etc.) can be the system to be tested in this application.

[0040] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0041] Figure 1 A schematic diagram of a scenario that can be applied to the fault detection method or fault detection device of the embodiment of the present disclosure is shown.

[0042] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present disclosure.

[0043] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0044] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, wearable devices, virtual reality devices, smart homes, etc.

[0045] The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received requests and other data, and feed back the processing results to the terminal device.

[0046] The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as basic cloud computing services such as big data and artificial intelligence platforms, etc. This disclosure does not impose any restrictions on this.

[0047] The server 105 may, for example, obtain current status information of the system to be detected within the current time period, where the current status information includes at least one of the current operating status, configuration, performance, and health status of the system to be detected within the current time period; the server 105 may, for example, predict the fault injection action of the system to be detected based on the current status information of the system to be detected, so as to generate a fault scenario through the fault injection action; the server 105 may, for example, determine that the system to be detected fails under the fault scenario.

[0048] It should be understood that Figure 1 The number of terminal devices, networks and servers is merely illustrative. The server 105 may be a single entity server or may be composed of multiple servers. It may have any number of terminal devices, networks and servers according to actual needs.

[0049] Figure 2 The method provided by the embodiment of the present disclosure can be executed by any electronic device with computing and processing capabilities. For example, the method can be executed by the above-mentioned Figure 1 The server or terminal device in the embodiment may be executed, or the server and the terminal device may be executed together. In the following embodiments, the server is used as the execution subject for example, but the present disclosure is not limited to this.

[0050] Reference Figure 2 The fault detection method provided by the embodiment of the present disclosure may include the following steps.

[0051] Step S202: obtaining current status information of the system to be detected in the current time period, where the current status information includes at least one of the current operating status, configuration, performance, and health status of the system to be detected in the current time period.

[0052] In some embodiments, the system to be detected can be any system requiring fault detection, and can be a software system, a combination of software and hardware, or a pure hardware system, etc., and this application does not limit this. For example, the system to be detected can be a drone air traffic management system, but this application does not limit this.

[0053] In some embodiments, the status information of the system to be tested may include the operating status of the system to be tested (such as whether it is running, whether it is in normal, abnormal or maintenance status, the system's startup time, operating duration and recent restart records), configuration status, performance indicators, health status, and other relevant information at a specific time point or time period. This information can reflect the overall health status of the system, its operating efficiency, and whether it meets the expected functional and performance requirements.

[0054] For example, a UTM system may include the status of a drone swarm and the status of a ground airport. The status of the drone swarm may include at least one of the flight trajectory, speed, health status, and operating status of each drone, which is not limited in this application. The status of the ground airport may include the location of ground obstacles, the location of drones, etc., which is not limited in this application.

[0055] In some embodiments, the current status information of the system to be tested in the current time period may include status information of the system to be tested during testing.

[0056] In some embodiments, the system to be detected may be a UAV traffic management system. The current status information of the UAV traffic management system may include one or more of the following: power information, environmental data, weather conditions, airspace restrictions, battery information, payload information, and route information of the UAVs managed by the UAV traffic management system.

[0057] Step S204 : predicting a fault injection action of the system to be detected based on the current state information of the system to be detected, so as to generate a fault scenario through the fault injection action.

[0058] Step S206: determining whether the system to be detected fails in the failure scenario.

[0059] In some embodiments, the fault injection action may be an action that can generate a fault scenario. The fault injection action may include at least two types of actions: (1) a one-time physical action; and (2) a short-duration digital action. The one-time physical action may be, for example, wind setting (e.g., setting wind exceeding a certain size or direction), or obstacle setting (e.g., setting an obstacle of a certain size at a certain location in the drone's route). The short-duration digital action may be, for example, setting network jitter (e.g., temporarily disconnecting the network for a certain period of time).

[0060] In some embodiments, a fault injection action for the system to be detected can be predicted based on the current state information of the system to be detected. This fault injection action is then used to manipulate the original scenario of the system to be detected to generate a new scenario, in which the system to be detected may fail. For example, a cluster of aircraft in a UTM system may experience trajectory deviations when wind speeds exceed a certain level, causing the UTM system to fail.

[0061] Figure 3 The present invention is a test system operating environment shown according to an exemplary embodiment.

[0062] Specifically, Figure 3 This study addresses the operational environment or scenarios for Unmanned Aerial Vehicle Traffic Management (UTM) systems during System Under Test (SUT). UTM systems can operate in a variety of environments, including urban, suburban, and rural areas. Each environment presents different challenges, such as high air traffic density in urban areas and limited infrastructure in rural areas, requiring robust management and coordination strategies.

[0063] In some embodiments, during the testing of the system to be tested, a newly generated fault injection action can be injected into the system to be tested to test whether the system to be tested will fail in a new scenario corresponding to the fault injection action, thereby completing the testing of the system to be tested.

[0064] For example, the above method can be used to generate a fault injection action corresponding to the UTM system, and then the scenario generated by the fault injection action can be injected into the UTM system for evaluation to test the vulnerabilities of the UTM system.

[0065] In some embodiments, the target neural network may be used to perform predictive processing on the current state information of the system to be detected to predict the fault injection action of the system to be detected.

[0066] In some embodiments, the target neural network model can be trained using a reinforcement learning framework. The reinforcement learning strategy model can be built using a causal transformer.

[0067] like Figure 4 As shown, observation data of the system to be detected can be obtained from the system to be detected (such as a UTM system) 401. The observation data can include current status information of the system to be detected.

[0068] like Figure 4 As shown, the target neural network 402 can perform predictive processing on the current state information of the system to be detected to predict the fault injection action of the system to be detected.

[0069] like Figure 4 As shown, the action output by the target neural network (such as the fault injection action) can be injected into the system to be detected 401 to test whether the system to be detected will fail under the fault injection action, thereby realizing vulnerability testing of the system to be detected.

[0070] The above embodiment can determine the fault injection action of the system to be detected by the current state information of the system to be detected in the current time period, so as to determine the fault of the system to be detected under the fault injection action. The embodiment provided by the present application can generate the fault injection action of the system to be detected in a targeted manner according to the current state information of the system to be detected, so as to test the fault of the system to be tested under the fault injection action in a targeted manner, thereby improving the accuracy of system fault detection. Through the above method, the search space can be effectively narrowed down to high-risk scenarios, reducing unnecessary exploration of low-risk scenarios, thereby accelerating the fault detection process of the system to be detected and improving the efficiency of fault detection.

[0071] Figure 5 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0072] refer to Figure 5 , the above fault injection action prediction method may include the following steps.

[0073] Step S502 : obtaining current status information of the system to be detected in the current time period, where the current status information includes at least one of the current operating status, configuration, performance, and health status of the system to be detected in the current time period.

[0074] Step S504: obtaining first historical status information of the system to be detected within a first historical time period, where the first historical time period is a time period before the current time period.

[0075] Step S506: concatenate the first historical state information with the current state information to obtain first context-enhanced state information.

[0076] Step S508: predicting a fault injection action of the system to be detected based on the first context enhanced state information.

[0077] In some embodiments, predicting a fault injection action for the system to be inspected based on the first context-enhanced state information may include: inputting the first context-enhanced state information into a target neural network model to obtain probabilities corresponding to multiple preset actions; and selecting one or more actions with the highest probability from the multiple preset actions as the fault injection action. For example, the target neural network may be a self-attention network model (e.g., a Transformer), and the first context-enhanced state information may be input into the self-attention network model to obtain the probabilities of each preset action.

[0078] In some embodiments, the current state information can be enhanced by the previous historical state information of the system to be detected, and the historical state data and real-time system state data are used as input to capture time dependencies and operational dynamics. This involves running models to identify patterns in similar operational dynamics and generalizing them to unseen environments, thereby enabling more accurate prediction of fault injection actions of the system to be detected.

[0079] Figure 6 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0080] refer to Figure 6 , the above fault injection action prediction method may include the following steps.

[0081] Step S602: obtaining current status information of the system to be detected in the current time period, where the current status information includes at least one of the current operating status, configuration, performance, and health status of the system to be detected in the current time period.

[0082] Step S604: obtaining first historical status information of the system to be detected within a first historical time period, where the first historical time period is a time period before the current time period.

[0083] Step S606: concatenate the first historical state information with the current state information to obtain first context-enhanced state information.

[0084] like Figure 7 As shown, the first historical status information or the summary information of the first historical status information within the first historical time period (such as Figure 7 The historical summary in the context) is spliced ​​with the current state information (such as the observation value currently obtained from the system to be detected) to obtain the first context enhanced state information.

[0085] It should be noted that Figure 7 The observation data in the may be observation data obtained from the system to be detected, and the observation data may include state information of the system to be detected.

[0086] Step S608: Obtain expected reward value and available action information.

[0087] In some embodiments, the fault injection action can be predicted by a target neural network, which can be trained by reinforcement learning, and the reward of the reinforcement learning can be determined by the last generated fault injection action.

[0088] In some embodiments, the higher the probability that the fault injection action generates a fault scenario, the greater the corresponding reinforcement learning reward. In other words, the probability that the system to be tested will fail in the fault scenario is proportional to the reinforcement learning reward.

[0089] In some embodiments, the expected return value (Return-to-go) may be the sum of all rewards after the current moment.

[0090] In reinforcement learning, an agent generates a series of states, actions, and rewards during its interaction with the environment, which together form a trajectory. Typically, a trajectory is represented as a sequence of states, actions, and rewards, such as: τ = (s1, a1, r1, s2, a2, r2, …, sT, aT, rT), where si represents the i-th state, ai represents the i-th action, ri represents the i-th reward, and T represents the length of the trajectory, with i being an integer greater than or equal to 1 and less than or equal to T. The expected return (RTG) is a special representation of this trajectory, focusing on the sum of all rewards from a given moment t to the end of the trajectory.

[0091] In some embodiments, the available actions may be pre-set available actions. For example, there may be multiple actions (such as N actions) corresponding to the system to be tested, but in this test, only n actions are set to be available, then the n actions may be the above available actions. In some embodiments, the unavailable actions in the above N actions may be masked to generate corresponding available action information, such as Figure 7 The action mask in . N is an integer greater than or equal to 1, and n is an integer greater than or equal to 1 and less than or equal to N.

[0092] Step S610 : predicting a fault injection action of the system to be detected based on the first context enhanced state information, the expected reward value, and the available action information.

[0093] In some embodiments, the first context-enhanced state information, expected reward value, and available action information may be concatenated (e.g., Figure 7As shown in the figure, the historical summary, observation, expected reward value and action mask are concatenated together) and input into the target neural network (as shown in the figure). Figure 7 The causal transformer in the algorithm is used to predict the probability of each action, and then the actions with higher probabilities are used as fault injection actions for the system to be detected.

[0094] In some embodiments, the target neural network model can be trained through reinforcement learning, the probability of failure of the system to be detected in a fault scenario is positively correlated with the reward of reinforcement learning, and the preset action is the available action among the actions corresponding to the system to be detected.

[0095] In some embodiments, inputting the first context-enhanced state information into the target neural network model to obtain the probabilities corresponding to the plurality of preset actions may include: inputting the first context-enhanced state information (such as Figure 7 Observations + historical summaries in ), rewards (e.g. Figure 7 Return-to-go in ), one or more of the action masks of multiple preset actions (such as Figure 7 The action mask in the image is spliced ​​and input into the target neural network model to obtain the probabilities corresponding to multiple preset actions.

[0096] Figure 7 This diagram illustrates the architecture of a policy model (PM) for reinforcement learning, according to an exemplary embodiment. This PM architecture utilizes a Transformer-based reinforcement learning framework, taking the historical and real-time states of the test system as input tokens to capture temporal dependencies and system dynamics. The actions generated by the model can include environmental manipulation (e.g., placing obstacles) and internal state changes (e.g., network degradation).

[0097] Because in the reinforcement learning process of this application, the higher the reward, the greater the probability that the corresponding action will generate a fault scenario. The above method incorporates the expected reward value when predicting the fault injection action, thereby increasing the accuracy of the final predicted fault injection action. Furthermore, the above method incorporates available action information when predicting the fault injection action, ensuring that the final predicted action is an available action.

[0098] In some embodiments, before or after obtaining the probabilities corresponding to the plurality of preset actions, the method may further include: screening the plurality of preset actions through an action sampler. The action sampler may be

[0099] Among them, screening multiple preset actions through the action sampler may include: obtaining action preferences, where the action preferences are determined based on historical data or preset preferences; and performing preference selection on multiple preset actions based on the action preferences.

[0100] In some embodiments, after performing preference selection on a plurality of preset actions through action preferences, the process may further include: sampling and action filtering the preset actions after the preference selection.

[0101] Figure 8 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0102] refer to Figure 8 , the above fault injection action prediction method may include the following steps.

[0103] Step S802: obtaining current status information of the system to be detected in the current time period, where the current status information includes at least one of the current operating status, configuration, performance, and health status of the system to be detected in the current time period.

[0104] Step S804: predict candidate actions of the system to be detected based on the current state information of the system to be detected.

[0105] In some embodiments, when predicting candidate actions of the system to be detected, historical state information before the current state information may also be combined, but this application does not impose any restrictions on this.

[0106] In some embodiments, the probability of each action may be predicted based on the current state information of the system to be detected, and one or more actions with higher probabilities may be the candidate actions.

[0107] Step S806: obtaining action preferences, where the action preferences are determined based on experience or expert preferences.

[0108] In some embodiments, the above-mentioned action preference may also be determined based on historical data or preset preferences.

[0109] In some embodiments, the action preference may also be a relatively preferred action or action probability set in combination with expert knowledge or experiential knowledge.

[0110] For example, based on experience, it is determined that the setting of action B often causes the system to be detected to malfunction. In this case, action B can be set as the preferred action, and its corresponding preference probability can be higher.

[0111] For another example, if action A is set too many times in the past or the setting of action A does not affect the failure result of the system to be detected, then action A can be set as a non-preferred action.

[0112] Step S808: performing preference selection on candidate actions based on action preferences to obtain a preference-selected action.

[0113] In some embodiments, the action preference may include at least one action, and each action may correspond to a probability.

[0114] In some embodiments, a preference selection can be performed on candidate actions using action preferences to obtain a preference-selected action.

[0115] For example, the probability of each action in the action preference can be multiplied by the predicted probability of the candidate action to determine the preference selection probability corresponding to each action; then the actions with the highest preference selection probability are used as fault injection actions.

[0116] Through the above method, action preferences can be pre-set in combination with experience or expert knowledge to screen out fault injection actions that meet the action preferences.

[0117] Figure 9 The figure is a flowchart of a fault injection action prediction method according to an exemplary embodiment.

[0118] In some embodiments, after the fault injection action is determined by the target neural network, the fault injection action can be screened in combination with expert knowledge or experience knowledge, so that the final fault injection action can better meet actual needs.

[0119] like Figure 10 As shown, a target neural network (which may be, for example, a reinforcement learning strategy model) 1002 may be used to perform predictive processing on the current state information of the detection system 1001 to obtain the action probability corresponding to each preset action.

[0120] like Figure 10 As shown, the action sampler 1003 can also be used to filter the actions output by the target neural network 1002 to filter out fault injection actions that meet the requirements.

[0121] Below, this application will combine Figure 9 The embodiment shown is used to describe the above process in detail.

[0122] refer to Figure 9 , the above fault injection action prediction method may include the following steps.

[0123] Step S902 : obtaining current status information of the system to be detected in the current time period, where the current status information includes at least one of the current operating status, configuration, performance, and health status of the system to be detected in the current time period.

[0124] Step S904: predict candidate actions of the system to be detected based on the current state information of the system to be detected.

[0125] like Figure 10As shown, the current state information of the system to be detected can be processed by the target neural network 1002 to predict the action probability of each candidate action of the system to be detected, where the candidate action can be an action that may become a fault injection action.

[0126] like Figure 10 As shown, after obtaining the action probability of each candidate action (or before), the action sampler 1003 can also be used to screen each candidate action.

[0127] Next, this application will combine steps S906 to S910 and Figure 11 The specific working process of the action sampler is explained and illustrated.

[0128] Step S906: obtaining action preferences, where the action preferences are determined based on experience or expert preferences.

[0129] Step S908: performing preference selection on candidate actions based on action preferences to obtain a preferred action.

[0130] Step S910 : sampling and filtering the preferred selected actions to determine the fault injection action.

[0131] In some embodiments, the preferred selection actions may be sampled and action filtered to determine the fault injection action.

[0132] In some embodiments, further sampling and filtering may be performed on the preference selection actions.

[0133] For example, a Top-k sampling method can be used to filter the preferred actions, and then the filtered actions can be further screened using the available action information to determine the fault injection actions.

[0134] Figure 11 The following is a working process of an action sampler according to an exemplary embodiment.

[0135] In some embodiments, candidate actions of the system to be detected and the probability of each candidate action (Action logits) may be predicted first based on the current state information of the system to be detected.

[0136] like Figure 11 As shown, the action preferences and the probabilities of each action preference (Preference logits) can be used to screen the candidate actions to obtain the preferred actions and the probabilities of the preferred actions.

[0137] like Figure 11 As shown in , Top-k sampling can also be used to sample the preferred actions to obtain unfiltered actions.

[0138] like Figure 11 As shown, the unfiltered actions can also be filtered using masks corresponding to usable actions (masked actions cannot be used) to obtain usable filtered actions.

[0139] like Figure 11 As shown, random sampling or the like can also be used to sample the filtered actions to obtain the final required fault injection actions.

[0140] Through the above method, the fault injection action finally screened out can be combined with the experience preference or expert preference, and the fault injection action can also be made usable.

[0141] Figure 12 The figure is a flow chart of a method for training a target neural network according to an exemplary embodiment.

[0142] In some embodiments, fault injection actions are predicted by a target neural network.

[0143] In some embodiments, the target neural network can be a policy model in a reinforcement learning framework.

[0144] refer to Figure 12 , the above target neural network can be trained through the following steps.

[0145] Step S1202: Obtain second historical state information of the system to be tested within a second historical time period, a second historical action corresponding to the second historical state information, and a second historical reward corresponding to the second historical action. The lower the safety of the scenario generated based on the second historical action, the greater the second historical reward.

[0146] In one embodiment, the higher the probability of a failure in the system to be detected in the scenario where the second historical action is generated, the greater the second historical reward.

[0147] Step S1204: Acquire third historical status information corresponding to the system to be detected in a third historical time period, where the third historical time period is a time period previous to the second historical time period.

[0148] Step S1206: Concatenate the second historical state information and the third historical state information to obtain second context-enhanced state information.

[0149] Step S1208: The target neural network predicts the second context-enhanced state information and determines a predicted action corresponding to the second historical state of the system under test and a predicted reward corresponding to the predicted action. The lower the safety of the scenario generated by the predicted action, the greater the predicted reward.

[0150] In some embodiments, the second context-enhanced state information can be input into the target neural network model to determine the predicted action corresponding to the system to be detected in the second historical state and the predicted reward corresponding to the predicted action.

[0151] In some embodiments, the probability of a failure of the system to be detected in a failure scenario generated by the predicted action may be directly proportional to the reward of reinforcement learning.

[0152] Step S1210: determining a first loss based on the second historical action and the predicted action, and determining a second loss based on the second historical reward and the predicted reward.

[0153] Step S1212: Train the target neural network according to the first loss and the second loss.

[0154] Through the above method, the trained target neural network can accurately predict the fault injection work that can generate the fault scenario according to the current state information of the system to be detected.

[0155] Below, the present application will describe in detail the fault detection method proposed in the present application in conjunction with the UTM system.

[0156] Testing in UTM is particularly challenging due to the long tail of potential failure scenarios. In UTM, the majority of operational scenarios can be safely managed by the system's self-healing design, while a small subset of rare and unpredictable conditions can lead to severe security risks and system-wide failures. These low-probability, high-risk scenarios are difficult to identify due to their low frequency and subtlety among the larger number of relatively safe scenarios. However, traditional testing methods such as random scenario injection and coverage-based techniques struggle to effectively detect rare, critical edge cases. The random nature of these methods leads to over-exploration of low-risk scenarios, resulting in wasted resources and a low probability of exposing the most dangerous vulnerabilities in an appropriate amount of time.

[0157] Research in fuzz testing and mutation testing has demonstrated the feasibility of automated testing by systematically exploring all possible configurations. However, due to the multiple mission objectives in UTM, test configuration introduces an additional timeline. Test cases can vary not only in their initial setup but also in the sequence of actions taken over time. To optimize efficiency, this embodiment uses a reduced set of test scenarios to represent the test space, rather than exhaustively exploring all possibilities along each parameter axis. This makes the quality of the scenarios a top priority.

[0158] This embodiment proposes a scenario-oriented testing framework to implement the above fault detection method.

[0159] In this application, the state information of the UTM system can be processed by a target neural network to predict the fault injection action of the UTM system.

[0160] To address the challenge of scene quality, this embodiment uses an RL (reinforcement learning) framework to construct a target neural network, and then uses an action sampler to screen fault injection actions.

[0161] like Figure 10 As shown in FIG, the scenario-oriented testing framework can be composed of two main modules: (1) a Transformer-based reinforcement learning policy model (PM) 1002 (i.e., a target neural network) for generating fault injection actions based on the real-time status data and historical data of the UTM system; (2) an action sampler (AS) 1003 for executing predefined security rules and filtering out unnecessary actions.

[0162] The construction of RL allows to simulate the complex temporal dependencies inherent in UTM (UAV Traffic Management) systems and learn effective strategies for exploring the state-action space. Specifically, let S represent the state space of the UTM system, A represent the set of possible actions or fault injections, and R: S × A → R is a reward function that quantifies the impact of actions on the safety and performance of the system. The goal is to find a set of actions that can make the cumulative reward Maximize the sequence of states and actions τ: {(s1,a1),...,(sn,an)}, where high rewards correspond to discovering critical failure modes. n is the number of sequences, i is an integer greater than or equal to 1 and less than or equal to n, s i Represents the state, a i In the following paragraphs, this embodiment defines the status, action, and reward in the UTM test.

[0163] State: For complex decision-making systems like UTM, the state includes temporal information, spatial information, and runtime mission status. This embodiment defines the state s∈S to include the current configuration of the UTM system, including: power information (position, velocity, and acceleration of all drones), environmental data (obstacles (such as whether obstacles are placed), weather conditions, and airspace restrictions), and mission-specific details (such as battery power, payload capacity, and route destination). To capture temporal dependencies, this embodiment represents the state as a sequence of observations within a fixed time window T, s={o1,...,ot}, where each is the vector of d relevant features corresponding to time step i. t is an integer greater than or equal to 1, and i is an integer greater than or equal to 1 and less than or equal to t.

[0164] Actions (scenario setting actions or scenario generating actions): The action space A contains a set of discrete fault injection operations, each targeting a specific component or aspect of the UTM system. This embodiment defines A as the Cartesian product of two sets A = D × F, where D represents the set of n available target drones and F is the set of m available actions. Formally, D = {d1, d2, ..., dn}, F = {f1, f2, ..., fm}, n is an integer greater than or equal to 1, and m is an integer greater than or equal to 1. Therefore, each action a∈A is a tuple (d, f), where d∈D and f∈F. This construction allows for a combinatorial exploration of fault scenarios.

[0165] Reward: The reward function R is designed to capture the safety and operational efficiency of the system, as represents the reward at time step t, where r i (s t ,a t ) is a single reward component (e.g., collision avoidance, task completion, system stability), α i are their respective weights.

[0166] Next, this application will introduce the offline reinforcement learning training process of the policy model.

[0167] This embodiment requires pre-collection of UTM system trajectories as a sample dataset, represented as Where s is the current state, a is the action taken, r is the reward received, and s′ is the resulting next state. In this embodiment, the goal of reinforcement learning is constructed as searching for Where E stands for expectation, T stands for trajectory, and the goal is to find an optimal action selection strategy to maximize the average value of cumulative rewards), where θ is the parameter of the policy network, R π is the cumulative reward under policy π. This example uses a Transformer-based architecture that models complex temporal relationships through a self-attention mechanism. This architecture enables the model to discover complex failure modes over longer time horizons that would be overlooked by limited temporal reasoning capabilities.

[0168] In addition, this embodiment incorporates context-aware scenario generation to enhance the generalization capabilities of its policy model and address the challenge of distribution shift between offline data and online deployment. This embodiment lets C denote a context set consisting of a small number of relevant historical trajectories. This embodiment augments the state representation to include this context s~ = [C; s], where [·; ·] represents concatenation. This context-aware construction allows the model to adjust its behavior based on relevant historical examples, potentially improving performance in new or underrepresented scenarios.

[0169] The above reinforcement learning method can be used to train a policy model so that the policy model can be used to process the state information of the system to be detected and predict the fault injection action of the system to be detected.

[0170] In addition, to ensure the physical credibility and relevance of the generated scenarios, this embodiment introduces a constrained action sampling mechanism. Let Φ(s) represent a set of domain-specific constraints that define the space of possible actions given the current state s. After obtaining the reinforcement learning policy, this embodiment performs an action selection process a~π(a|s)·1[a∈Φ(s)], where π(a|s) is the policy learned by the target neural network and 1[·] is an indicator function. This indicator function only takes effect during the inference process, allowing expert knowledge and system-specific constraints to be combined without affecting the learning efficiency and flexibility of the learning policy.

[0171] In some embodiments, the constrained action sampling mechanism may only function during testing (and may not function during training).

[0172] In general, this embodiment proposes a scenario-oriented fault detection method to search for long-tail conditions, so as to accelerate the fault detection process of the system to be detected and improve the fault detection accuracy.

[0173] Below, this application will provide a detailed description of the above fault detection method.

[0174] In this section, this example introduces an automatic framework capable of complex scene generation and an interface for prior preference alignment and knowledge accumulation. Figure 10 As shown, in this framework, this embodiment uses a Transformer-based model as a policy model (PM) to start the process by generating a set of actions based on the system state. Based on the action space defined above, actions can be generated, which can be interpreted as potential fault injections to be applied to the UTM. Subsequently, these actions are sampled by an action sampler (AS). The AS has two purposes: (1) to ensure that the actions generated by the PM are usable in a specific UTM context. (2) to leverage expert knowledge to resample the selected actions and agents. Only the sampled actions can be injected into the system test. After the system test is completed, a new system state and an evaluation (reward) of the action are generated and fed back to the PM. Therefore, the PM continuously improves the scenario to discover potential vulnerabilities.

[0175] This example will first describe the design of the Policy Model (PM). The Policy Model serves as the generation engine for this framework, leveraging the Transformer architecture to capture complex temporal dependencies and system dynamics. During training, the PM models the UTM trajectory sequences and learns internal properties from offline datasets. During inference, the PM processes the real-time UTM context and generates recommended fault injection actions for the AS.

[0176] Time series and action modeling: This embodiment designs a unified time series format, where each agent's observation o (equivalent to state S), action a, return R, and reward r are embedded in a homogeneous space after linear projection. The original reward r is used to construct a summary token by summarizing the increments of the last T time steps, similar to constructing the return-to-go token R (summarizing T incoming time steps). Therefore, when the center of attention is at the current time step T, the input sequence carries a total of 3×T time steps of data. The tokens will then be arranged into a sequence of length T time steps.<O,R,A> Array of tuples.

[0177] Embedding and Causality: To enhance the modeling of causal dependencies in the policy model, this example uses a multifaceted approach. This example augments sequentially sampled multi-agent drone observation data with position embedding. In addition, Figure 7 The input sequence is augmented with different classification (CLS) tokens as a powerful discriminator to reduce the ambiguity of the predicted target. In addition, the most recent observation is placed closest to the CLS token, ensuring that the model pays special attention to the latest information when making decisions.

[0178] like Figure 7 As shown, this embodiment can sequentially express the prediction task of reinforcement learning as a fault detection method Where S represents the summary token of the previous T time steps, o represents the state information in the current time step, and f θ represents the Transformer model with parameters θ, Mt is the constraint, R t is the expected return value (Return-to-go).

[0179] Next, this embodiment will introduce the action sampler AS.

[0180] Inductive bias and generalization are major drawbacks of traditional offline RL methods. This example designs a set of sampling strategies as a solution. In this section, we first introduce preference bias as a representation of human feedback. We then describe the action sampler function between the PM and the SUT.

[0181] This embodiment introduces preference bias and adds domain expert knowledge to unify the imbalance of model predictions and the gap between human prior preferences.

[0182] Action sampling: such as Figure 11 As shown, the action logits predicted by PM can be augmented and compensated based on the preference probability. To address the long-tail effect and improve fairness, Top-K sampling is introduced after augmentation based on the preference probability to maintain variance. Considering the realistic capabilities of the system state, an immediate action mask is applied to filter out intolerable actions. Finally, the masked logits are obtained through uniform sampling.

[0183] The framework of this embodiment achieves a balance between exploring complex failure scenarios and adhering to real-world constraints by combining the generative power of a Transformer-based policy model with a sophisticated selection process for action samplers. This approach makes UTM system testing more efficient and effective, discovering critical vulnerabilities that traditional methods might miss.

[0184] This example utilizes a Transformer-based policy model to capture the dynamics of the subject UTM system from an offline dataset to accelerate exploitation. This example's approach was evaluated for over 700 hours in a large-scale, industrial-grade simulation environment. Empirical results demonstrate that this example's approach improves vulnerability discovery efficiency by over eight times compared to traditional expert-guided random walk exploitation, demonstrating the potential of machine learning to enhance mission-critical systems.

[0185] Results show that our framework accelerates vulnerability discovery by over eight times compared to traditional methods, while also identifying critical scenarios that traditional techniques fail to detect. Furthermore, we investigated the effect of model scaling and found that increasing the parameter size to 2 billion significantly improved performance. This highlights the scalability and robustness of our approach for improving the security of mission-critical systems.

[0186] This example proposes a novel scenario-oriented testing framework for identifying vulnerabilities in mission-critical systems, particularly for UTM. The method of this example utilizes a Transformer-based policy model to deal with the long-tail effect and efficiency challenges in fault detection. The use of context in the policy model improves the generalizability in unseen areas. The results of this example highlight the potential of learning and expert hybrid methods in strengthening mission-critical systems. End-to-end autoregressive learning methods are worthy of investigation. Future work can explore the application of this framework to other mission-critical areas beyond UTM, such as autonomous vehicles or industrial control systems.

[0187] It should be noted that the various steps in the various embodiments of the above-mentioned fault detection method can be interleaved, replaced, added, or deleted. Therefore, these reasonable permutations and combinations of the fault detection method should also fall within the scope of protection of this disclosure, and the scope of protection of this disclosure should not be limited to the embodiments.

[0188] Based on the same inventive concept, the present disclosure also provides a fault detection device, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0189] Figure 13 FIG. 1 is a block diagram of a fault detection device according to an exemplary embodiment. Figure 13 The fault detection device 1300 provided by the embodiment of the present disclosure may include: a current state information acquisition module 1301, a fault injection action prediction module 1302 and a fault determination module.

[0190] Among them, the current status information acquisition module 1301 can be used to obtain the current status information of the system to be detected in the current time period, and the current status information includes at least one of the current operating status, configuration, performance and health status of the system to be detected in the current time period; the fault injection action prediction module 1302 can be used to predict the fault injection action of the system to be detected based on the current status information of the system to be detected, so as to generate a fault scenario through the fault injection action; the fault determination module 1303 can be used to determine whether the system to be detected has a fault under the fault scenario.

[0191] It should be noted that the current state information acquisition module 1301, fault injection action prediction module 1302, and fault determination module 1303 described above correspond to S202 to S206 in the method embodiment. These modules and corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in the method embodiment. It should be noted that these modules, as part of the apparatus, can be executed in a computer system, such as a set of computer-executable instructions.

[0192] In some embodiments, the fault injection action prediction module 1302 may include: a first historical state information acquisition submodule, a first splicing submodule, and a first fault injection action prediction submodule.

[0193] Among them, the first historical status information acquisition submodule can be used to obtain the first historical status information of the system to be detected within the first historical time period, and the first historical time period is the time period before the current time period; the first splicing submodule can be used to splice the first historical status information with the current status information to obtain the first context-enhanced status information; the first fault injection action prediction submodule can be used to predict the fault injection action of the system to be detected based on the first context-enhanced status information.

[0194] In some embodiments, the first fault injection action prediction submodule may include: a reward value acquisition unit and a fault injection action determination unit.

[0195] Among them, the reward value acquisition unit can be used to obtain the expected reward value and available action information; the fault injection action determination unit can be used to predict the fault injection action of the system to be detected based on the first context enhanced state information, the expected reward value and the available action information.

[0196] In some embodiments, the first fault injection action prediction submodule includes: a probability prediction unit and a probability selection unit.

[0197] Among them, the probability prediction unit can be used to input the first context enhanced state information into the target neural network model to obtain the probabilities corresponding to multiple preset actions; the probability selection unit can be used to select one or more actions with the highest probability from multiple preset actions as fault injection actions.

[0198] In some embodiments, the target neural network model is trained through reinforcement learning, the probability of failure of the system to be detected in a fault scenario is positively correlated with the reward of reinforcement learning, the preset action is the available action among the actions corresponding to the system to be detected, and the probability prediction unit may include: a splicing sub-unit.

[0199] Among them, the splicing subunit can be used to splice one or more of the first context enhanced state information, rewards, and action masks of multiple preset actions, and input them into the target neural network model to obtain the probabilities corresponding to multiple preset actions.

[0200] In some embodiments, fault injection actions are predicted by a target neural network.

[0201] In some embodiments, the fault detection device 1300 may further include: a second historical state information acquisition module, a third historical state information acquisition module, a second splicing module, a prediction module, a loss determination module, and a training module.

[0202] Among them, the second historical state information acquisition module can be used to obtain the second historical state information of the system to be detected within the second historical time period, the second historical action corresponding to the second historical state information, and the second historical reward corresponding to the second historical action; wherein the higher the probability of failure of the system to be detected in the scenario generated by the second historical action, the greater the second historical reward; the third historical state information acquisition module is used to obtain the third historical state information corresponding to the system to be detected within the third historical time period, and the third historical time period is the previous time period of the second historical time period; the second splicing module is used to splice the second historical state information and the third historical state information to obtain the second context-enhanced state information; the prediction module is used to input the second context-enhanced state information into the target neural network model to determine the predicted action corresponding to the system to be detected in the second historical state and the predicted reward corresponding to the predicted action; the loss determination module is used to determine the first loss based on the second historical action and the predicted action, and to determine the second loss based on the second historical reward and the predicted reward; the training module is used to train the target neural network based on the first loss and the second loss.

[0203] In some embodiments, the first fault injection action prediction submodule further includes: a preset action screening unit.

[0204] The preset action screening unit may be configured to screen the plurality of preset actions through the action sampler before or after obtaining the probabilities corresponding to the plurality of preset actions.

[0205] In some embodiments, the preset action screening unit may include: an action preference determining subunit and a preference selecting subunit.

[0206] The action preference determination subunit may be used to obtain action preferences, which are determined based on historical data or preset preferences; and the preference selection subunit may be used to perform preference selection on multiple preset actions based on the action preferences.

[0207] In some embodiments, the preset action screening unit may further include: a sampling subunit.

[0208] The sampling subunit may be used to sample and filter the preset actions after preference selection is performed on a plurality of preset actions through action preferences.

[0209] In some embodiments, the system to be detected is a drone traffic management system, and the current status information includes one or more of the power information, environmental data, weather conditions, airspace restrictions, battery information, load information, and route information of the drones managed by the drone traffic management system.

[0210] In some embodiments, the fault injection action prediction module 1302 may include: a candidate action prediction submodule, an action preference acquisition submodule, and a preference selection submodule.

[0211] Among them, the candidate action prediction submodule can be used to predict the candidate actions of the system to be detected based on the current state information of the system to be detected; the action preference acquisition submodule can be used to obtain action preferences, which are determined based on experience or expert preferences; the preference selection submodule can be used to select candidate actions based on action preferences to obtain fault injection actions.

[0212] In some embodiments, the preference selection submodule may include: a preference selection action determination unit and an action filtering unit.

[0213] The preference selection action determination unit is used to perform preference selection on candidate actions through action preferences to obtain preference selection actions; the action filtering unit is used to sample and filter the preference selection actions to determine the fault injection action.

[0214] Since the functions of the apparatus 1300 have been described in detail in the corresponding method embodiments, they will not be described in detail in this disclosure.

[0215] The modules and / or submodules and / or units described in the embodiments of the present disclosure may be implemented in software or hardware. The modules and / or submodules and / or units described may also be provided in a processor. The names of these modules and / or submodules and / or units do not, in certain circumstances, limit the modules and / or submodules and / or units themselves.

[0216] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module or a portion of a program segment, which contains one or more executable instructions for implementing a specified logical function.

[0217] Figure 14 Schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure is shown. Figure 14 The electronic device 1400 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0218] like Figure 14As shown, electronic device 1400 includes a central processing unit (CPU) 1401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1402 or a program loaded from a storage portion 1408 into a random access memory (RAM) 1403. Various programs and data required for the operation of electronic device 1400 are also stored in RAM 1403. CPU 1401, ROM 1402, and RAM 1403 are connected to each other via a bus 1404. An input / output (I / O) interface 1405 is also connected to bus 1404.

[0219] The following components are connected to the I / O interface 1405: an input section 1406 including a keyboard, a mouse, and the like; an output section 1407 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1408 including devices such as a hard disk; and a communication section 1409 including a network interface card such as a LAN card or a modem. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the I / O interface 1405 as needed. Removable media 1411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1410 as needed, so that computer programs read therefrom can be installed in the storage section 1408 as needed.

[0220] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes computer program instructions for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1409, and / or installed from a removable medium 1411. When the computer program is executed by the central processing unit (CPU) 1401, the above-mentioned functions defined in the system of the present disclosure are performed.

[0221] It should be noted that the computer-readable storage medium described in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable computer program instructions. Such a propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0222] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by a device, the device can implement functions including: obtaining current status information of the system to be detected in the current time period, the current status information including at least one of the current operating status, configuration, performance, and health status of the system to be detected in the current time period; predicting a fault injection action of the system to be detected based on the current status information of the system to be detected, so as to generate a fault scenario through the fault injection action; and determining that the system to be detected has failed under the fault scenario.

[0223] According to one aspect of the present disclosure, a computer program product or computer program is provided, the computer program product or computer program including computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium and executed by a processor to implement the methods provided in various optional implementations of the above embodiments.

Claims

1. A fault detection method, characterized in that: include: Acquire current status information of the system to be detected in a current time period, wherein the current status information includes at least one of a current operating status, configuration, performance, and health status of the system to be detected in the current time period; Predicting a fault injection action of the system to be detected based on current state information of the system to be detected, so as to generate a fault scenario through the fault injection action; It is determined that the system to be detected fails under the failure scenario.

2. The method according to claim 1, characterized in that: Predicting a fault injection action of the system to be detected according to the current state information of the system to be detected includes: Acquire first historical status information of the system to be detected in a first historical time period, where the first historical time period is a time period before the current time period; Concatenating the first historical state information with the current state information to obtain first context-enhanced state information; A fault injection action of the system to be detected is predicted according to the first context enhanced state information.

3. The method according to claim 2, characterized in that: Predicting a fault injection action of the system to be detected according to the first context enhanced state information includes: Inputting the first context-enhanced state information into a target neural network model to obtain probabilities corresponding to a plurality of preset actions; From the multiple preset actions, one or more actions with the highest probability are selected as the fault injection action.

4. The method according to claim 3, characterized in that The target neural network model is trained by reinforcement learning, the probability of the failure of the system to be detected in the failure scenario is proportional to the reward of the reinforcement learning, the preset action is an available action among the actions corresponding to the system to be detected, and the first context enhanced state information is input into the target neural network model to obtain the probabilities corresponding to multiple preset actions, including: The first context-enhanced state information, the reward, and one or more of the action masks of the multiple preset actions are concatenated and input into the target neural network model to obtain the probabilities corresponding to the multiple preset actions.

5. The method according to claim 1, characterized in that: The fault injection action is predicted by a target neural network; wherein the training method of the target neural network includes: Acquire second historical state information of the system to be detected within a second historical time period, a second historical action corresponding to the second historical state information, and a second historical reward corresponding to the second historical action; wherein, the higher the probability of failure of the system to be detected in the scenario generated by the second historical action, the greater the second historical reward; Acquire third historical status information corresponding to the system to be detected in a third historical time period, where the third historical time period is a time period before the second historical time period; Concatenate the second historical state information and the third historical state information to obtain second context enhanced state information; Inputting the second contextual enhanced state information into the target neural network model to determine a predicted action corresponding to the system to be detected under the second historical state and a predicted reward corresponding to the predicted action; Determine a first loss according to the second historical action and the predicted action, and determine a second loss according to the second historical reward and the predicted reward; The target neural network is trained according to the first loss and the second loss.

6. The method according to claim 3, characterized in that: Before or after obtaining the probabilities corresponding to the plurality of preset actions, the method further includes: The multiple preset actions are filtered through an action sampler.

7. The method according to claim 6, characterized in that The plurality of preset actions are screened by an action sampler, including: Obtaining action preferences, where the action preferences are determined based on historical data or preset preferences; The plurality of preset actions are selected by preference according to the action preference.

8. The method according to claim 7, characterized in that: After the plurality of preset actions are selected by preference through the action preference, the method further includes: The preset actions after the preference selection are sampled and action filtered.

9. The method according to claim 1, characterized in that: The system to be detected is an unmanned aerial vehicle traffic management system, and the current status information includes one or more of the power information, environmental data, weather conditions, airspace restrictions, battery information, load information, and route information of the unmanned aerial vehicle managed by the unmanned aerial vehicle traffic management system.

10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement the fault detection method according to any one of claims 1 to 9.

Images