Information processing system, information processing method, and computer-readable storage medium

Abstract

The present application provides an information processing system, an information processing method, and a computer readable storage medium capable of verifying an abnormal event based on abnormal time series data including a plurality of signal data. The information processing system includes: an acquisition unit that acquires a plurality of time series data, each time series data including a plurality of signal data, the plurality of time series data including abnormal time series data and a plurality of normal time series data; and a verification unit that determines, for each signal data, whether the abnormal time series data and each normal time series data in each time interval are similar to each other, and verifies an abnormal event based on the determination result.

Description

Technical Field

[0001] This disclosure relates to an information processing system, information processing method, and program, and more particularly to a technique for verifying abnormal events. Background Technology

[0002] International Publication No. 2020 / 245968 discloses an information processing system for detecting anomalous events based on the time-series data of an object. Summary of the Invention

[0003] An information processing system is desired that can perform verification of anomalous events (e.g., detection of anomalous events or determination of the cause of anomalous events) based on anomalous time series data containing multiple signal data (e.g., location data of multiple objects).

[0004] This disclosure was made to solve such a problem, and its purpose is to provide an information processing system, information processing method and program that can verify anomalous events based on anomalous time series data containing multiple signal data.

[0005] The information processing system in this embodiment includes: an acquisition unit that acquires multiple time series data, each time series data including multiple signal data, the multiple time series data including abnormal time series data and multiple normal time series data; and a verification unit that, for each signal data, determines whether the abnormal time series data and each normal time series data in each time interval are similar to each other, and verifies the abnormal event based on the determination result.

[0006] The information processing method in this embodiment includes: a step of a computer acquiring multiple time series data, each time series data containing multiple signal data, the multiple time series data including abnormal time series data and multiple normal time series data; a step of judging whether the abnormal time series data and each normal time series data in each time interval are similar to each other for each signal data, and verifying the abnormal event based on the judgment result.

[0007] The program in this embodiment causes a computer to execute the following information processing method, which includes: a step of acquiring multiple time series data, each time series data containing multiple signal data, the multiple time series data including abnormal time series data and multiple normal time series data; a step of judging whether the abnormal time series data and each normal time series data in each time interval are similar to each other for each signal data, and verifying the abnormal event based on the judgment result.

[0008] According to this disclosure, an information processing system, information processing method, and program can be provided that can verify anomalous events based on anomalous time series data containing multiple signal data.

[0009] The above and other objects, features and advantages of this disclosure will be more fully understood from the detailed description given below and the accompanying drawings, which are given by way of example only, and should therefore not be considered as limitations on this disclosure. Attached Figure Description

[0010] Figure 1 This is a block diagram illustrating the structure of the information processing apparatus according to Embodiment 1.

[0011] Figure 2 This is a diagram used to illustrate the similarity of each signal combination.

[0012] Figure 3 This is a flowchart illustrating the flow of the information processing method involved in Implementation Method 1.

[0013] Figure 4 A schematic diagram to represent the outline of a simulation of robots passing each other.

[0014] Figure 5 A schematic diagram showing the robot's starting point and destination.

[0015] Figure 6 A summary diagram to illustrate the overview of search-based testing.

[0016] Figure 7 This is a diagram used to illustrate failed target points.

[0017] Figure 8 This is a diagram used to illustrate the successful data set.

[0018] Figure 9 This diagram illustrates the method used to extract similar IDs.

[0019] Figure 10 This is a diagram used to explain the topic periodically.

[0020] Figure 11(A) is a diagram used to illustrate the results of determining the cause signal.

[0021] Figure 11(B) is a diagram used to illustrate the results of determining the cause signal. Detailed Implementation

[0022] Hereinafter, although the present invention will be described through embodiments, it is not intended to limit the invention to these embodiments. Furthermore, not all structures described in the embodiments are necessarily necessary means to solve the problem.

[0023] Implementation Method 1

[0024] Hereinafter, the information processing apparatus according to Embodiment 1 will be described with reference to the accompanying drawings. Figure 1 This is a block diagram illustrating the structure of the information processing apparatus 100 according to Embodiment 1. The information processing apparatus 100 is an example of an information processing system. The information processing apparatus 100 may also be an edge terminal. A system that performs processing within an edge terminal may also be included in the information processing system. As explained last, the information processing system may also include a server.

[0025] The information processing apparatus 100 includes an acquisition unit 110, a data extraction unit 120, and a verification unit 130. Furthermore, the information processing apparatus 100 includes a processor and a memory (not shown). By executing a program through the processor, the information processing apparatus 100 functions as the acquisition unit 110, the data extraction unit 120, and the verification unit 130.

[0026] The acquisition unit 110 acquires multiple time-series data. Each time-series data set contains multiple signal data. These multiple time-series data sets include anomalous time-series data and multiple normal time-series data sets. Anomalous time-series data are also referred to as failed target data or defective data. Anomalous time-series data can also be data that violates system requirements. Normal time-series data are also referred to as successful data. Although the following explanation focuses on the case where each time-series data set is simulated data, it can also be real-world experimental data.

[0027] The acquisition unit 110 can also perform search-based tests aimed at finding data (defect data) that violates system requirements, and extract the required information from the test results. This allows the user to efficiently analyze the defect data. As a result of the search-based tests, the acquisition unit 110 can also receive information related to whether each simulation data meets the system requirements. Search-based testing can efficiently search for violations of system requirements through optimization.

[0028] In addition, the acquisition unit 110 can also perform tests other than search-based tests. For example, the acquisition unit 110 can also perform tests with simulation conditions set at equal intervals (e.g., grid tests) or tests with simulation conditions arbitrarily set by the user.

[0029] Specifically, the acquisition unit 110 simulates the movement of multiple objects (e.g., robot R1, robot R2, and robot R3) whose relative positions change according to time. In this case, the starting point and starting time of each object can be set as simulation conditions.

[0030] The following explanation focuses on the case of simulating the movement of multiple objects. However, the time series data only needs to be data used to verify whether the actions of the control system meet system requirements under various conditions. The control of the objects could be, for example, engine control, merging control of autonomous vehicles, or smart grid control.

[0031] In engine control, the deviation between the target and measured vehicle speed can be verified using multiple signal data (e.g., accelerator and brake input trajectories) conditioned on the inputs of the accelerator and brakes. In merging control of autonomous vehicles, collisions at various initial positions of the autonomous vehicle can be verified using multiple signal data (e.g., the autonomous vehicle's movement trajectory). In smart grid control, power surpluses and shortages in various facilities under various supply and demand conditions can be verified using multiple signal data (e.g., electricity consumption and power generation).

[0032] The acquisition unit 110 can acquire multiple time series data by extracting the required information from the results of search-based tests, etc. As described above, each time series data contains multiple signal data. For example, the acquisition unit 110 can set the (x, y) data of robot R1 as the first signal data, the (x, y) data of robot R2 as the second signal data, and the (x, y) data of robot R3 as the third signal data based on the simulation results of robots R1, R2, and R3 passing each other.

[0033] Furthermore, the acquisition unit 110 can set any data associated with unexpected actions as signal data. For example, in a robot control simulation, the robot's (x, y) data can be set as first signal data, and the obstacle's position data as second signal data. In this case, unexpected actions of robot movement within a specific obstacle configuration can be detected. In an engine control simulation, the accelerator input trajectory can be set as first signal data, the brake input trajectory as second signal data, the external temperature as third signal data, and the gear position as fourth signal data.

[0034] The data extraction unit 120 extracts anomalous and normal time series data from multiple time series datasets. For example, anomalous time series data may sometimes be included in time series data related to the movement of multiple objects due to the interaction of these objects. Specifically, when objects pass by other objects, they may sometimes collide, stop, or detour in ways that are not necessary. Such phenomena are referred to as anomalous events or unexpected actions.

[0035] For example, time series data where the intended object arrives at its destination before the target time (e.g., 120 seconds after the start of the simulation) is classified as normal time series data. All other time series data are classified as abnormal time series data.

[0036] Specifically, the data extraction unit 120 classifies simulated data that violates system requirements as abnormal time-series data. System requirements can be arbitrarily determined by the user. For example, system requirements could be the arrival of a predetermined moving object at its destination within a predetermined time. Information related to whether system requirements are violated can be included in the results of a search-based test. Defective data can be judged based on thresholds or on whether conditions expressed in STL (Signal Temporal Logic) are met.

[0037] Furthermore, the data extraction unit 120 may extract defective data based on the requirements of a subsystem, rather than the requirements of the entire system. For example, sometimes, in a system requirement that multiple objects (e.g., autonomous moving bodies) reach their destination within a time period, there is a subsystem requirement that the position inference error of each object is below a threshold. In such cases, the data extraction unit 120 can classify the simulated data where the position inference error of each object is below the threshold as normal time series data, and classify the other simulated data as abnormal time series data.

[0038] The verification unit 130 determines, for each signal data, whether the abnormal time series data and the normal time series data within each time interval are similar to each other. For example, the verification unit 130 may determine whether the success point described later is included within a predetermined range of the failure target point described later.

[0039] The verification unit 130 verifies the abnormal event based on the judgment result. The verification of the abnormal event includes at least one of the following: detecting the timing of the abnormal event (hereinafter referred to as the project timing) and determining the cause of the abnormality. Determining the cause of the abnormality refers to identifying the signal data associated with the cause of the abnormal event from among the multiple signal data contained in the abnormal time series data. The verification unit 130 can determine a single signal or a combination of signals.

[0040] The verification unit 130 includes a partial time serialization unit 131, a parameter calculation unit 132, an ID extraction unit 133, a similarity calculation unit 134, a timing determination unit 135, and a signal determination unit 136.

[0041] The partial time serialization unit 131 first performs partial time serialization on each signal data. For example, the partial time serialization unit 131 generates partial time series data by sliding a 10-second time window by 1 second at a time. In this case, the time interval is represented, for example, as [0-10, 1-11, ..., 110-120 (sec)]. The width of the window is not limited to 10 seconds, and the sliding step is not limited to 1 second. For example, if the time window starts entirely from 0 seconds, the time interval is represented as [0-1, 0-2, ..., 0-120 (sec)].

[0042] In the following text, there are instances where the data obtained after partially time-series processing of signal data is referred to as partial time-series signal data. There are also instances where the data obtained after partially time-series processing of signal data contained in abnormal time-series data is referred to as the first part of the time-series signal data. And there are instances where the data obtained after partially time-series processing of signal data contained in normal time-series data is referred to as the second part of the time-series signal data. Partial time-series data are also referred to as time-series data within each time interval.

[0043] The parameter calculation unit 132 calculates the characteristic parameters of each time series data in each time interval for each signal data. In other words, the parameter calculation unit 132 calculates the characteristic parameters of each part of the time series signal data. These characteristic parameters are also called characteristic points. The characteristic parameters of the first part of the time series signal data are called failure target points, and the characteristic parameters of the second part of the time series signal data are called success points.

[0044] The parameter calculation unit 132 can also calculate the feature data by, for example, extracting the position coordinates (e.g., the x-coordinate and y-coordinate of robot R1) of robot R1 at each of multiple time points (e.g., 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, and 30 seconds after the start of the simulation) and arranging (connecting) the extracted position coordinates.

[0045] The method for calculating feature parameters is not limited to the method described above, which involves extracting signal data values ​​from multiple time points and concatenating them. The parameter calculation unit 132 can also use methods such as sparse coding, wavelet transform, shapelet transform, singular spectral decomposition, and nonnegative matrix decomposition to calculate feature data.

[0046] For each signal data, the ID extraction unit 133 extracts the ID (hereinafter referred to as the similar ID) of normal time series data that is similar to the abnormal time series data in each time interval. For example, the ID extraction unit 133 calculates the feature parameters of each time series data in each time interval for each signal data.

[0047] Specifically, the ID extraction unit 133 determines the successful points contained within a predetermined range of the failed target points and extracts the IDs of the normal time series data corresponding to the determined successful points. In other words, if the Euclidean distance between the feature parameters of the first part of the time series data and the feature parameters of the second part of the time series data is below a threshold, the IDs of the normal time series data corresponding to the second part of the time series data are extracted as similar IDs.

[0048] The ID extraction unit 133 may extract similar IDs based on Mahalanobis distance or cosine similarity, rather than Euclidean distance. Furthermore, the ID extraction unit 133 may use any method, such as probability calculation based on a probability density distribution estimated from kernel density, to determine the similarity between the first part of the time series data and the second part of the time series data. In such a case, the information processing apparatus 100 may not need to include the parameter calculation unit 132. The ID extraction unit 133 may also calculate characteristic values ​​(e.g., average, dispersion, frequency characteristics) based on feature parameters and extract similar IDs based on the similarity of these characteristic values.

[0049] The similarity calculation unit 134 calculates the common portion of similar IDs for each signal combination and calculates the time variation of the number of similar IDs contained in the common portion. The number of similar IDs is also called the similarity. Here, the signal combination includes not only combinations of two signal data, but also combinations of one signal data (single signal). The signal combination may also include combinations of three or more signal data. The signal combination includes combinations of all signal data. Since the similar IDs are extracted for each part of the time series, the similarity calculation unit 134 can calculate the time variation of the number of similar IDs.

[0050] The following describes the case where each time series data contains three signal data (e.g., signal 1, signal 2, and signal 3). In this case, the signal combinations are categorized into six types: combination 1, combination 2, combination 3, combination 1·2, combination 1·3, combination 2·3, and combination 1·2·3. Combinations 1, 2, and 3 are single-signal combinations of signals 1, 2, and 3, respectively. Combination 1·2 is a combination of signals 1 and 2, combination 1·3 is a combination of signals 1 and 3, and combination 2·3 is a combination of signals 2 and 3. Combinations 1·2·3 are referred to as all signal combinations or all combinations, which are combinations of signals 1, 2, and 3.

[0051] Reference Figure 2 The time variation of the number of similar IDs calculated by the ID extraction unit 133 is explained. Figure 2The top of the graph shows the sets of similar IDs A1, A2, A3, A12, A13, A23, and A123 for each of the six types of signal combinations (combination 1, combination 2, combination 3, combination 1·2, combination 1·3, combination 2·3, combination 1·2·3). Sets A1, A2, and A3 represent the sets of similar IDs in a single signal. The bottom of the graph shows the temporal variation of the similarity C1 for signal 1 (combination 1), the similarity C2 for signal 2 (combination 2), and the similarity C3 for signal 3 (combination 3). The vertical axis of the graph represents similarity, and the horizontal axis represents time.

[0052] The common components of combinations of two or more signals will be explained below. Furthermore, the term "common component" can also be applied to a single signal. For example, there exists a set A1 that is called the common component of a signal A1.

[0053] Sets A12, A13, and A23 represent sets of similar IDs in two signal combinations. Set A12 of similar IDs in combination 1.2 is the common part (intersection) of sets A1 and A2. Set A13 of similar IDs in combination 1.3 is the common part of sets A1 and A3. Similar IDs in combination 2.3 are the common part of sets A2 and A3. The time variations of similarity C12 in combination 1.2, similarity C13 in combination 1.3, and similarity C23 in combination 2.3 are shown below. The vertical axis of the graph represents similarity, and the horizontal axis represents time.

[0054] Set A123 represents the set of similar IDs in the three signal combinations. The set A123 of similar IDs in combinations 1, 2, and 3 (all combinations) is a common part of sets A1, A2, and A3. Below, the time variation of the similarity C123 count in combinations 1, 2, and 3 is shown. The vertical axis of the graph represents similarity, and the horizontal axis represents time.

[0055] Return to Figure 1 Continuing the explanation, the timing determination unit 135 determines the timing of issues related to the occurrence of abnormal events based on the time change of the similarity (the number of IDs included in the common part mentioned above) among all signal combinations. Specifically, the timing determination unit 135 determines the timing when the similarity among all combinations is below a threshold.

[0056] The signal determination unit 136 determines at least one of a single signal and a combination of signals associated with the cause of the abnormal event based on the time change of similarity (the time change of the number of IDs included in the common part mentioned above). It may be a single signal data that is associated with the cause of the abnormal event, or it may be a combination of signal data that is associated with the cause of the abnormal event.

[0057] Based on the aforementioned issue timing, the signal determination unit 136 determines single signals and signal combinations associated with the cause of the abnormal event. Specifically, if the number of similar IDs (similarity) in any signal data is below a threshold at the issue timing, the signal data is associated with the cause of the abnormal event. Furthermore, if the number of similar IDs (similarity) in any signal combination is below a threshold at the issue timing, the signal combination is associated with the cause of the abnormal event. The threshold can also be set to a different value for each combination.

[0058] Next, refer to Figure 3 The information processing method described in Embodiment 1 will now be explained. Specifically, the case of simulating the passing of three robots and performing a search-based test verification will be explained. First, the acquisition unit 110 of the information processing device 100 acquires the results of the search-based test verification implemented by the simulation (step S101). Assume that among the 500 simulation data, there are 134 defect data (e.g., data where the robot failed to reach the destination as planned) and 366 success data.

[0059] Figure 4 This is a schematic diagram to illustrate the simulation. Robots R1, R2, and R3 intersect at a T-shaped passage (corner) B. Symbol T1 represents the trajectory of robot R1, symbol T2 represents the trajectory of robot R2, and symbol T3 represents the trajectory of robot R3.

[0060] Reference Figure 5 This section describes the starting and ending points of robots R1, R2, and R3. The positions of robots R1, R2, and R3 indicate their starting points. Symbol T1 schematically represents the movement trajectory of robot R1, with the top of the arrow indicating its destination. Symbol T2 schematically represents the movement trajectory of robot R2, with the top of the arrow indicating its destination. Symbol T3 schematically represents the movement trajectory of robot R3, with the top of the arrow indicating its destination.

[0061] The simulation time is 120 seconds, with times ranging from t = 0 to 120 [s]. The starting time of robot R1 is fixed at t = 16 [s]. The starting time of robot R2 is selected from times between t = 16 [s] and t = 26 [s]. The starting time of robot R3 is selected from times between t = 7 [s] and t = 37 [s].

[0062] The system requirements are that either robot R1 or robot R2 must reach the destination within 120 seconds, and robot R3 must also reach the destination within 120 seconds. The inventors conducted search-based tests to search for scenarios that violate the system requirements, and performed 500 simulations.

[0063] The start times of robots R2 and R3 correspond to the simulation conditions. By setting the horizontal axis to the start time of robot R2 and the vertical axis to the start time of robot R3, points corresponding to each of the 500 simulation conditions can be plotted. In search-based testing, the distance between adjacent points is generally not fixed. On the other hand, in mesh testing, the distance between adjacent points is generally fixed. Search-based testing is more efficient at searching for defect data compared to mesh testing.

[0064] Figure 6 A summary diagram to illustrate the overview of search-based testing. Figure 6 The system shown includes an SBT (Search-Based Testing) unit 111 and a target model 112. Alternatively, the acquisition unit 110 of the information processing device 100 may also include an SBT unit 111 and a target model 112.

[0065] The SBT unit 111 uses the target model 112 to perform search-based testing. The target model 112 includes a model / simulator 1121 and control software 1122 for controlling the model / simulator.

[0066] In SBT unit 111, the aforementioned system requirements are set, and a test scenario is input to SBT unit 111. The test scenario includes simulation conditions. Target model 112 receives the test scenario and returns an output signal representing the simulation result. The output signal may also include signals representing the time change of the position coordinates of robot R1, signals representing the time change of the position coordinates of robot R2, and signals representing the time change of the position coordinates of robot R3.

[0067] Return to Figure 3 Continuing the explanation, the acquisition unit 110 of the information processing device 100 extracts the (x, y) data of each robot from the simulation results (step S102). Steps S102 to S106 are performed on the signal data of robot R1, robot R2, and robot R3, respectively. The calculation of the similarity ID for robot R1 will be explained below.

[0068] Next, the data extraction unit 120 of the information processing device 100 extracts one abnormal time series data (referred to as failed target data) that has become the target (step S103A). Then, the data extraction unit 120 extracts a group of successful data (e.g., 366 normal time series data) (step S103B).

[0069] Next, the partial time seriesification unit 131 of the information processing device 100 performs partial time seriesification on the failed target data (step S104A) and on each normal time series data contained in the successful data group (step S104B). The partial time seriesification unit 131 generates multiple partial time series by sliding a 10-second time window by 1 second each time.

[0070] Next, the parameter calculation unit 132 of the information processing device 100 calculates characteristic parameters based on partial time series data of the failed target data (step S105A), and calculates characteristic parameters based on partial time series data of each normal time series data (step S105B). These characteristic parameters are used for determining the cause signal, as described later. Specifically, the characteristic parameters are those obtained by extracting (x, y) information per second from each partial time series data and arranging it. For example, if (x, y) data for 11 robots R1 are extracted, the characteristic parameters become 22-dimensional data.

[0071] The parameter calculation unit 132 extracts feature parameters from each part of the time series data, thereby determining the distribution of a failed target point and 366 successful points (also known as the success data distribution). Figure 7 A summary diagram representing the failed target point P1. The failed target point P1 is plotted in 22-dimensional space. Figure 8 A summary diagram representing the distribution of successful data G. Multiple successful points P2 are plotted in 22-dimensional space.

[0072] Return to Figure 3 Continuing the explanation, next, the ID extraction unit 133 of the information processing device 100 extracts successful points from the successful distribution whose Euclidean distance to the failed target points is below a tolerance threshold (e.g., 0.2), and obtains the ID of the corresponding normal time series data (step S106). The ID of normal time series data similar to the failed target data is obtained.

[0073] Reference Figure 9The method for extracting similar IDs is explained in detail below. The left side of the hollow arrow mark represents the successful data group G, and the right side of the hollow arrow mark represents the extracted successful point. Region D represents the area where the distance between the failed target point P1 and the target point P1 is below the allowable error threshold. The ID extraction unit 133 extracts the successful point P2 contained in region D and obtains the corresponding ID.

[0074] Return to Figure 3 To continue the explanation. As mentioned above, the processing in steps S102 to S106 also applies to the position data of robot R2 and robot R3 (step S200).

[0075] Next, the similarity calculation unit 134 of the information processing device 100 calculates the common part of the similar IDs for each signal combination, and calculates the change in the number of similar IDs (similarity) contained in the common part (step S107). The similar IDs are also called common similar IDs. The similarity calculation unit 134 calculates the similar IDs for six types of signal combinations, namely, combinations of position data (signal 1) of robot R1, position data (signal 2) of robot R2, and position data (signal 3) of robot R3. The similarity calculation unit 134 calculates the number of similar IDs at each time point (time interval) and in each combination by calculating the similar IDs of all partial time series data.

[0076] Next, the timing determination unit 135 of the information processing device 100 determines the timing (time point) of the above-mentioned similarity (ID number) in all signal combinations below the threshold S (for example, one) as the task timing (step S108). All signal combinations refer to the combination of position data of robot R1, position data of robot R2, and position data of robot R3.

[0077] Reference Figure 10 The specific methods for periodic testing of the project are explained. Figure 10 This is a graph representing the similarity C123 in all signal combinations 1, 2, and 3. The vertical axis represents the similarity, and the horizontal axis represents time. The timing determination unit 135 detects timings where the similarity C123 is below a threshold S (e.g., 80.0 seconds). It can be assumed that an unexpected action occurred at this timing.

[0078] Return to Figure 3 Continuing the explanation. Next, the signal determination unit 136 of the information processing device 100 determines the cause signal. The signal determination unit 136 first determines whether the number of similar IDs (similarity) of a single signal at the time of the issue is below a threshold S (for example, one) (step S109). A single signal refers to the position data of robot R1, the position data of robot R2, and the position data of robot R3.

[0079] If the number of similar IDs for a single signal is below the threshold (Yes in step S109), the signal determination unit 136 determines the single signal as the cause signal (step S111). If the number of similar IDs for a single signal is greater than the threshold (No in step S109), the signal determination unit 136 proceeds to the processing in step S110.

[0080] In step S110, the signal determination unit 136 determines whether the number of similar IDs (similarity) of each signal combination is below a threshold S. When the total number of signals is N, the signal determination unit 136 varies the number of combinations k between 2 and N and repeatedly performs the determination process. If the determination result is true (yes in step S110), the signal determination unit 136 determines that signal combination as the cause. Alternatively, when k = N, all signal combinations can be determined as the cause.

[0081] Specifically, the signal determination unit 136 first confirms the number of similar IDs (similarity) of the single signals (signal 1, signal 2, signal 3) for the timing of the task. If there is a single signal with a similar ID count below the threshold S, the signal determination unit 136 determines that single signal as the signal that caused the timing of the task.

[0082] If no single signal with a similarity below the threshold is found, the signal determination unit 136 confirms the similarity of combinations of two signals at the problem timing (combination 1.2, combination 1.3, combination 2.3). Then, if there is a signal combination with a similarity below the threshold, the signal determination unit 136 determines that signal combination as the signal causing the problem timing.

[0083] If no signal combination with similarity below the threshold is found, the signal determination unit 136 determines all signal combinations (combinations 1, 2, and 3) as signal combinations that cause the timing of the problem.

[0084] Figures 11(A) and 11(B) are schematic diagrams illustrating the method for determining the causal signal. Figure 11(A) shows the simulation results of the movement of robots R1, R2, and R3. The time interval for the task is set to 80.0 to 90.0 seconds. The thick boxes indicate the locations where unexpected actions occur, which are determined by judging the causal signal.

[0085] Figure 11(B) shows the similarity C1, C2, C3, C12, C13, C23, and C123 in signal combinations 1, 2, 3, 1·2, 1·3, 2·3, and 1·2·3. The vertical axis represents the similarity and the horizontal axis represents the time.

[0086] The signal determination unit 136 determines whether C1, C2, C3, C12, C13, and C23 are below the threshold S at the task timing point. In Figure 11(B), C2 is below the threshold S at the task timing point. In this case, the signal determination unit 136 determines signal 2 as the cause signal. In this case, it is determined that the task timing occurred due to the robot R2.

[0087] Finally, the effects of the information processing device involved in Implementation Method 1 will be explained. There are cases where tests are conducted through simulation or experimentation to verify whether the control system has been created according to the developer's requirements. If the test results contain defective data, the developer needs to confirm the system's operation and determine the necessity of correction, identify the cause of the defect, and correct the system.

[0088] Determining the cause of defects in multidimensional time series data, especially those containing multiple signal information, requires analyzing the phenomena of multiple cascading signals, necessitating domain knowledge of the control system and significant man-hours. Generally, defect cause determination involves identifying and analyzing signals representing actions that differ from those envisioned by the developers to meet system requirements. However, identifying unintended actions requires manual analysis performed by individuals with detailed domain knowledge.

[0089] Therefore, this invention proposes a technique to support the extraction of unintended actions without requiring detailed domain knowledge. When detecting unintended actions, machines struggle to understand the developer's assumptions, i.e., "the action requirements of the implicit part of each signal." Therefore, this proposal assumes that "successful data that does not produce defects satisfies system requirements by fulfilling the action requirements of the implicit part," using successful data obtained through testing, and determining unintended actions for failed target data. Specifically, by determining the timing (task timing) of actions that are not present in the successful data and the signals believed to be the cause, a system is proposed to support the determination of defect causes.

[0090] According to the information processing apparatus of Embodiment 1, the timing of a problem and its cause signal that are considered to have caused an unexpected action can be automatically determined. Therefore, detailed domain knowledge of the developer is not required, and the time spent determining the cause of the defect can be reduced.

[0091] If only the characteristic parameters required for detecting the timing and cause signals of the problem—that is, the signal information that we want to focus on—can be set, then only a few hyperparameters (partial time series data length, similarity threshold) need to be set thereafter. Therefore, in the information processing method described in Implementation 1, no expertise in the control system being studied is required.

[0092] When calculating the similarity of multiple signals, it is common practice to set a similarity threshold T in the feature parameters of each combination (for example, in the case of a three-signal combination, T). A T B T C T A·B T A·C T B·C T A·B·C In contrast, in this invention, a similarity threshold T is set for each signal (e.g., T1). A T B T C Then extract their common ID to calculate the similarity of multiple signals.

[0093] The information processing system may not necessarily have all functional elements integrated into the information processing device 100. For example, the function of the verification unit 130 may be performed by the computing unit of a server connected to the information processing device 100 via a network. In this case, the server sends the verification result to the information processing device 100. Thus, the information processing system can also be configured to include both a server and the information processing device 100. The aforementioned processor and memory may be located in the server, or in both the information processing device 100 and the server.

[0094] In the above examples, the program includes a set of instructions (or software code) that, when read by a computer, causes the computer to perform one or more functions described in the implementation. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example, and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disc storage, cassette tape, magnetic tape, disk storage, or other magnetic storage devices. The program may also be transmitted on a transient computer-readable medium or a communication medium. By way of example, and not limitation, transient computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagation signals.

[0095] Furthermore, the present invention is not limited to the above-described embodiments, and appropriate modifications can be made without departing from the spirit of the invention.

[0096] It will be apparent from this disclosure that embodiments of the present disclosure can be modified in a variety of ways. Such modifications should not be considered as departing from the spirit and scope of the present disclosure, and all such modifications should be included within the scope of the invention by those skilled in the art.

Claims

1. An information processing system comprising: an acquisition unit that acquires a plurality of time series data, each time series data including a plurality of signal data, the plurality of time series data including abnormal time series data and a plurality of normal time series data, the plurality of time series data being position data of a plurality of objects whose relative positions are changed with respect to time; a data extraction unit that extracts the abnormal time series data and the normal time series data from the plurality of time series data; a verification unit that determines, for each signal data, whether the abnormal time series data and each normal time series data in each time interval are similar to each other, and verifies an abnormal event based on a result of the determination, the acquisition unit performs simulation of movement of the plurality of objects, and sets a departure place and a departure time of each object as a simulation condition, the verification unit partially time-series each signal data in a manner that a time window of a predetermined time interval is shifted by one second at a time, and extracts position coordinates of the object at each of a plurality of time points, the verification unit calculates, for each signal data, a feature point of each time series data in each time interval based on the position coordinates, and determines whether the feature point of each normal time series data is included in a predetermined range of the feature point of the abnormal time series data, the abnormal time series data is data that violates a system requirement including a first requirement that at least one object reaches a destination within a predetermined time and a second requirement that an error of position estimation of each of the objects is below a threshold value, the object is an autonomous mobile body.

2. The information processing system according to claim 1, wherein the abnormal event is generated due to interaction of the plurality of objects.

3. The information processing system according to claim 1 or 2, wherein the verification unit extracts, for each signal data, an ID of a normal time series data similar to the abnormal time series data in each time interval, and for each signal combination, a common part of the IDs is found, and for each signal combination, a time change in a number of IDs included in the common part is calculated, and based on a result of the calculation, at least any one of a single signal and a signal combination associated with a cause of the abnormal event is determined.

4. The information processing system according to claim 3, wherein the verification unit determines, based on the time change in the number of IDs included in the common part of all signal combinations, a timing associated with the abnormal occurrence, and based further on the timing, at least any one of the single signal and the signal combination associated with the cause of the abnormal event is determined.

5. An information processing method comprising: A computer acquires a plurality of time-series data in an acquisition step, each of the time-series data including a plurality of signal data, the plurality of time-series data including abnormal time-series data and a plurality of normal time-series data, the plurality of time-series data being position data of a plurality of objects whose relative positions change with respect to time; an extraction step of extracting the abnormal time-series data and the normal time-series data from the plurality of time-series data; a verification step of verifying an abnormal event based on a result of determining whether the abnormal time-series data and each of the normal time-series data in each time interval are similar to each other for each signal data, in the acquisition step, a simulation of movement of the plurality of objects is performed, and a departure place and a departure time of each object are set as a simulation condition, in the verification step, each signal data is partially time-series for each time interval in a manner that a time window of a predetermined time interval is shifted by one second each time, and a position coordinate of the object at each of a plurality of time points is extracted, and a feature point of each time-series data in each time interval is calculated for each signal data based on the position coordinate, and it is determined whether the feature point of each normal time-series data is included in a predetermined range of the feature point of the abnormal time-series data, the abnormal time-series data is data that violates a system requirement including a first requirement that at least one object reaches a destination within a predetermined time and a second requirement that an error of a position estimation of each object is below a threshold value, the object is an autonomous mobile body.

6. A computer-readable storage medium storing a program that causes a computer to execute an information processing method including: an acquisition step of acquiring a plurality of time-series data, each of the time-series data including a plurality of signal data, the plurality of time-series data including abnormal time-series data and a plurality of normal time-series data, the plurality of time-series data being position data of a plurality of objects whose relative positions change with respect to time; an extraction step of extracting the abnormal time-series data and the normal time-series data from the plurality of time-series data; a verification step of verifying an abnormal event based on a result of determining whether the abnormal time-series data and each of the normal time-series data in each time interval are similar to each other for each signal data, in the acquisition step, a simulation of movement of the plurality of objects is performed, and a departure place and a departure time of each object are set as a simulation condition, in the verification step, each signal data is partially time-series for each time interval in a manner that a time window of a predetermined time interval is shifted by one second each time, and a position coordinate of the object at each of a plurality of time points is extracted, and a feature point of each time-series data in each time interval is calculated for each signal data based on the position coordinate, and it is determined whether the feature point of each normal time-series data is included in a predetermined range of the feature point of the abnormal time-series data, the abnormal time-series data is data that violates a system requirement including a first requirement that at least one object reaches a destination within a predetermined time and a second requirement that an error of a position estimation of each object is below a threshold value, the object is an autonomous mobile body. In the verification process, each signal data is partially time-sequenced in a manner that a time window of a predetermined time interval is shifted by 1 second each time, and the position coordinates of the object at each of a plurality of time points are extracted, and, by the position coordinates, feature points of each time-sequenced data in each time interval are calculated for each signal data, and it is determined whether the feature points of each normal time-sequenced data are included in a predetermined range of the feature points of the abnormal time-sequenced data, The abnormal time-sequenced data is data that violates system requirements including a first requirement that at least one object reaches a destination within a predetermined time and a second requirement that an error of position estimation of each of the objects is below a threshold value, The object is an autonomous mobile body.

Images