Method and Apparatus for Evaluating Non-Fusion Reliability by Using Multi-Sensor-Based Object Relationship Analysis

Abstract

A non-converging reliability evaluation method and apparatus using multiple sensor-based object relationship analysis are disclosed. The present embodiment provides a non-fusion type reliability evaluation method and apparatus using multi-sensor-based object relationship analysis, which calculates reliability based on the semantic consistency of the system by analyzing the correspondence between object recognition results generated independently from each sensor and the relationships between objects without directly fusing sensor data in a multi-sensor-based autonomous system, detects abnormal situations using the calculated reliability and the amount of change thereof, and enables appropriate control according to the result.

Description

Technology Field

[0001] One embodiment of the present invention relates to an autonomous system using a plurality of sensors, and in particular, to a non-fusion type reliability evaluation method and apparatus using a plurality of sensor-based object relationship analysis that evaluates the reliability of the system by analyzing the relationships between objects independently generated from each sensor without directly fusing sensor data, and performs control based thereon.

[0002] The present invention is not limited to specific industries and can be applied to all autonomous systems requiring sensor-based perception and decision-making (autonomous vehicles, mobile robots, unmanned aerial vehicles / drones, autonomous ships, industrial automation systems, etc.). Background Technology

[0003] The following description merely provides background information related to the present embodiment and does not constitute prior art.

[0004] Autonomous systems, such as autonomous vehicles and robot systems, perceive their surrounding environment using multiple sensors, including cameras, LiDAR, and radar. In other words, it is common for these multi-sensor-based systems to improve recognition accuracy by generating a single representation through sensor fusion.

[0005] However, existing technology focuses on integrating sensor data to generate a single recognition result and does not include a structure to verify the mutual consistency of results between sensors. This can lead to the following problems.

[0006] First, even if sensor inputs are physically manipulated or distorted, it is difficult to detect this through software integrity verification alone.

[0007] Second, there is a lack of a structure to determine and respond to cases where recognition results between different sensors are inconsistent.

[0008] Third, even if integrity verification is performed at the time of installation, semantic accuracy at the time of execution cannot be guaranteed.

[0009] Fourth, in sensor fusion structures, errors from specific sensors or specific cognitive results can be propagated throughout the entire final fusion result, and there are structural limitations that make it difficult to interpret which sensor or relationship information formed the final judgment result.

[0010] Therefore, a new technical approach is required to evaluate the semantic consistency and reliability of the system at the execution stage by utilizing object information independently generated from each sensor, as well as the correspondence relationships between those objects and the relationships between objects, without directly fusing sensor data. Prior art literature

[65535] Korean Patent Publication No. 10-2025-0173430 (20251210) The problem to be solved

[0011] The present embodiment aims to provide a non-fusion type reliability evaluation method and apparatus using multi-sensor-based object relationship analysis, which calculates reliability based on the semantic consistency of the system by analyzing the correspondence relationships between object recognition results generated independently from each sensor and the relationships between objects, without directly fusing sensor data in a multi-sensor-based autonomous system, detects abnormal situations using the calculated reliability and the amount of change thereof, and enables appropriate control according to the results. means of solving the problem

[0012] According to one aspect of the present embodiment, a sensor unit comprising a plurality of sensors; a recognition unit that generates a set of objects for each sensor based on sensor data acquired from the sensor unit; an object matching unit that performs object matching between the set of objects for each sensor to generate an object matching result; a relationship configuration unit that generates relationship information, which is structured data including relationships between objects, based on the object matching result; and a plurality of consistency indicators R based on the relationship information. i A consistency analysis unit that calculates (t); the plurality of consistency indicators R i Weights (α) corresponding to each (t) i A non-converging reliability evaluation device is provided, characterized by comprising: a reliability calculation unit that calculates reliability T(t) in a non-compensatory structure by reflecting ); an anomaly detection unit that generates an anomaly state determination result by determining an anomaly state using the reliability T(t) and a change amount Δ T(t); and a control unit that performs control to change system operation according to the anomaly state determination result.

[0013] According to another aspect of the present embodiment, a sensor unit comprising a plurality of sensors acquires a plurality of sensor data; a recognition unit generates a set of objects for each sensor based on the sensor data acquired from the sensor unit; an object matching unit performs object matching between the sets of objects for each sensor to generate an object matching result; a relationship building unit generates relationship information, which is structured data including relationships between objects, based on the object matching result; and a consistency analysis unit generates a plurality of consistency indicators R based on the relationship information. i A process for calculating (t); a reliability calculation unit, the plurality of consistency indicators R i Weights (α) corresponding to each (t) iA non-converging reliability evaluation method is provided, characterized by comprising: a process of calculating reliability T(t) in a non-compensatory structure by reflecting ); a process in which an anomaly detection unit generates an anomaly state determination result by determining an anomaly state using the reliability T(t) and a change amount Δ T(t); and a process in which a control unit performs control to change the system operation according to the anomaly state determination result. Effects of the invention

[0014] As described above, according to the present embodiment, in a multi-sensor-based autonomous system, without directly fusing sensor data, reliability based on the semantic consistency of the system is calculated by analyzing the correspondence between object recognition results generated independently from each sensor and the relationships between objects, and abnormal situations are detected using the calculated reliability and the amount of change thereof, and appropriate control is performed according to the result. Brief explanation of the drawing

[0015] FIG. 1 is a block diagram schematically showing a non-fusion type reliability evaluation device according to the present embodiment. FIG. 2 is a diagram illustrating the object matching and relationship configuration process according to the present embodiment. FIG. 3 is a diagram illustrating a reliability calculation structure using a plurality of consistency indicators according to the present embodiment. FIG. 4 is a diagram illustrating the flow of anomaly detection and state determination based on the amount of change in reliability according to the present embodiment. FIG. 5 is a diagram illustrating the operation structure of a sensor module according to the present embodiment. FIG. 6 is a diagram illustrating the structure for generating object sets per sensor by a cognitive system according to the present embodiment. FIG. 7 is a diagram illustrating the structure for forming object correspondence relationships between sensors by an object matching module according to the present embodiment. FIG. 8 is a diagram illustrating the structure for generating relationship information between objects by a relationship configuration module according to the present embodiment. FIG. 9 is a diagram illustrating a structure for calculating a plurality of consistency indicators by a consistency analysis module according to the present embodiment. FIG. 10 is a diagram illustrating the structure for calculating overall reliability by the reliability calculation module according to the present embodiment. FIG. 11 is a diagram illustrating a state determination structure based on the amount of change in reliability by an anomaly detection module according to the present embodiment. FIG. 12 is a diagram illustrating various industrial application examples of a non-converged reliability evaluation device according to the present embodiment. FIG. 13 is a diagram illustrating a persistence-based event determination flow according to the present embodiment. FIG. 14 is a drawing illustrating an optional evidence preservation structure including a pre-event section, an event section, and a post-event section according to the present embodiment. FIG. 15 is a diagram illustrating an anomaly type classification and severity-based response structure according to the present embodiment. FIG. 16 is a diagram illustrating the structure of an accountability packet according to the present embodiment. FIG. 17 is a diagram illustrating a response policy map by severity level according to the present embodiment. FIG. 18 is a diagram illustrating the overall structure of a non-converging reliability assessment, event-based evidence preservation, and risk-adaptive response system according to a second embodiment. FIG. 19 is a diagram illustrating a reliability calculation structure based on multiple indicators according to a second embodiment. FIG. 20 is a diagram illustrating an abnormal candidate detection and state determination structure according to a second embodiment. FIG. 21 is a diagram illustrating a plurality of consistency indicator calculation structures according to a second embodiment. FIG. 22 is a diagram illustrating a reliability calculation and feedback structure according to a second embodiment. FIG. 23 is a diagram illustrating an anomaly candidate detection structure based on reliability change amount according to a second embodiment. FIG. 24 is a drawing illustrating an industrial application and external linkage structure according to a second embodiment. Specific details for implementing the invention

[0016] Hereinafter, the present embodiment will be described in detail with reference to the attached drawings.

[0017] The names and reference numerals of each component described below are used to correspond with the components illustrated in the attached drawings, and the same reference numerals may be assigned to components performing the same or corresponding functions. Furthermore, the reference numerals of the detailed components illustrated in the drawings are exemplary for convenience of explanation and may be partially modified or subdivided within the scope of the technical concept of the present embodiment.

[0018] The non-fusion structure according to the present embodiment may refer to a structure in which data acquired from each sensor at the perception level is not directly integrated into a single object representation, a single feature representation, or a single recognition result, and subsequent object matching and relationship analysis are performed while the object sets for each sensor maintain their independence. Object relationships may include relationship information such as distance, location, and attributes between objects, and the non-compensation structure may refer to a structure in which the overall reliability is reflected to decrease together when a single indicator value decreases. The relationship information may be structured data including distance, location, attributes, and semantic relationships between objects. An object set refers to a set of objects detected by sensor k at time t, an object correspondence set refers to a set representing the correspondence relationship between objects detected by different sensors k and l, and a relationship set refers to a set that integrates object correspondence sets or relationship information formed for each of multiple sensor pairs. The consistency index Ri(t) is a value that quantifies the semantic consistency or degree of agreement of object relationships by sensor at time t, and may include at least one of object correspondence, spatial consistency, reliability consistency, and temporal consistency, and reliability T(t) may represent an overall state evaluation value of the system calculated based on the plurality of consistency indices. Accordingly, the present embodiment may include auxiliary processing such as referencing auxiliary information between sensors, time synchronization, or coordinate system alignment, but can be distinguished from conventional sensor fusion structures in that the object sets by sensor are not directly fused into a single representation at the perception stage.

[0019] The non-fusion structure according to the present embodiment refers to a method of evaluating reliability based on the relationships between object recognition results independently generated from each sensor, without directly combining raw data or feature data acquired from different sensors. Object relationships include relationship information such as distance, location, and attributes between objects. The non-compensation structure refers to a structure where overall reliability decreases if a single indicator is low. Relationship information refers to structured data including distance, location, attributes, and semantic relationships between objects. Object set M k (t) refers to the set of objects detected by sensor k at time t. Object correspondence set M( k,l )(t) refers to a set representing the correspondence relationships between objects detected by different sensors k and l. Relationship set M pair (t) refers to a set integrating object correspondence sets or relationship information formed for each of multiple sensor pairs. Consistency index R i (t) is a value that quantifies the semantic consistency or degree of agreement of object relationships by sensor at time t, and may include at least one of object correspondence, spatial consistency, reliability consistency, and temporal consistency. Reliability T(t) is a plurality of consistency indices R i It refers to the comprehensive state evaluation value of the system calculated based on (t).

[0020] FIG. 1 is a block diagram schematically showing a non-fusion type reliability evaluation device according to the present embodiment.

[0021] The non-converging reliability evaluation device (100) is configured so that multiple modules are processed in parallel or through pipeline processing.

[0022] Unlike a sensor fusion method that directly combines sensor data collected from a sensor unit (110) to generate a single expression, the non-fusion type reliability evaluation device (100) according to the present embodiment has a non-fusion structure that evaluates system reliability using correspondence relationships between object recognition results formed independently for each sensor and relationships between objects.

[0023] The non-converging reliability evaluation device (100) according to the present embodiment may be implemented in an electronic control unit (ECU) for a vehicle, a GPU / NPU-based high-performance computing module, or an edge AI platform. The non-converging reliability evaluation device (100) according to the present embodiment may process sensor data processing, object recognition, object matching, consistency analysis, and reliability calculation in parallel by separating them into different processing modules.

[0024] Each module included in the non-converged reliability evaluation device (100) is connected via a message-based interface and can transmit data using a Robot Operating System (ROS), a Data Distribution Service (DDS)-based distributed system, or similar middleware. The non-converged reliability evaluation device (100) can perform real-time processing through hardware acceleration using a CPU, GPU, or NPU.

[0025] Each module included in the non-converged reliability evaluation device (100) may be implemented with one or more processors, memory and program instructions, and may include a hardware acceleration device. The non-converged reliability evaluation device (100) may support real-time processing in the range of about 50 to 100 ms, but this is merely one embodiment and may vary depending on the system configuration, number of sensors, computational performance, or application field.

[0026] The non-converged reliability evaluation device (100) is designed to operate in a real-time autonomous system environment, and the entire processing pipeline is executed within a preset time limit. The total processing delay of the non-converged reliability evaluation device (100) can be implemented in the range of approximately 50ms to 100ms, but this is merely illustrative and may vary depending on the system configuration, number of sensors, computational resources, and application environment. The non-converged reliability evaluation device (100) may apply a method of selectively performing analysis only on objects with high reliability or regions of interest.

[0027] The non-converging reliability evaluation device (100) can be configured so that the execution of each module overlaps using parallel processing and pipeline processing techniques.

[0028] The non-converged reliability evaluation device (100) allows the system to operate adaptively by reflecting the control result or state judgment result in the reliability calculation, threshold setting, or weight adjustment at the next point in time. The non-converged reliability evaluation device (100) can perform feedback in the same cycle or a subsequent cycle, and can dynamically adjust the threshold, weight, or priority of the analysis target according to the system state.

[0029] The non-fusion type reliability evaluation device (100) has a non-fusion structure that evaluates reliability based on the relationship between object recognition results generated independently from each sensor, without directly combining sensor data. The non-fusion type reliability evaluation device (100) is applied to autonomous vehicles, mobile robots, drones, autonomous ships, industrial automation systems, or smart infrastructure systems.

[0030] The non-converged reliability evaluation device (100) according to the present embodiment includes a sensor unit (110), a perception unit (120), an object matching unit (130), a relationship configuration unit (140), a consistency analysis unit (150), a reliability calculation unit (160), an anomaly detection unit (170), a control unit (180), a reliability feedback unit (190), and an external interface (192). The components included in the non-converged reliability evaluation device (100) are not necessarily limited thereto.

[0031] Each component included in the non-converged reliability evaluation device (100) can be connected to a communication path connecting a software module or a hardware module inside the device and operate organically with one another. These components communicate using one or more communication buses or signal lines.

[0032] Each component of the non-converging reliability evaluation device (100) illustrated in FIG. 1 represents a unit that processes at least one function or operation, and can be implemented as a software module, a hardware module, or a combination of software and hardware.

[0033] The sensor unit (110) includes a plurality of sensors. The sensor unit (110) includes at least one of a camera, lidar, radar, ultrasonic sensor, and infrared sensor.

[0034] The sensor unit (110) includes multiple sensors such as a camera, LiDAR, and radar. Each sensor data included in the sensor unit (110) is processed independently without being fused with one another. The sensor unit (110) includes a camera, LiDAR, radar, or other environment recognition sensors and acquires data about the external environment.

[0035] The recognition unit (120) generates a set of objects for each sensor based on data obtained from the sensor unit (110). The recognition unit (120) recognizes objects from each of the sensor units (110) and generates a set of objects for each sensor.

[0036] The recognition unit (120) generates a set of objects from each of the sensor units (110) and provides it to the object matching unit (130). The recognition unit (120) detects objects and generates a set of objects based on data input from the sensor units (110). The recognition unit (120) has a non-fusion structure that does not directly fuse sensor data.

[0037] The recognition unit (120) generates a set of objects for each sensor based on sensor data obtained from the sensor unit (110). The recognition unit (120) has a non-fusion structure that does not directly fuse the sensor data.

[0038] The object matching unit (130) forms a correspondence relationship between sets of objects generated from different sensors. The object matching unit (130) forms a correspondence relationship by performing object matching between sets of objects generated from different sensors.

[0039] The object matching unit (130) matches objects between different sensors. The object matching unit (130) creates a correspondence relationship between sets of objects generated from different sensors and performs matching between sensors for the same object.

[0040] The object matching unit (130) identifies unmatched objects (224). The object matching unit (130) or the relationship configuration unit (140) performs coordinate transformation using external parameters to correct the coordinate system difference between sensors.

[0041] The object matching unit (130) uses the unmatched object (224) as separate mismatch information for reliability calculation.

[0042] The object matching unit (130) performs object matching between object sets for each sensor to generate an object matching result. The object matching unit (130) performs object matching between object sets for each sensor to identify unmatched objects (224).

[0043] The relationship configuration unit (140) generates relationship information (228), which is structured data including relationships between objects based on correspondence relationships. The relationship configuration unit (140) configures relationship information including distance, relative position, attributes, or semantic relationships between objects based on object matching results.

[0044] The relationship configuration unit (140) generates relationship information (228) based on correspondence relationships. The relationship configuration unit (140) generates relationship information between objects based on object matching results. The relationship configuration unit (140) can generate relationships including distance, relative position, attributes, or semantic relationships between objects.

[0045] The relationship configuration unit (140) generates relationship information (228) including distance, location, and semantic relationship between objects.

[0046] The relationship configuration unit (140) generates relationship information, which is structured data including relationships between objects, based on the object matching results. The relationship configuration unit (140) generates the relationship information including distance, relative position, attributes, or semantic relationships between objects based on the object matching results.

[0047] The consistency analysis unit (150) has multiple consistency indicators R based on relationship information. i (t) is calculated. The consistency analysis unit (150) calculates multiple consistency indicators R based on relationship information. i Calculate (t).

[0048] The consistency analysis unit (150) calculates multiple consistency indicators based on relationship information (228). The consistency analysis unit (150) uses the relationship information (228) to calculate multiple consistency indicators R i (t) is calculated. The consistency analysis unit (150) calculates a plurality of consistency indicators R including the degree of agreement in the number of objects, location difference, distance error, or whether the class matches. i (t) can be generated.

[0049] The consistency analysis unit (150) generates a consistency indicator including at least one of object correspondence, spatial consistency, reliability consistency, and temporal consistency.

[0050] The consistency analysis unit (150) has multiple consistency indicators R based on relationship information. i (t) is calculated. The consistency analysis unit (150) calculates a consistency index R including at least one of object correspondence, spatial consistency, reliability consistency, and temporal consistency based on relationship information. i Calculate (t).

[0051] The consistency analysis unit (150) has a plurality of consistency indicators R i To calculate (t), a stepwise processing pipeline of relation information, feature extraction, quantification, and normalization is performed.

[0052] The consistency analysis unit (150) has a relationship information-based analysis structure.

[0053] The consistency analysis unit (150) receives relationship information including object matching results (matched object pairs, unmatched objects), relationship information (distance, relative position, attribute, semantic relationship, etc.), and time series information (previous time point relationship). That is, the consistency analysis unit (150) uses a relationship graph between objects as input, rather than sensor raw data.

[0054] The consistency analysis unit (150) analyzes the consistency indicator R based on relationship information. i Calculate (t).

[0055] The consistency analysis unit (150) extracts relationship features from relationship information. The consistency analysis unit (150) separates indicator-specific features from the relationship information. For example, the consistency analysis unit (150) separates the matching status from the relationship information as a correspondence feature. The consistency analysis unit (150) separates the distance / coordinate difference from the relationship information as a spatial feature. The consistency analysis unit (150) separates the confidence value from the relationship information as a reliability feature. The consistency analysis unit (150) separates the time change from the relationship information as a dynamic feature.

[0056] The consistency analysis unit (150) defines a quantification function for each indicator based on relationship information. The consistency analysis unit (150) ensures that each indicator has a different evaluation function.

[0057] The consistency analysis unit (150) calculates object correspondence based on relationship information. The consistency analysis unit (150) calculates object correspondence (R1(t)), which is the ratio of matching identical objects between sensors, using [Equation 1].

[0058]

[0059] The consistency analysis unit (150) reflects the number of unmatched objects as a penalty. The consistency analysis unit (150) calculates a higher value as the objects match each other better.

[0060] The consistency analysis unit (150) calculates spatial alignment based on relationship information. The consistency analysis unit (150) calculates spatial alignment based on distance / position error between matched objects using [Equation 2] or [Equation 3].

[0061]

[0062]

[0063] The consistency analysis unit (150) determines that the more similar the position / distance relationship is, the higher the value.

[0064] The consistency analysis unit (150) calculates confidence coherence based on relationship information. The consistency analysis unit (150) calculates confidence coherence based on the difference in confidence values ​​per sensor using [Equation 4] or [Equation 5].

[0065]

[0066]

[0067] The consistency analysis unit (150) determines that the reliability of judgment between sensors is higher the more similar they are.

[0068] The consistency analysis unit (150) calculates temporal consistency based on relationship information. The consistency analysis unit (150) calculates temporal consistency, which is the stability of relationship change relative to a previous point in time, using [Equation 6] or smoothing.

[0069]

[0070] The consistency analysis unit (150) determines that the less rapid the change over time, the higher the value.

[0071] The consistency analysis unit (150) performs normalization as a common process based on relationship information. The consistency analysis unit (150) transforms all indicators to satisfy [Equation 7].

[0072]

[0073] The consistency analysis unit (150) enables the combination of different scale indices using min-max normalization, sigmoid, and exponential decay.

[0074] The consistency analysis unit (150) reflects non-matching objects based on relationship information. Based on relationship information, the consistency analysis unit (150) does not simply remove non-matching objects, but reflects them as a factor reducing the responsiveness indicator and a separate inconsistency penalty. The consistency analysis unit (150) reflects non-matching objects to ensure differentiation of the non-fusion structure.

[0075] The consistency analysis unit (150) has a multiple indicator configuration structure based on relationship information. The consistency analysis unit (150) finally generates a multidimensional consistency vector using [Equation 8].

[0076]

[0077] The consistency analysis unit (150) calculates each indicator independently, reflects different semantic perspectives, and uses them as inputs for subsequent reliability calculations.

[0078] The consistency analysis unit (150) applies a plurality of evaluation functions that quantify object correspondence, spatial consistency, reliability consistency, and temporal consistency, respectively, based on object matching results and relationship information, and normalizes each evaluation result to obtain a plurality of consistency indicators R i Calculate as (t).

[0079] The consistency analysis unit (150) applies a plurality of evaluation functions to quantify correspondence, spatial consistency, reliability consistency, and temporal consistency, respectively, from object matching results and relationship information between objects, and calculates a plurality of consistency indicators Ri(t) by normalizing each result.

[0080] The reliability calculation unit (160) calculates the reliability T(t) using information on object sets per sensor, object correspondence relationships, and relationships between objects, without directly fusing sensor data. The reliability calculation unit (160) calculates a plurality of consistency indicators R i The reliability T(t) is calculated using a non-compensatory structure by reflecting the weights corresponding to (t).

[0081] The reliability calculation unit (160) has a plurality of consistency indicators R i Reliability T(t) is calculated in a non-compensatory structure by reflecting weights corresponding to (t). The reliability calculation unit (160) has a plurality of consistency indicators R i Reliability T(t) is calculated using (t). The reliability calculation unit (160) has a plurality of consistency indicators R i The reliability value T(t) of the entire system is calculated based on (t). The reliability calculation unit (160) can calculate the reliability value T(t) using a non-compensatory combination of multiple indicators.

[0082] The reliability calculation unit (160) applies a weight αi corresponding to each consistency indicator.

[0083] The reliability calculation unit (160) has a non-compensatory structure in which, when performing reliability calculation, the overall reliability decreases if one indicator value is low.

[0084] The reliability calculation unit (160) comprises a plurality of consistency indicators Ri(t) and a weight α corresponding to each consistency indicator. i Calculate the reliability T(t) using [Equation 15], and calculate the reliability T(t) according to [Equation 15].

[0085] The reliability calculation unit (160) performs object matching and relationship analysis for each of the plurality of sensor pairs, and calculates the reliability T(t) using relationship information or consistency indicators calculated from the plurality of sensor pairs.

[0086] The reliability calculation unit (160) has a plurality of consistency indicators R i Weights (α) corresponding to each (t) i Reliability T(t) is calculated in a non-compensatory structure by reflecting ). The reliability calculation unit (160) calculates reliability T(t) using object sets per sensor, object correspondence relationships, and relationship information between objects without directly fusing sensor data.

[0087] The reliability calculation unit (160) has a plurality of consistency indicators Ri (t) is combined in a non-compensatory manner rather than through simple summation, so that the decline in a specific indicator is directly reflected in the overall reliability.

[0088] The reliability calculation unit (160) uses a set of consistency indicators such as [Equation 9] and weights corresponding to each indicator such as [Equation 10] as inputs.

[0089]

[0090]

[0091] The reliability calculation unit (160) checks conditions such as [Equation 11].

[0092]

[0093] The reliability calculation unit (160) provides a non-compensatory combination structure.

[0094] The reliability calculation unit (160) uses a weighted product-based combination (geometric mean structure) rather than a weighted sum.

[0095] The reliability calculation unit (160) uses [Equation 15] or [Equation 12].

[0096]

[0097] The reliability calculation unit (160) uses T=∑αiRi in the case of the weighted sum method, so that even if some indicators are low, it is not a compensatory structure where other indicators are high, but rather T(t)=∏Ri(t). αi It has a non-compensatory structure that causes the total to decrease if even one is lower using. The reliability calculation unit (160) makes T(t)→0 if Rk(t)→0.

[0098] The reliability calculation unit (160) sensitively detects abnormal situations. The reliability calculation unit (160) ensures that specific sensor mismatches, object mismatches, etc., are immediately reflected as a decrease in overall reliability.

[0099] The reliability calculation unit (160) preserves semantic consistency so that the reliability is high only when all conditions are satisfied simultaneously. The reliability calculation unit (160) has an AND condition-based combination structure.

[0100] The reliability calculation unit (160) reflects the importance of each indicator.

[0101] For example, the reliability calculation unit (160) reflects a high weight for object correspondence and a secondary weight for temporal consistency.

[0102] The reliability calculation unit (160) applies R in the application method i (t) αi The reliability calculation unit (160) determines that if the weight αi increases, the influence of the corresponding indicator increases, and if the weight αi decreases, the influence decreases.

[0103] The reliability calculation unit (160) performs clipping as in [Equation 13] to prevent collapse to 0.

[0104]

[0105] The reliability calculation unit (160) calculates the log domain (logT(t)) to prevent underflow.

[0106] The reliability calculation unit (160) performs temporal smoothing using [Equation 14] to mitigate noise.

[0107]

[0108] The reliability calculation unit (160) selectively reflects only indicators below a threshold value or excludes some indicators.

[0109] The reliability calculation unit (160) receives consistency indicators, applies weights to each indicator, and then performs a weighted product (geometric mean) to calculate the final reliability T(t) after stabilization processing (clipping, log, etc.) is completed.

[0110] The reliability calculation unit (160) implements a non-compensatory combination structure by performing a weighted product operation that reflects weights αi corresponding to each of the multiple consistency indicators Ri(t) in exponential form, and calculates the total reliability T(t) so that the deterioration of a specific consistency indicator is not compensated by other indicators.

[0111] The anomaly detection unit (170) determines an abnormal state using reliability T(t) and / or the amount of change therein ΔT(t). The anomaly detection unit (170) determines an abnormal state based on reliability T(t) and the amount of change therein ΔT(t).

[0112] The anomaly detection unit (170) determines the system state using reliability T(t) and / or change amount ΔT(t). The anomaly detection unit (170) determines the abnormal state of the system using reliability value T(t) and change amount ΔT(t) over time.

[0113] The anomaly detection unit (170) calculates the reliability change amount ΔT(t) using [Equation 16]. The anomaly detection unit (170) classifies the anomaly state into normal, warning, and emergency based on a threshold value.

[0114] The anomaly detection unit (170) generates an anomaly state determination result by determining an anomaly state using reliability T(t) and a change amount ΔT(t). The anomaly detection unit (170) calculates the reliability change amount ΔT(t) based on the difference between the reliability T(t) at the current time point and the reliability T(t-1) at the previous time point, and generates an anomaly state determination result by determining an anomaly state using reliability T(t) and the reliability change amount ΔT(t).

[0115] The anomaly detection unit (170) may selectively preserve information regarding the event-previous period, event period, and event-post-period corresponding to the generated event. The anomaly detection unit (170) may classify anomaly types and calculate severity based on the preserved event-related information. In one embodiment, the anomaly type may include at least one of sensor-dominant inconsistency, multiple indicator simultaneous degradation inconsistency, or structural semantic inconsistency, and the severity may be calculated by combining element scores or judgment results corresponding to the magnitude, persistence, rapid change, and simultaneous degradation of multiple indicators, respectively, of the reliability degradation. For example, a threshold comparison result or a score per section may be assigned to each of the elements, and a severity level of Level 0 to Level 3 may be determined by combining them in a summation, weighted sum, or rule-based manner, and the threshold, section, or weight may be adjusted according to the system status, operational history, or reliability feedback results.

[0116] The control unit (180) performs control to change the system operation according to the result of the state determination. The control unit (180) controls the system according to the result of the abnormal state determination. The control unit (180) performs deceleration, path change, warning output, or safety mode switching according to the abnormal state.

[0117] The control unit (180) can perform deceleration, path change, warning output, or switching to a safe mode based on the result of determining an abnormal state. The control unit (180) controls the system operation based on the result of determining an abnormal state and can perform control such as deceleration, path change, warning output, or switching to a safe mode.

[0118] The control unit (180) processes sensor data that is continuously input within a preset time limit.

[0119] The control unit (180) performs control to change the system operation according to the result of determining an abnormal state. The control unit (180) controls one or more operations among deceleration, path change, warning output, or safe mode switching according to the result of determining an abnormal state.

[0120] The reliability feedback unit (190) can receive a reliability value, a state judgment result, or a control result calculated from the reliability calculation unit (160) or the anomaly detection unit (170) and reflect it in the judgment criteria, weight, or threshold adjustment for the next time point.

[0121] The reliability feedback unit (190) can reflect the above state judgment result in the threshold or weight adjustment at the next time point.

[0122] The reliability feedback unit (190) may include a closed-loop structure that utilizes control results and system status as inputs. The reliability feedback unit (190) can adaptively respond to environmental changes and sensor status changes by utilizing the closed-loop structure. The reliability feedback unit (190) may reflect previous reliability values, control results, or status judgment results in the judgment criteria, threshold values, or weight adjustments for the next point in time.

[0123] The reliability feedback unit (190) can adjust the threshold or weight using the state judgment result, the control result, or the reliability value from the previous time point.

[0124] The external interface (192) outputs reliability values, status information, or control results to an external controller, control system, user terminal, or higher-level autonomous system, or receives setting values, threshold values, policy information, or update information from an external device.

[0125] The external interface (192) can transmit and receive results to and from an external controller or control system. The external interface (192) can transmit and receive information to and from an external controller or control system.

[0126] The event determination module evaluates the persistence of the above-mentioned anomaly candidate and can generate an event only if it is not a temporary change.

[0127] The evidence preservation module may selectively preserve information regarding the pre-event period, event period, and post-event period corresponding to the generated event. The event evaluation module may classify anomaly types and calculate severity based on the preserved event-related information. In one embodiment, the anomaly type may include at least one of a sensor-dominant inconsistency, a multiple indicator simultaneous degradation inconsistency, or a structural semantic inconsistency, and the severity may be calculated by combining element scores or judgment results corresponding to the magnitude, persistence, rapid change, and simultaneous degradation of multiple indicators, respectively, of the reliability degradation. For example, a threshold comparison result or a score per section may be assigned to each of the elements, and a severity level of Level 0 to Level 3 may be determined by combining them in a summation, weighted sum, or rule-based manner, and the threshold, section, or weight may be adjusted according to the system status, operational history, or reliability feedback results.

[0128] The event log module can generate a structured event log corresponding to the event. In one embodiment, the structured event log may be an accountability packet and may include event identification information, time information, reliability information, consistency indicator information, persistence information, anomaly type information, severity information, response history information, and evidence reference information.

[0129] The response control module may perform at least one of warning output, log retention, evidence locking, mitigation support signal generation, deceleration, route change, or safe mode switching depending on the severity. The reliability feedback module may adjust the threshold or weight for the next time point using the response result, status judgment result, or reliability information from the previous time point.

[0130] The external interface can transmit and receive the above event records, status determination results, or control results with an external controller, control system, audit system, post-analysis system, or user terminal. Accordingly, the present embodiment can provide an integrated operational structure that goes beyond non-convergence-based reliability assessment and control to include event-driven evidence preservation, severity-based response, structured record generation, and external linkage.

[0131] FIG. 2 is a diagram illustrating the object matching and relationship configuration process according to the present embodiment.

[0132] Referring to FIG. 2, the object matching unit (130) can form a correspondence between sets of objects (212, 214) generated from different sensors and identify matching results (222) and unmatched objects (224).

[0133] The object matching unit (130) can perform matching using at least one of a distance-based method, an IoU-based method, a feature vector similarity-based method, or a multi-frame tracking-based method. The relationship configuration unit (140) can generate relationship information (228) including distance, location, speed, direction, or class relationship between objects based on the matching results.

[0134] Referring to FIG. 2, object set A (212) and object set B (214) represent sets of objects generated from different sensors or different recognition results.

[0135] Object set A (212) may include multiple objects (O1, O2, O3, etc.), and object set B (214) may include multiple objects (L1, L2, etc.).

[0136] The object matching unit (130) creates a correspondence between objects included in object set A (212) and object set B (214) and performs matching between sensors for the same object. Object O1 can be matched with object L1, and object O2 can be matched with object L2. Object O3 can be treated as an unmatched object because there is no corresponding object.

[0137] The relationship configuration unit (140) creates relationships between objects based on object matching results. The relationship configuration unit (140) can create relationships including distance, relative position, attributes, or semantic relationships between objects.

[0138] The relationship configuration unit (140) outputs the generated relationship as relationship information (228).

[0139] The relationship information (228) can subsequently be used as an input to calculate a consistency index in the consistency analysis unit (150). Unmatched objects are not simply removed but can be reflected as separate discrepancies.

[0140] The relationship information (228) provides a structure that allows for reliability evaluation to be performed without directly combining sensor data by utilizing object-to-object matching and relationship configuration.

[0141] FIG. 3 is a diagram illustrating a reliability calculation structure using a plurality of consistency indicators according to the present embodiment.

[0142] As illustrated in FIG. 3, the consistency analysis unit (150) has a plurality of consistency indicators R based on relationship information (228). i (t) is calculated, and the reliability calculation unit (160) calculates the total reliability T(t) using the plurality of consistency indicators.

[0143] The consistency analysis unit (150) can calculate semantic consistency indicators.

[0144] The consistency analysis unit (150) can calculate semantic consistency indicators such as object correspondence, which include an indicator representing the ratio of object pairs corresponding to the same object between different sensors or the stability of correspondence.

[0145] The consistency analysis unit (150) can calculate semantic consistency indicators, such as spatial alignment, which include indicators calculated based on the position or distance error between corresponding pairs of objects.

[0146] The consistency analysis unit (150) can calculate semantic consistency indicators such as confidence coherence, which include indicators calculated based on the difference or variance of object detection confidence values ​​per sensor.

[0147] The consistency analysis unit (150) can calculate a temporal consistency index based on the stability of the change in correspondence relationships and relationship information at consecutive points in time.

[0148] The reliability calculation unit (160) calculates the total reliability T(t) using multiple consistency indicators based on [Equation 15].

[0149] In one embodiment, if any one of the plurality of consistency indicators is calculated as 0 or an extremely small value due to transient noise, sensor loss, or momentary detection failure, a minimum bottom value ε may be applied or a clamping process may be performed to limit the consistency indicator to a value greater than or equal to a preset minimum value in order to mitigate the phenomenon where the total confidence T(t) converges excessively to 0. Additionally, if necessary, the influence of transient outliers may be mitigated by using a recent average value, a moving average value, a mitigation filter, or an exception handling branch.

[0150]

[0151] Here, T(t) represents the system reliability at time t.

[0152] R i (t) represents the i-th consistency index at time t.

[0153] α i represents a weight that reflects the importance of the i-th consistency indicator above.

[0154] N represents the number of consistency indicators used.

[0155] The reliability calculation unit (160) can be performed using a geometric mean method or a non-compensatory aggregation structure, and by ensuring that a low value (deterioration) of a specific indicator is not compensated for by other indicators, abnormal situations can be detected more sensitively. Each consistency indicator R i (t) can be set to be normalized to a range of 0 to 1, and α i The sum of can be set to be 1, but is not limited to this.

[0156] weight α i It reflects the importance of each indicator and can be a fixed value or dynamically adjusted according to system status, environmental changes, and sensor status.

[0157] Referring to FIG. 3, multiple consistency indices R1(t), R2(t), R3(t), R4(t)..., R N (t) represents each consistency value calculated at time t. These consistency indicators are input into the reliability calculation unit (160), and the reliability calculation unit (160) calculates the total reliability T(t) by combining multiple consistency indicators.

[0158] The reliability calculation unit (160) calculates the reliability T(t) according to [Equation 15].

[0159] The reliability calculation method has a non-compensatory structure, and a specific consistency indicator R i It has the characteristic that the overall confidence T(t) decreases significantly when (t) decreases.

[0160] The reliability calculation unit (160) can detect abnormal situations more sensitively by ensuring that the deterioration of some indicators is not compensated by other indicators.

[0161] FIG. 4 is a diagram illustrating the flow of anomaly detection and state determination based on the amount of change in reliability according to the present embodiment.

[0162] Referring to FIG. 4, the anomaly detection unit (170) can operate with a preset time-based analysis structure.

[0163] The anomaly detection unit (170) defines the amount of change in reliability over time ΔT(t) as in [Equation 16].

[0164]

[0165] ΔT(t) represents the difference between the confidence level at the current time point and the confidence level at the previous time point.

[0166] ΔT(t) can be reliably calculated by applying a moving average, analysis of variance, or filtering as needed.

[0167] The anomaly detection unit (170) can determine the state by considering the absolute reliability value T(t), the change amount ΔT(t), or a combination thereof.

[0168] The anomaly detection unit (170) can determine the system state based on reliability T(t) and / or change amount ΔT(t), and can improve the stability of the determination by applying a moving average, analysis of variance, or filtering as needed. The anomaly detection unit (170) can distinguish between a normal state when T(t)≥τ1, a warning state when τ2<T(t)<τ1, and an emergency state when T(t)≤τ2.

[0169] The control unit (180) performs control according to the situation (state). Depending on the situation (state), the control unit (180) can control deceleration, path change, warning output, or switching to safety mode.

[0170] The control unit (180) can perform control in consideration of functional safety requirements such as ISO 26262, and, if necessary, can use the reliability feedback unit (190) to reflect the control results in the next threshold or weight adjustment.

[0171] Referring to Fig. 4, the reliability value T(t) can be analyzed together with the change amount ΔT(t) over time and used to determine whether there is an anomaly.

[0172] The above reliability value and change amount are input to the anomaly detection unit (170), and the anomaly detection unit (170) can classify the system state into Normal, Warning, or Emergency states.

[0173] The anomaly detection unit (170) may be determined to be in a normal state when the reliability value T(t) is greater than or equal to a reference value, and may be determined to be in a warning or emergency state when the reliability value decreases or the change amount ΔT(t) exceeds a specific threshold value.

[0174] The anomaly detection unit (170) transmits the state classification result to the control unit (180). The control unit (180) performs appropriate control according to the system state.

[0175] For example, the control unit (180) can perform control operations such as slowing down the vehicle, changing the route, outputting an alert, or switching to safe mode.

[0176] The abnormality detection unit (170) can determine an abnormal state based on reliability and the amount of change therefrom, and the control unit (180) can improve the safety of the system by performing control corresponding to the result of the abnormality detection unit (170).

[0177] FIG. 5 is a diagram illustrating the operation structure of a sensor module according to the present embodiment.

[0178] Referring to FIG. 5, the operation structure of the sensor unit (110) is illustrated.

[0179] The sensor unit (110) includes multiple sensors such as a camera, LiDAR, and radar, and each sensor collects data independently of each other. The sensor unit (110) does not directly fuse sensor data, but processes the data acquired from each sensor individually and transmits it to the perception unit (120). The sensor unit (110) has a non-fusion structure based on independent processing rather than the combination of sensor data.

[0180] FIG. 5 is a diagram schematically illustrating the overall input and perception structure of a multi-sensor-based non-fusion reliability evaluation system. The sensor unit (110) can independently transmit data input from a plurality of sensors to the perception unit (120). For example, the sensor unit (110) may include a camera, LiDAR, RADAR, infrared sensor, ultrasonic sensor, and similar sensors, but is not limited thereto.

[0181] The sensor unit (110) can evaluate reliability while maintaining the object recognition results for each sensor independently, without directly fusing the sensor data. As illustrated in FIG. 5, each sensor data may not be directly merged or fused before being transmitted to the recognition unit (120).

[0182] The recognition unit (120) can detect objects based on each sensor input and generate basic information for analyzing correspondence relationships and relationship information between objects in a subsequent processing step. The recognition unit (120) can more clearly evaluate structural inconsistencies or semantic inconsistencies between recognition results by sensor while reducing information loss or distortion that may occur during the direct fusion process between sensors.

[0183] FIG. 6 is a diagram illustrating the structure for generating object sets per sensor by a cognitive system according to the present embodiment.

[0184] As illustrated in FIG. 6, the recognition unit (120) performs object recognition based on data input from each sensor. The recognition unit (120) maintains the object sets generated from the sensor unit (110) independently of each other, and can be represented, for example, as a first object set (212) and a second object set (214). Each object set (212, 214) is maintained in a separated state by sensor and performs subsequent processing at the object level.

[0185] FIG. 6 is a diagram showing a structure in which a set of objects generated from each sensor input is maintained in a separated state. A first set of objects (212) and a second set of objects (214) may be generated from a first sensor and a second sensor, respectively. The first set of objects (212) and the second set of objects (214) may represent sets of objects individually detected by each sensor.

[0186] FIG. 6 illustrates an exemplary object set creation structure corresponding to the first sensor and the second sensor for convenience of explanation, and this embodiment can be extended and applied in the same or similar manner to a plurality of sensor configurations including two or more of a camera, lidar, radar, ultrasonic sensor and infrared sensor.

[0187] As illustrated in FIG. 6, the first object set (212) and the second object set (214) are maintained in a state where they are not mutually integrated or fused, and can each be transmitted as input to the subsequent object matching unit (130).

[0188] The recognition unit (120) can perform subsequent comparison and evaluation while keeping each set of objects separated, without combining the object recognition results for each sensor into a single integrated representation in advance.

[0189] The recognition unit (120) maintains the independence of the recognition results for each sensor, thereby enabling a more direct evaluation in subsequent steps of whether object correspondence between different sensors is established, the existence of objects that failed to match, differences in the number of objects, or inconsistencies in structural relationships.

[0190] FIG. 7 is a diagram illustrating the structure for forming object correspondence relationships between sensors by an object matching module according to the present embodiment.

[0191] FIG. 7 illustrates an exemplary structure for forming object correspondence relationships between two sensors for convenience of explanation, and this embodiment can be extended to perform object matching sequentially or in parallel between sets of objects generated from a plurality of sensors.

[0192] Referring to FIG. 7, the object matching unit (130) forms a correspondence between sets of objects generated from different sensors. At this time, the matched object pairs are composed of a matching result (222), and the unmatched objects are identified as unmatched objects (224). The unmatched objects (224) can be reflected as inconsistencies in the reliability evaluation of the system and can be used as a basis for calculating a separate inconsistency index.

[0193] FIG. 7 is a diagram showing a structure for identifying corresponding objects between sensors through an object matching module. The object matching unit (130) can receive a first object set (212) and a second object set (214) as inputs, identify objects corresponding to each other, and generate an object matching result (222).

[0194] The object matching unit (130) may perform object matching based on the position, size, direction of movement, speed, class information, or other feature values ​​of an object, but is not necessarily limited thereto. The object matching unit (130) generates an object matching result (222) that includes a set of object pairs determined to correspond to each other between multiple sensors or corresponding relationship information.

[0195] The object matching unit (130) may not be able to find a corresponding target in the object set of the relative sensor for some objects. The object matching unit (130) classifies the objects for which a corresponding target was not found as unmatched objects (224). The object matching unit (130) may use the object matching result (222) and the unmatched objects (224) as input information for subsequent reliability evaluation.

[0196] In particular, the object matching unit (130) can use the unmatched object (224) as a discrepancy element to calculate subsequent consistency indicators or as a basis for reliability degradation. Therefore, according to the structure illustrated in FIG. 7, not only are the matched objects considered, but the failure of the match itself can also be reflected as an important judgment element in the system reliability evaluation.

[0197] FIG. 8 is a diagram illustrating the structure for generating relationship information between objects by a relationship configuration module according to the present embodiment.

[0198] As illustrated in FIG. 8, the relationship configuration unit (140) generates relationship information between objects based on matched object pairs. The relationship configuration unit (140) generates relationship information (228) including distance between objects, relative position, attribute difference, and semantic relationship. The relationship configuration unit (140) generates relationship information based on object-to-object relations rather than sensor data.

[0199] FIG. 8 is a diagram showing a structure in which relationship information is generated based on object matching results. The relationship configuration unit (140) can generate relationship information (228) by receiving the object matching result (222) as input. The relationship configuration unit (140) can represent not only the relationships between corresponding objects but also the structural and semantic relationships of the entire object group as relationship information (228).

[0200] For example, the relationship configuration unit (140) can generate relationship information (228) including relative distance between objects, direction, placement order, cluster form, adjacency relationship, spatial arrangement, similarity of movement patterns, or semantic interrelationship. The relationship configuration unit (140) can use the relationship information as a basis for evaluating consistency at the level of the entire scene or object group, beyond whether the relationship information matches at the individual object level.

[0201] The reliability calculation unit (160) can perform reliability evaluation by not merely comparing sensor values ​​themselves or comparing only the existence of individual objects, but by comprehensively reflecting the structural and semantic relationships between object groups recognized by multiple sensors. Accordingly, the reliability calculation unit (160) can utilize the relationship information (228) illustrated in FIG. 8 as key input information for subsequent consistency evaluation or reliability calculation.

[0202] FIG. 9 is a diagram illustrating a structure for calculating a plurality of consistency indicators by a consistency analysis module according to the present embodiment.

[0203] Referring to FIG. 9, the consistency analysis unit (150) calculates a plurality of consistency indicators based on relationship information (228). The consistency indicators include spatial consistency (location and distance error), object count consistency, confidence variance, and class matching. The consistency indicators are R i It is represented as (t) and can be composed of multiple elements. The consistency analysis unit (150) has an evaluation structure based on multiple indicators rather than a single indicator.

[0204] The consistency analysis unit (150) can calculate one or more consistency indicators based on object matching results (222) and relationship information (228). The consistency indicators may consist of a set of multiple indicators. For example, R1(t), R2(t), R3(t), R4(t) … , R NIt can be expressed as (t). Each consistency indicator can be configured to quantify consistency between sensors from different perspectives.

[0205] For example, specific consistency indicators may be calculated based on the ratio of corresponding objects, the number of failed matching objects, the degree of preservation of relative distance between objects, similarity of structural arrangement, consistency of semantic relationships, stability of change over time, or other established evaluation criteria. Multiple consistency indicators may reflect different characteristics, and each may be calculated independently or used complementarily.

[0206] The multiple consistency indicators calculated can be provided as input to the subsequent reliability calculation unit (160). The reliability calculation unit (160) can evaluate the state of the system more precisely by comprehensively reflecting multiple structural and semantic consistency indicators, rather than relying on a single indicator.

[0207] FIG. 10 is a diagram illustrating the structure for calculating overall reliability by the reliability calculation module according to the present embodiment.

[0208] As illustrated in FIG. 10, the reliability calculation unit (160) calculates the final reliability T(t) by combining a plurality of consistency indices Ri(t). The reliability T(t) is calculated based on a plurality of consistency indices and, in one embodiment, can be expressed as follows [Equation 15].

[0209] R i (t) represents a consistency index including at least one of object correspondence, spatial alignment, confidence coherence, and temporal consistency. α i represents a weight that reflects the importance of each consistency indicator.

[0210] The reliability calculation unit (160) uses a product-based combination structure and may include various mathematical variations.

[0211] The reliability calculation unit (160) ensures that the combined structure has non-compensatory characteristics, and a decrease in a specific indicator is reflected as a decrease in overall reliability. To mitigate cases where a specific indicator is temporarily lowered due to noise or sensor errors for industrial applications, the reliability calculation unit (160) may selectively reflect only indicators below a certain threshold, or apply log-domain transformation, clipping, lower limit setting, or temporal smoothing techniques. The reliability calculation unit (160) can use temporal smoothing techniques to increase the numerical stability of the reliability calculation and mitigate excessive sensitivity to transient sensor noise.

[0212] FIG. 10 is a diagram showing a configuration in which a final reliability is calculated by combining multiple consistency indicators, and the result is linked to a feedback structure. The reliability calculation unit (160) can calculate a final reliability T(t) by combining multiple consistency indicators Ri(t).

[0213] The confidence level T(t) can be expressed as in [Equation 15]. Here, a weight αi reflecting the relative importance of the corresponding indicator can be applied to each consistency indicator Ri(t). Here, Ri(t) represents the i-th consistency indicator at time t, and α i represents a weight that reflects the relative importance of the indicator, and N may represent the total number of consistency indicators used.

[0214] The final reliability T(t) can be calculated by combining multiple consistency indicators non-compensatorily. The reliability calculation unit (160) can appropriately lower the overall reliability even if other indicators are high when a specific indicator is significantly degraded, thereby allowing for a more sensitive reflection of abnormal conditions or potential risk situations.

[0215] The reliability calculation unit (160) transmits the calculated reliability T(t) to the reliability feedback unit (190) so that it can be used to adjust threshold values, weights, or control-related setting values. Accordingly, according to the structure illustrated in FIG. 10, the reliability calculation result is not limited to a simple output value but can be fed back to the adjustment of internal system parameters or subsequent state determination logic.

[0216] FIG. 11 is a diagram illustrating a state determination structure based on the amount of change in reliability by an anomaly detection module according to the present embodiment.

[0217] Referring to FIG. 11, the anomaly detection unit (170) determines the system state based on the amount of change in reliability. The anomaly detection unit (170) defines the amount of change as [Equation 16]. The anomaly detection unit (170) compares the amount of change with a threshold value to classify the system state as Normal, Warning, or Emergency.

[0218] For example, the anomaly detection unit (170) determines that it is normal when T(t) ≥ τ1. The anomaly detection unit (170) determines that it is warning when τ2 < T(t) < τ1. The anomaly detection unit (170) determines that it is emergency when T(t) ≤ τ2.

[0219] The anomaly detection unit (170) can preemptively determine a warning or emergency state (anomaly state) when ΔT(t) decreases rapidly, for example, when ΔT(t) is below a preset negative threshold. The anomaly detection unit (170) can use the result of the state determination as a criterion for subsequent control.

[0220] FIG. 11 is a diagram showing a structure for detecting an abnormal state based on reliability and a change in reliability. An abnormality detection unit (170) can determine the system state using reliability T(t) and a change amount ΔT(t). Here, ΔT(t) may be a value representing the degree of change in reliability between consecutive points in time.

[0221] The anomaly detection unit (170) can set the state judgment criteria using one or more threshold values ​​τ1, τ2.

[0222] For example, the anomaly detection unit (170) is in a normal state if T(t) ≥ τ1, in a warning state if τ2 < T(t) < τ1, T(t) ≤ If τ2, it can be judged as an emergency state.

[0223] The anomaly detection unit (170) can be additionally utilized as a basis for early detection of an anomaly state even if the current reliability value has not yet reached a complete risk zone when ΔT(t) decreases below a set negative threshold. The anomaly detection unit (170) can reflect not only the absolute reliability level but also the decreasing trend of reliability over time in the state judgment.

[0224] The anomaly detection unit (170) can improve judgment stability by applying moving averages, analysis of variance, filtering, or other time series stabilization techniques as needed. The anomaly detection unit (170) can reduce false detections caused by temporary noise and detect continuous or rapid declines in reliability more reliably.

[0225] FIG. 12 is a diagram illustrating various industrial application examples of a non-converged reliability evaluation device according to the present embodiment.

[0226] As illustrated in FIG. 12, the non-convergent reliability evaluation device (100) can be applied to various industrial fields. For example, the non-convergent reliability evaluation device (100) can be applied to autonomous vehicles, mobile robots, unmanned aerial vehicles (drones), autonomous ships, and industrial automation systems.

[0227] The non-converging reliability evaluation device (100) is not limited to specific industries and can be applied to any system requiring sensor-based perception and control. Additionally, it can transmit and receive reliability information, status judgment results, or control results by linking with a higher-level control system, an external controller, or a user terminal through an external interface (192).

[0228] The following description assumes that a non-fusion type reliability evaluation device (100) is applied to an autonomous vehicle. The sensor unit (110) includes at least one of a camera, LiDAR, and radar, and each sensor independently detects an object to generate an object set M_k(t). The perception unit (120) can generate information such as location, speed, direction, class, and reliability for each object. The object matching unit (130) compares objects between different sensors to determine whether they are the same object and creates a correspondence relationship. At this time, the matching can use a distance criterion between objects, an IoU (Object Bounding Box Overlap Ratio) criterion, a feature vector-based method, or a multi-frame tracking-based method. For example, if the camera and LiDAR recognize the same object, they can be determined to be the same object if the position difference between the two sensors is below a certain threshold. In addition, unmatched objects are not simply removed but are stored as separate mismatch information and can be reflected in subsequent reliability calculations. The consistency analysis unit (150) calculates a plurality of indicators T(t) including object correspondence, spatial consistency, reliability consistency, and temporal consistency based on this. The indicators may additionally include the degree of object number matching, class matching, attribute difference, or stability at consecutive points in time as needed. The reliability calculation unit (160) calculates the total reliability T(t) by combining the indicators in the form of a geometric mean or a non-compensatory combination. The anomaly detection unit (170) determines an abnormal state when the reliability T(t) decreases below a threshold value or when the change amount ΔT(t) decreases rapidly. At this time, judgment based on absolute value and judgment based on change amount may be used alone or in combination. When an abnormal state is detected, the control unit (180) performs measures such as vehicle deceleration, route correction, or outputting a warning signal. The data flow of the present invention is performed in the order of sensor input → object creation → object correspondence → relationship configuration → indicator calculation → reliability calculation → anomaly determination → control.

[0229] Hereinafter, the non-converging reliability evaluation device (100) is described under the assumption that it is applied to the field of sensor errors. When the object recognition result is incomplete due to an error in a specific sensor, the object matching unit (130) identifies the unmatched object and reflects it as a mismatch indicator. As a result, the object correspondence indicator or spatial consistency indicator decreases, and the overall reliability T(t) may decrease rapidly according to the non-compensatory structure. The anomaly detection unit (170) determines a warning or emergency state based on this. Therefore, the present invention can prevent a specific sensor error from being completely obscured by the normal result of another sensor.

[0230] The following description assumes that a non-converging reliability evaluation device (100) is applied to a dynamic environment. In an environment where objects move quickly, the reliability change amount ΔT(t) can change rapidly, and through this, the system can detect environmental changes or abnormal situations in real time. In addition, by tracking changes in relationship information at consecutive points in time, the decrease in temporal consistency can be reflected separately.

[0231] Hereinafter, the non-converging reliability evaluation device (100) is described under the assumption that it is applied to the robot field. The present invention can be applied to a warehouse automation system or a mobile robot, and can analyze the relationship between surrounding objects using multiple sensors and determine the risk of collision or an abnormal state. In this case, the control unit (180) can perform speed reduction, stop, detour, or path replanning.

[0232] Hereinafter, the non-converging reliability evaluation device (100) is described under the assumption that it is applied to the drone / UAV field. The present invention is applied to a drone to detect abnormal situations based on object recognition discrepancies between sensors during flight and to perform path control or safety mode switching.

[0233] Hereinafter, the non-convergent reliability evaluation device (100) is described assuming that it is applied to the shipbuilding field. The present invention can be applied to autonomous ships and can determine collision risk and control the route by analyzing the relationship between radar, camera, and AIS-based objects.

[0234] The following description assumes a case where a non-converging reliability evaluation device (100) is applied to industrial automation equipment. In a sensor-based process, an abnormal state can be detected by utilizing the mismatch in object relationships, and the operating state of the equipment can be controlled according to the result.

[0235] FIG. 12 is a diagram showing an example that can be applied to various industrial fields and an overall application architecture including an external interface and a feedback structure. The non-converging reliability evaluation device (100) can be applied to various industrial fields including autonomous vehicles, mobile robots, drones, autonomous ships, and industrial automation systems. However, this is merely an example and can be applied in the same or similar way to other technology fields where reliability evaluation of multiple sensor-based perception results is required.

[0236] The non-converged reliability evaluation device (100) can transmit and receive information to and from an external controller, a control system, or a higher-level management system through an external interface (192). The non-converged reliability evaluation device (100) may include a reliability feedback unit (190), through which a threshold value, weight, or control policy can be dynamically adjusted.

[0237] For example, if the system reliability of the non-converged reliability evaluation device (100) drops below a certain level, a deceleration command, a stop command, a warning signal output, an alternative sensor activation, an operation mode switching, or a notification to the control system may be performed through the external interface (192). Conversely, if the reliability state is maintained stably, a normal operation policy may be continued, or a relaxed adjustment of learning or correction parameters may be performed.

[0238] The lower implementation example of FIG. 12 may represent an example of an implementation architecture including an ECU, an edge AI platform, message-based middleware, and a parallel processing structure. Accordingly, the present invention does not remain a theoretical concept but can be implemented as a software and hardware structure that can be installed in actual industrial systems.

[0239] FIG. 13 is a diagram illustrating a persistence-based event determination flow according to the present embodiment.

[0240] The persistence-based event determination unit (1300) according to the present embodiment may include a reliability input unit (1310), an anomaly candidate detection unit (1320), a time window setting unit (1330), a persistence calculation unit (1340), a persistence standard comparison unit (1350), an event generation unit (1360), a candidate release unit (1370), and a subsequent processing connection unit (1380).

[0241] The reliability input unit (1310) may receive at least one of reliability T(t), reliability change amount ΔT(t), and consistency index degradation signal from the reliability calculation module (600) or the anomaly detection module (700). The anomaly candidate detection unit (1320) may determine whether an anomaly candidate has occurred based on the input information. In one embodiment, the anomaly candidate detection unit (1320) may detect an anomaly candidate when the reliability T(t) is less than or equal to a preset reference value, when the reliability change amount ΔT(t) is less than or equal to a preset reference value, or when at least one of a plurality of consistency indicators deviates from the reference.

[0242] When an anomaly candidate is detected, the time window setting unit (1330) may set a time interval (1331) of a recent predetermined length W and collect the history of anomaly candidate occurrences within the time interval. The persistence calculation unit (1340) may calculate at least one of the number of anomaly occurrences (1341), the anomaly occurrence rate (1342), and the continuous persistence length (1343) for the time interval.

[0243] The persistence standard comparison unit (1350) can determine whether the calculated persistence information is greater than or equal to a preset persistence standard value (1351). In one embodiment, the persistence standard may be individually set for at least one of the number of abnormal occurrences, the abnormal occurrence rate, and the continuous duration, or it may be set as a combination of multiple standards.

[0244] As a result of comparing the persistence criteria, if the persistence is greater than or equal to the criteria, the event generation unit (1360) can confirm the candidate for the abnormality as an event. The event generation unit (1360) can assign an event ID, record the start time of the event, and confirm the event status. On the other hand, if the persistence is less than the criteria, the candidate release unit (1370) can treat the candidate for the abnormality as a temporary change and not generate an event.

[0245] The subsequent processing connection unit (1380) can transmit the generated event to at least one of an evidence preservation module, an event evaluation module, and a response control module. Accordingly, the present embodiment does not immediately treat a temporary degradation at a single point in time as an event, but evaluates the persistence over multiple points in time and confirms only meaningful anomalies as events.

[0246] FIG. 14 is a drawing illustrating an optional evidence preservation structure including a pre-event section, an event section, and a post-event section according to the present embodiment.

[0247] The event-centered evidence preservation system (1400) according to the present embodiment may include a real-time data collection unit (1410), a circular buffer (1420), an event receiving unit (1430), an event pre-period setting unit (1440), an event period setting unit (1450), an event post-period setting unit (1460), an evidence extraction unit (1470), an evidence lock storage unit (1480), and a subsequent delivery unit (1490).

[0248] The real-time data collection unit (1410) can collect at least some of the information generated during system operation in real time. The collected information may include, for example, reliability history, consistency indicator history, anomaly candidate history, object correspondence relationship information, relationship information summary, sensor-specific detection result summary, but is not limited thereto.

[0249] The circular buffer (1420) can cyclically store the collected information over a recent predetermined time interval. That is, when new data is input, old data can be updated or overwritten. Accordingly, the operational context of the recent interval can be maintained without continuously storing the entire time series data.

[0250] The event receiving unit (1430) can receive an event ID and an event occurrence time from the event generating unit (1360) of FIG. 13. The event pre-period setting unit (1440), event period setting unit (1450), and event post-period setting unit (1460) can each set an event pre-period (1441), an event period (1451), and an event post-period (1461) based on the event occurrence time t_e.

[0251] The evidence extraction unit (1470) can selectively extract data corresponding to the event-previous period, event period, and event-postvious period from the circular buffer (1420). That is, instead of storing all data for the entire time axis, this embodiment can improve storage efficiency by selectively extracting only the context before and after the event.

[0252] In one embodiment, the evidence extraction unit (1470) or the evidence lock storage unit (1480) may differentially set the type, resolution, or scope of information to be preserved according to the severity level of the event. For example, if the severity is low, preservation may focus on reliability history, consistency indicator summaries, or status judgment logs; if the severity is medium or higher, detailed relationship information or response history of the entire event period, event period, and post-event period may be additionally preserved; and if the severity is high, raw sensor data, high-resolution detection results, or extended evidence including the entire related surrounding period may be preserved. Accordingly, the present embodiment can adaptively adjust storage efficiency and the level of evidence acquisition according to severity.

[0253] The evidence lock storage unit (1480) can store the extracted data as an event-unit preservation record (1481) and, if necessary, can assign lock status information (1482). The lock status information may include a function to restrict the stored evidence record from being modified or deleted. Accordingly, data related to the event can be maintained as preservation material that can be used for post-analysis and auditing, rather than as a simple temporary log.

[0254] The subsequent delivery unit (1490) can deliver the above-mentioned preservation record to at least one of an event evaluation module, a response control module, an accountability packet generation module, or an external interface. Accordingly, the present embodiment can secure sufficient contextual information before and after an event without storing the entire data at all times.

[0255] FIG. 15 is a diagram illustrating an anomaly type classification and severity-based response structure according to the present embodiment.

[0256] The event evaluation and response system (1500) according to the present embodiment may include an event information input unit (1510), an anomaly type classification unit (1520), a severity factor calculation unit (1530), a severity calculation unit (1540), a response level determination unit (1550), a response execution unit (1560), and a subsequent transmission unit (1570).

[0257] The event information input unit (1510) can receive at least one of the event ID, reliability change history, consistency indicator summary, and evidence information of the event pre-period, event period, and event post-period preserved according to FIG. 14.

[0258] An anomaly type classification unit (1520) can classify the anomaly type of an event based on the input information. In one embodiment, the anomaly type may include at least one of a sensor-dominant type inconsistency (1521), a multiple-indicator simultaneous degradation type inconsistency (1522), and a structural semantic inconsistency (1523). A sensor-dominant type inconsistency may refer to a case where an inconsistency occurs predominantly in a specific sensor or a specific sensor series; a multiple-indicator simultaneous degradation type inconsistency may refer to a case where multiple consistency indicators are simultaneously degraded; and a structural semantic inconsistency may refer to a case where a semantic contradiction occurs at the level of scene interpretation or object relationship between sensors.

[0259] The severity factor calculation unit (1530) can calculate at least one of the magnitude of degradation (1531), persistence (1532), abruptness of change (1533), and whether simultaneous degradation occurs (1534). The magnitude of degradation may refer to the degree of decrease in reliability or indicator values, persistence may refer to the time or frequency during which the abnormal state persists, abruptness of change may refer to the magnitude of the change in reliability, and whether simultaneous degradation occurs may refer to whether multiple indicators or multiple relationships have deteriorated simultaneously.

[0260] In one embodiment, the severity calculation unit (1540) may calculate the severity level by combining element scores or judgment results corresponding to the magnitude of degradation, persistence, rapid change, and whether there is simultaneous degradation, respectively. For example, the magnitude of degradation may be set to 0 points if it is less than a first criterion, 1 point if it is greater than or equal to the first criterion but less than the second criterion, and 2 points if it is greater than or equal to the second criterion. Similarly, the same or similar segment-specific scores or binary judgment results may be assigned to the persistence, rapid change, and whether there is simultaneous degradation. If the sum, weighted sum, or rule-based combination result of the element scores falls within a preset segment, the severity may be determined as Level 0, Level 1, Level 2, or Level 3.

[0261] The severity calculation unit (1540) can calculate the severity score or severity level (1541) of an event based on the above elements. In one embodiment, the severity level may be expressed as a multi-stage structure from Level 0 to Level 3, but is not limited thereto. In one embodiment, the severity calculation unit (1540) calculates the severity score by combining the results of comparisons of scores or thresholds corresponding to the magnitude of degradation, persistence, rapid change, and whether simultaneous degradation occurs, and maps the severity score to Level 0 to Level 3 depending on whether it falls within a preset range.

[0262] The response level determination unit (1550) can determine the response level according to the severity level. For example, it can decide to maintain observation (1551) for Level 0, retain logs (1552) for Level 1, lock evidence (1553) or issue a warning (1554) for Level 2, and provide a mitigation support signal (1555) or safety control (1556) for Level 3. Each of the responses does not need to be mutually exclusive, and multiple responses may be performed in parallel. At this time, the preset interval or threshold may be adjusted by the reliability feedback module (900) according to the system state, past event history, or application environment.

[0263] The response execution unit (1560) can actually perform the determined response and record the corresponding response history. The subsequent transmission unit (1570) can transmit the response result to a control module, a recording module, an accountability packet generation module, or an external interface. Accordingly, the present embodiment is not limited to simply detecting events, but can perform differential and proportional responses according to the type and severity of the anomaly.

[0264] FIG. 16 is a diagram illustrating the structure of an accountability packet according to the present embodiment.

[0265] The accountability packet (1600) according to the present embodiment is a structured record unit corresponding to one event and may include event identification information (1610), time information (1620), reliability information (1630), consistency indicator information (1640), persistence information (1650), anomaly type information (1660), severity information (1670), response history information (1680), and evidence reference information (1690).

[0266] Event identification information (1610) may include an event ID, a packet ID, and an event status. Time information (1620) may include an event start time, an event end time, and a packet creation time. Reliability information (1630) may include a reliability change history and a summary of the reliability change amount. Consistency indicator information (1640) may include summary information and degradation item information for a plurality of consistency indicators.

[0267] The persistence information (1650) may include the number of anomalies, the anomaly occurrence rate, and the duration of the continuous duration. The anomaly type information (1660) may include at least one of sensor-dominant inconsistency, multiple indicator simultaneous degradation inconsistency, and structural semantic inconsistency. The severity information (1670) may include a severity score and a severity level.

[0268] The response history information (1680) may include information on the actual response performed for an event. In one embodiment, the response history information may include at least one of whether to maintain observation, whether to preserve a log, whether to lock evidence, whether to output a warning, whether to provide mitigation support, and whether to perform safety control. Accordingly, it is possible to record in a traceable manner what response the system performed for a single event.

[0269] The evidence reference information (1690) may include information for linking to the preserved evidence record in FIG. 14. In one embodiment, the evidence reference information (1690) may include pre-event reference information (1691), event section reference information (1692), and post-event section reference information (1693). Accordingly, the accountability packet (1600) may provide a connection relationship between the event-related key summary information and the preserved evidence.

[0270] The above accountability packet (1600) can be transmitted to a control module, a log module, an external interface, an audit system, or a post-analysis system. Accordingly, the present embodiment can integrate the results of the occurrence, evaluation, response, and evidence preservation of an event into a single structured packet for use in post-analysis, auditing, external reporting, and control history tracking.

[0271] In one embodiment, the accountability packet may be transmitted to a control system, an audit system, a post-analysis system, or an incident investigation system via an external interface. Accordingly, the accountability packet may be utilized for analyzing the cause of an event, verifying response history, evaluating the appropriateness of operational policies, presenting explainable grounds for judgment, or generating legal or administrative evidence. For example, in the event of an abnormal event occurring in an autonomous vehicle or mobile robot, the accountability packet may be used as foundational data for post-event review and accountability tracking by providing the context before and after the event, changes in reliability, severity assessment, and the response measures taken.

[0272] FIG. 17 is a diagram illustrating a response policy map by severity level according to the present embodiment.

[0273] Referring to FIG. 17, the response policy map (1700) according to the present embodiment may include a severity input unit (1710), a Level 0 policy area (1720), a Level 1 policy area (1730), a Level 2 policy area (1740), a Level 3 policy area (1750), and a policy execution result output unit (1760).

[0274] The severity input unit (1710) can receive a severity level calculated from the event evaluation module (770). In one embodiment, the severity level may be divided into a multi-level structure of Level 0, Level 1, Level 2, and Level 3, but is not limited thereto. The severity level may be calculated based on at least one of the magnitude of the event's degradation, persistence, rapid change, and whether multiple indicators deteriorate simultaneously, as described in FIG. 15.

[0275] Level 0 policy area (1720) may include observation maintenance (1721). Observation maintenance (1721) may be a policy that continues to monitor the current state without performing immediate intervention. For example, if an anomaly candidate or event is likely to be of low severity or transient, the system may be configured to perform only additional log accumulation or state tracking.

[0276] Level 1 policy area (1730) may include log retention (1731). Log retention (1731) may be a policy that stores operational history, buffer logs, or status information related to an event. In one embodiment, the log retention may be performed to increase the possibility of post-analysis of event-related information.

[0277] The Level 2 policy area (1740) may include evidence locking (1741) and warning output (1742). Evidence locking (1741) may be a policy that restricts preservation records or summary information related to an event from being changed or deleted. Warning output (1742) may be a policy that notifies an operator, a higher-level control system, or an external monitoring system of an abnormal situation. In one embodiment, the Level 2 policy may be a step for performing preservation enhancement and notification for event-like events beyond a simple record level.

[0278] The Level 3 policy area (1750) may include a mitigation support signal (1751) and a safety control (1752). The mitigation support signal (1751) may be a signal to support the system to operate more conservatively, and the safety control (1752) may include deceleration, route change, stop, switch to safety mode, or equivalent direct intervention control. In one embodiment, the Level 3 policy may be a response step to prioritize ensuring system safety for high-severity events.

[0279] The above Level 0 to Level 3 policies do not need to be completely mutually exclusive, and lower-level policies may be executed together with higher-level policies as needed. For example, in the case corresponding to Level 3, log retention, evidence locking, and warning output may be executed together, while additional mitigation support signals or safety controls may be performed.

[0280] The policy execution result output unit (1760) can transmit the result of the executed policy to one or more subsequent modules. In one embodiment, the policy execution result output unit (1760) may include a record module transmission unit (1761), a control module transmission unit (1762), an accountability packet generation module transmission unit (1763), and an external interface transmission unit (1764).

[0281] The record module transmission unit (1761) can transmit the executed policy and response history to the record module. The control module transmission unit (1762) can transmit control-related results among the executed policies to the control module. The accountability packet generation module transmission unit (1763) can transmit information to the generation module to include the response history and policy execution results for the above event in the accountability packet. The external interface transmission unit (1764) can transmit the policy execution results to an external monitoring system, an audit system, a post-analysis system, or an external reporting system.

[0282] Accordingly, the present embodiment can apply response intensity differentially in stages according to the severity level of the event, and can provide a policy-based response system that includes recording, preservation, warning, mitigation, and safety control, going beyond simple detection. In addition, by linking the results of the policy execution with a recording module, a control module, an accountability packet generation module, and an external interface, event-centric traceability and the possibility of post-analysis can be enhanced.

[0283] FIG. 18 is a diagram illustrating the overall structure of a non-converging reliability assessment, event-based evidence preservation, and risk-adaptive response system according to a second embodiment.

[0284] Referring to FIG. 18, the system according to the present embodiment may include a sensor module (100), a perception system (200), an object matching module (300), a relationship configuration module (400), a consistency analysis module (500), a reliability calculation module (600), an anomaly detection module (700), an event determination module (750), an evidence preservation module (760), an event evaluation module (770), an event recording module (780), a response control module (800), a reliability feedback module (900), and an external interface (1000).

[0285] The sensor module (100) may include a camera, LiDAR, radar, or other environment recognition sensor and may acquire data about the external environment. The present embodiment is characterized by a non-fusion structure that performs subsequent processing based on object recognition results generated independently from each sensor without directly fusing sensor data.

[0286] The perception system (200) can detect objects based on data input from the sensor module (100) and generate a set of objects per sensor. The object matching module (300) can form a correspondence relationship between sets of objects generated from different sensors. The relationship configuration module (400) can generate relationship information including distance, relative position, attributes, or semantic relationships between objects based on the correspondence relationship.

[0287] The consistency analysis module (500) can calculate a plurality of consistency indicators based on the relationship information. The consistency indicators may include at least one of object correspondence, spatial consistency, reliability consistency, and temporal consistency. The reliability calculation module (600) can calculate the reliability T(t) of the system by combining the plurality of consistency indicators in a non-compensatory structure.

[0288] The anomaly detection module (700) can detect anomaly candidates based on the reliability T(t) or the reliability change amount ΔT(t). The event determination module (750) can evaluate the persistence of the anomaly candidates and generate an event only if it is not a temporary change.

[0289] The evidence preservation module (760) may selectively preserve information regarding the event pre-period, event period, and event post-period corresponding to the generated event. The event evaluation module (770) may classify anomaly types and calculate severity based on the preserved event-related information. In one embodiment, the anomaly types may include at least one of sensor-dominant inconsistency, multiple indicator simultaneous degradation inconsistency, or structural semantic inconsistency, and the severity may be calculated by combining element scores or judgment results corresponding to the magnitude, persistence, rapid change, and simultaneous degradation of multiple indicators, respectively, of the reliability degradation. For example, a threshold comparison result or a score per section may be assigned to each of the elements, and a severity level of Level 0 to Level 3 may be determined by combining them in a summation, weighted sum, or rule-based manner, and the threshold, section, or weight may be adjusted according to the system status, operational history, or reliability feedback results.

[0290] The event record module (780) can generate a structured event record corresponding to the event. In one embodiment, the structured event record may be an accountability packet and may include event identification information, time information, reliability information, consistency indicator information, persistence information, anomaly type information, severity information, response history information, and evidence reference information.

[0291] The response control module (800) may perform at least one of warning output, log retention, evidence locking, mitigation support signal generation, deceleration, route change, or safe mode switching according to the severity. The reliability feedback module (900) may adjust the threshold or weight for the next point in time using the response result, status judgment result, or reliability information from the previous point in time.

[0292] The external interface (1000) can transmit and receive the event record, status determination result, or control result to an external controller, control system, audit system, post-analysis system, or user terminal. Accordingly, the present embodiment can provide an integrated operational structure that goes beyond non-convergence-based reliability evaluation and control to include event-centric evidence preservation, severity-based response, structured record generation, and external linkage.

[0293] FIG. 19 is a diagram illustrating a reliability calculation structure based on multiple indicators according to a second embodiment.

[0294] Referring to FIG. 19, the consistency analysis module (500) can calculate a plurality of consistency indicators (510), and the reliability calculation module (600) can calculate the overall reliability T(t) using the plurality of consistency indicators (510) and weights (610) corresponding to each consistency indicator.

[0295] In one embodiment, the plurality of consistency indicators (510) may include at least one of object correspondence R₁(t), spatial consistency R₂(t), reliability consistency R₃(t), and temporal consistency R₄(t). Object correspondence may indicate the degree to which objects detected by different sensors correspond to the same object, spatial consistency may indicate the degree of positional or distance matching between corresponding objects, reliability consistency may indicate the degree of matching of object detection reliability values ​​per sensor, and temporal consistency may indicate the degree to which relationship information is stably maintained at consecutive points in time.

[0296] The weights (610) may reflect the relative importance of each consistency indicator. In one embodiment, the weights (610) may include α₁, α₂, α₃, and α₄, each of which may be used to adjust the influence of the corresponding consistency indicator. The weights may be set as fixed values ​​and may be adjusted as needed according to system status, environmental changes, sensor status, or external settings.

[0297] The reliability calculation module (600) can calculate the overall reliability T(t) by combining the plurality of consistency indicators (510) in a non-compensatory structure. Here, a non-compensatory structure refers to a combination structure that prevents other indicators from completely offsetting a significant decrease in one core consistency indicator, even if they have high values.

[0298] In one embodiment, the reliability calculation may be performed by a geometric mean multiplication structure. For example, the overall reliability T(t) may be expressed according to the following mathematical formula 1.

[0299] In one embodiment, if any one of the plurality of consistency indicators is calculated as 0 or an extremely small value due to transient noise, sensor loss, or momentary detection failure, a minimum bottom value ε may be applied or a clamping process may be performed to limit the consistency indicator to a value greater than or equal to a preset minimum value in order to mitigate the phenomenon where the total confidence T(t) converges excessively to 0. Additionally, if necessary, the influence of transient outliers may be mitigated by using a recent average value, a moving average value, a mitigation filter, or an exception handling branch.

[0300]

[0301] Here represents the i-th consistency index at time t, and represents the weight corresponding to the i-th consistency indicator, and N represents the number of consistency indicators used. The consistency indicators may be normalized to a range of 0 to 1, and the sum of the weights may be set to be 1, but is not limited thereto.

[0302] The overall reliability T(t) calculated by the reliability calculation module (600) can be output through the overall reliability output unit and can be transmitted to the anomaly detection module (700). Accordingly, the anomaly detection module (700) can detect anomaly candidates based on the overall reliability T(t) and, if necessary, the reliability change amount ΔT(t).

[0303] That is, the structure of FIG. 19 according to the present embodiment provides a core structure that calculates a comprehensive reliability T(t) by non-compensatorily combining a plurality of semantic and structural consistency indicators with weights, and enables the calculated reliability T(t) to be used as an input for subsequent anomaly detection, state determination, and event-based processing.

[0304] FIG. 20 is a diagram illustrating an abnormal candidate detection and state determination structure according to a second embodiment.

[0305] Referring to FIG. 20, the anomaly detection module (700) may receive a reliability T(t), a reliability T(t1) at a previous time point, and, if necessary, one or more consistency indicators. The reliability change amount calculation unit (710) may calculate a reliability change amount ΔT(t) based on the difference between the reliability T(t) at the current time point and the reliability T(t-1) at a previous time point.

[0306] ΔT(t)=T(t)-T(t-1)

[0307] Anomaly candidate detection can detect anomaly candidates based on whether the reliability T(t), the reliability change amount ΔT(t), or at least one of the plurality of consistency indicators has decreased. In one embodiment, the anomaly candidate detection unit (730) can determine that an anomaly candidate exists if the reliability T(t) is below a preset threshold value, if the reliability change amount ΔT(t) is below a preset threshold value, or if at least one of the plurality of consistency indicators has decreased below the threshold.

[0308] If the existence of an anomaly candidate is denied, the system is determined to be in a no-candidate state and can maintain normal monitoring. On the other hand, if an anomaly candidate exists, the state determination unit (720) can classify the system state into normal, warning, or emergency states based on the reliability T(t), the reliability change amount ΔT(t), or the deterioration state of the consistency index.

[0309] Subsequent transmission can transmit the above state determination result or abnormal candidate information to the corresponding control module (800) and the event determination module (750). Accordingly, the corresponding control module (800) can perform control corresponding to the state at the current time point, and the event determination module (750) can determine whether to generate an event by further evaluating whether the above abnormal candidate persists over multiple time points.

[0310] That is, the structure of FIG. 20 according to the present embodiment provides a higher-level judgment structure that detects anomaly candidates at the current time and classifies the state based on reliability and change amount, and the judgment result can be used as an input for subsequent persistence-based event judgment and response control.

[0311] FIG. 21 is a diagram illustrating a plurality of consistency indicator calculation structures according to a second embodiment.

[0312] Referring to FIG. 21, the consistency analysis module (500) can receive object correspondence results and relationship information between objects from the relationship information input unit (410) and calculate a plurality of consistency indicators (510). In one embodiment, the relationship information may include object matching results, distance between objects, relative positions, attribute relationships, or temporal relationship information.

[0313] The consistency analysis module (500) may include an object correspondence calculation unit (520), a spatial consistency calculation unit (530), a reliability consistency calculation unit (540), and a temporal consistency calculation unit (550).

[0314] The object correspondence calculation unit (520) can calculate the degree to which objects detected from different sensors are stably matched to the same object. In one embodiment, the object correspondence calculation unit (520) can calculate an object correspondence index R₁(t) based on the ratio of corresponding objects, the existence of non-corresponding objects, or matching stability.

[0315] The spatial consistency calculation unit (530) can calculate the degree of agreement of the position, distance, or geometric relationship between corresponding objects. In one embodiment, the spatial consistency calculation unit (530) can calculate a spatial consistency index R₂(t) based on the position error, distance error, or geometric agreement.

[0316] The reliability consistency calculation unit (540) can calculate the degree of reliability consistency by comparing the difference or variance of object detection reliability values ​​for each sensor. In one embodiment, the reliability consistency calculation unit (540) can calculate a reliability consistency index R₃(t) based on the difference, variance, or deviation of detection reliability.

[0317] The temporal consistency calculation unit (550) can calculate the degree to which relationship information is stably maintained at consecutive points in time. In one embodiment, the temporal consistency calculation unit (550) can calculate a temporal consistency index R₄(t) based on the stability of consecutive points in time, the degree of agreement of the change trend, or the variation characteristics of the relationship information over time.

[0318] The above object correspondence index R₁(t), spatial consistency index R₂(t), reliability consistency index R₃(t), and temporal consistency index R₄(t) can be output as a plurality of consistency indices (510). The plurality of consistency indices (510) can be normalized to a range of 0 or more and 1 or less as needed, and can be transmitted to a reliability calculation module (600) to be used for calculating the overall reliability T(t).

[0319] That is, the structure of FIG. 21 according to the present embodiment represents a structure that calculates a plurality of semantic and structural consistency indicators in parallel based on relationship information between objects, and provides the calculated indicators as inputs for subsequent reliability calculations.

[0320] FIG. 22 is a diagram illustrating a reliability calculation and feedback structure according to a second embodiment.

[0321] Referring to FIG. 22, the reliability calculation module (600) can calculate the reliability T(t) of the system using a plurality of consistency indicators (510) calculated at time t and weights (610) corresponding to each indicator.

[0322] The reliability calculation module (600) can calculate the reliability T(t) by combining the plurality of consistency indicators into a non-compensatory structure. In one embodiment, the non-compensatory structure may be a combination structure that prevents other indicators from completely offsetting the deterioration of one core consistency indicator. The reliability calculation may be performed by a geometric mean multiplication structure or a similar non-compensatory combination method.

[0323] The calculated reliability T(t) can be output through the reliability output unit and can be transmitted to the anomaly detection module (700). Accordingly, the anomaly detection module (700) can detect anomaly candidates based on the reliability T(t) and, if necessary, the reliability change amount ΔT(t).

[0324] Meanwhile, the reliability feedback module (900) may receive at least one of a state determination result, a control result, and external policy or setting information. In one embodiment, the state determination result may include information regarding a normal, warning, or emergency state, the control result may include results of deceleration, path change, warning output, mitigation support, or safety control execution, and the external policy or setting information may include a threshold value, policy value, or setting value input from an external device or a higher-level system.

[0325] The reliability feedback module (900) can adjust at least one of the weight, threshold, or judgment criterion based on the input information. The result of the adjustment can be reflected in the weight (610), the reliability calculation module (600), or the anomaly detection module (700).

[0326] That is, the structure of FIG. 22 according to the present embodiment does not limit the calculation of reliability T(t) to a fixed one-time calculation, but forms an adaptive closed-loop structure that reflects state judgment results, control results, and external policy information, thereby enabling a more flexible response to environmental changes and system state changes.

[0327] FIG. 23 is a diagram illustrating an anomaly candidate detection structure based on reliability change amount according to a second embodiment.

[0328] Referring to FIG. 23, the anomaly detection module (700) may receive the reliability T(t) at the current time point and the reliability T(t-1) at the previous time point. The reliability change amount calculation unit (710) may calculate the reliability change amount ΔT(t) based on the difference between the reliability T(t) at the current time point and the reliability T(t-1) at the previous time point. The reliability change amount ΔT(t) may be calculated according to Equation 16.

[0329] The absolute value standard comparison unit (730) can determine whether the reliability T(t) at the current time is less than or equal to a preset absolute value standard, and the change amount standard comparison unit (740) can determine whether the reliability change amount ΔT(t) is less than or equal to a preset change amount standard.

[0330] The abnormal candidate determination unit (725) can determine an abnormal candidate based on at least one of the absolute value criterion comparison result and the change amount criterion comparison result. The state determination unit (720) can classify the system state into normal, warning, or emergency states based on the abnormal candidate.

[0331] The subsequent transmission unit can transmit the above abnormal candidate information or state judgment result to the event judgment module (750) and the response control module (800). Accordingly, the event judgment module (750) can determine whether to generate an event by evaluating whether the above abnormal candidate persists over multiple time points, and the response control module (800) can perform response control according to the state at the current time point.

[0332] That is, the structure of FIG. 23 according to the present embodiment can sensitively detect anomaly candidates at the current point in time by considering both the absolute value and the amount of change of reliability, and can provide a multi-stage judgment structure that links the detection result at a single point in time to a subsequent persistence-based event judgment without immediately confirming it as a final event.

[0333] FIG. 24 is a drawing illustrating an industrial application and external linkage structure according to a second embodiment.

[0334] Referring to FIG. 24, the common operation platform (1210) may include non-fusion reliability evaluation, event-based evidence preservation, severity-based response, and accountability packet generation functions. That is, the common operation platform (1210) can provide a common functional layer capable of evaluating reliability based on relationships between objects and semantic consistency without directly fusing sensor data in a multi-sensor-based autonomous system, and performing event-based evidence preservation, severity assessment, and phased response to abnormal situations.

[0335] The above common operating platform (1210) may be applied to at least one of an autonomous vehicle application unit (1220), a mobile robot application unit (1230), a drone application unit (1240), an autonomous vessel application unit (1250), and an industrial automation equipment application unit (1260). Accordingly, this embodiment is not limited to a specific industry or a specific mobile body, but can be applied to various autonomous systems that recognize the environment, determine the state, and perform control using multiple sensors. For example, in the mobile robot application unit (1230), the common operating platform may be linked to perform obstacle avoidance, speed reduction, or path replanning based on discrepancies between sensors. Additionally, in the industrial automation equipment application unit (1260), it may be linked to equipment speed reduction, process pause, or operator warning output upon the occurrence of an event.

[0336] Meanwhile, the above common operation platform (1210) may be connected to an external control system linkage unit (1270), an audit system linkage unit (1280), and a post-analysis / reporting system linkage unit (1290). The external control system linkage unit (1270) may be a linkage structure for transmitting event occurrence information, status judgment results, warning information, or control results to an external control system, or for receiving policy information, setting values, or operation commands from an external control system. The audit system linkage unit (1280) may be a linkage structure that enables post-audit or accountability tracking using preserved event records, response history, or accountability packets. The post-analysis / reporting system linkage unit (1290) may be a linkage structure for analyzing information, abnormality types, severity, and response results of the pre-event, event, and post-event periods, or for utilizing this in the form of a report.

[0337] That is, the structure of FIG. 24 according to the present embodiment shows that the common operating platform of the present embodiment can be commonly applied to various autonomous systems including autonomous vehicles, mobile robots, drones, autonomous ships, and industrial automation facilities, and at the same time indicates that event recording and response results can be linked with external control, audit, post-analysis, and reporting systems.

[0338] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment. Explanation of the symbols

[0339] 100: Non-converged reliability evaluation device 110: Sensor section 120: Cognitive Department 130: Object Matching Section 140: Relationship component 150: Consistency Analysis Department 160: Reliability Calculation Unit 170: Anomaly Detection Unit 180: Control unit 190: Reliability Feedback Section 192: External Interface

Claims

Claim 1 A sensor unit comprising a plurality of sensors; a recognition unit that generates a set of objects for each sensor based on sensor data acquired from the sensor unit; an object matching unit that performs object matching between the set of objects for each sensor to generate an object matching result; a relationship configuration unit that generates relationship information, which is structured data including relationships between objects, based on the object matching result; and a plurality of consistency indicators R based on the relationship information. i A consistency analysis unit that calculates (t); the plurality of consistency indicators R i Weights (á) corresponding to each (t) i A non-converged reliability evaluation device comprising: a reliability calculation unit that calculates reliability T(t) in a non-compensatory structure by reflecting ); an anomaly detection unit that generates an anomaly state determination result by determining an anomaly state using the reliability T(t) and the reliability change amount Δ T(t); and a control unit that performs control to change system operation according to the anomaly state determination result; wherein the non-converged reliability evaluation device further comprises an event recording module that generates an accountability packet including event identification information, time information, reliability information, consistency indicator information, persistence information, anomaly type information, severity information, response history information, and evidence reference information. Claim 2 A non-fusion type reliability evaluation device according to claim 1, wherein the sensor unit comprises at least one of a camera, lidar, radar, ultrasonic sensor, and infrared sensor. Claim 3 A non-fusion type reliability evaluation device according to claim 1, wherein the above-mentioned perception unit has a non-fusion structure that does not directly fuse the sensor data. Claim 4 A non-fusion type reliability evaluation device according to claim 1, wherein the object matching unit performs object matching between object sets for each sensor and identifies unmatched objects. Claim 5 A non-fusion type reliability evaluation device according to claim 1, wherein the relationship component generates relationship information including distance between objects, relative position, attributes, or semantic relationships based on the object matching result. Claim 6 In claim 1, the consistency analysis unit includes at least one of object correspondence, spatial consistency, reliability consistency, and temporal consistency based on the relationship information, the consistency index R i A non-fusion type reliability evaluation device characterized by calculating (t). Claim 7 A non-fusion type reliability evaluation device according to claim 1, wherein the reliability calculation unit calculates the reliability T(t) using sensor-specific object sets, object correspondence relationships, and inter-object relationship information without directly fusing the sensor data. Claim 8 A non-fusion type reliability evaluation device according to claim 1, wherein the anomaly detection unit calculates the reliability change amount ΔT(t) based on the difference between the reliability T(t) at the current time point and the reliability T(t-1) at the previous time point, and generates an anomaly state determination result by determining an anomaly state using the reliability T(t) and the reliability change amount ΔT(t). Claim 9 A non-fusion type reliability evaluation device according to claim 1, wherein the control unit controls one or more operations among deceleration, path change, warning output, or safety mode switching according to the result of the abnormal state determination. Claim 10 A non-fusion type reliability evaluation device according to claim 1, wherein the anomaly detection unit further includes an event determination module that determines whether the anomaly state determination result persists over multiple time points and generates an event. Claim 11 In claim 10, the non-fusion type reliability evaluation device is characterized by further including an evidence preservation module that selectively preserves information of the event-previous period, event period, and event-post-period corresponding to the event. Claim 12 delete Claim 13 A process in which a sensor unit including multiple sensors acquires multiple sensor data; a process in which a recognition unit generates a set of objects for each sensor based on the sensor data acquired from the sensor unit; a process in which an object matching unit performs object matching between the object sets for each sensor to generate an object matching result; a process in which a relationship configuration unit generates relationship information, which is structured data including relationships between objects, based on the object matching result; and a process in which a consistency analysis unit generates multiple consistency indicators R based on the relationship information. i A process for calculating (t); a reliability calculation unit, the above plurality of consistency indicators R i Weights (á) corresponding to each (t) i A non-converging reliability evaluation method characterized by comprising: a process of calculating reliability T(t) in a non-compensatory structure by reflecting ); a process in which an anomaly detection unit generates an anomaly state determination result by determining an anomaly state using the reliability T(t) and the reliability change amount Δ T(t); a process in which a control unit performs control to change system operation according to the anomaly state determination result; and a process in which an event recording module generates an accountability packet including event identification information, time information, reliability information, consistency indicator information, persistence information, anomaly type information, severity information, response history information, and evidence reference information.

Images