Small target description method based on causal reasoning
Through a small-object description method based on causal reasoning and combined with C-AOG inference diagram, the problem of small-object detection in autonomous driving systems is solved, especially effective detection under occlusion, which improves the safety of autonomous driving systems.
Patent Information
- Application Number
- CN202411539118.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-07
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In autonomous driving systems, it is difficult for vision sensors to effectively detect small targets because these small targets are easily blocked, resulting in rapid sparseness of pixel-level features, affecting the response capability and safety of the autonomous driving system.
Using a small-objective description method based on causal inference, map BEV-perceptual sequences to specific by constructing atomic knowledge nodes, sampling data based on long sequences, encoding behavioral events using KNN classifiers, and performing probability inference through C-AOG inference graphs. visual status.
It achieves small targets that can still be effectively detected when small targets are blocked, and improves the safety and response capabilities of the autonomous driving system.
Smart Images
Figure CN119398175A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection for autonomous driving vehicles based on causal reasoning, and in particular to a small target perception algorithm for autonomous driving vehicles. Background Art
[0002] In autonomous driving systems, the widespread use of visual sensors such as cameras is due to their low cost, high resolution, and ability to obtain rich color and texture information. This makes visual sensors play an important role in scene understanding. However, a major challenge for visual sensors is small target detection. Because even small objects, such as children, pets, bicycles, or roadside obstacles, can have a huge impact on vehicle driving safety. Such small targets may exist on the road surface, on the roadside, or in complex traffic environments. Detecting these small objects is crucial to preventing accidents.
[0003] In actual road environments, occlusion is a common phenomenon. Moving vehicles, trees, buildings, or other objects may occlude small targets, making it impossible for the autonomous driving system to perceive the existence of these small objects in a timely manner. When small targets interact with other targets in complex traffic scenarios, occlusion often leads to rapid sparseness of pixel-level features. In this case, BEV tracking may be interrupted intermittently, thereby reducing the overall situational awareness of traffic dynamics, causing the responsiveness of the autonomous driving system to be seriously affected, further bringing potential hazards, such as: increased risk of safety accidents, autonomous driving decision-making errors, and human-machine trust crisis. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the prior art and propose a small target description method based on causal reasoning. The present invention designs a small target state description method based on causal reasoning and constructs a reasoning graph based on C-AOG.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A small target description method based on causal reasoning, characterized by comprising the following steps:
[0007] S1: Construct atomic knowledge nodes to represent the behavioral characteristics of small targets and surrounding targets;
[0008] S2: Constrain the historical data of small targets and surrounding targets into a behavior set based on long sequence sampling data;
[0009] S3: Encode behaviors scattered in the BEV space into events using a KNN-based classifier;
[0010] S4: Construct a reasoning graph by fuzzy aggregation of behaviors and map BEV perception sequences to specific visualization states based on probabilistic reasoning;
[0011] S5: Evaluate and optimize the model
[0012] Preferably, atomic knowledge nodes are constructed in S1 to represent the behavioral characteristics of the small target and surrounding targets.
[0013] Preferably, the specific steps in S1 are as follows:
[0014] The BEV perception sequence S from time 1 to T is defined as:
[0015] S={s t , t=1,…,T}
[0016] Among them, s t Represents the BEV information of each time segment t, including a certain number of perception targets o ti Therefore, s t is defined as:
[0017] s t ={o ti , i=1,…,N t}
[0018] N t represents the total number of objects perceived in time t, each o ti Contains the type of the target c ti , and the position relationship related to the vehicle BEV mapping l ti , visibility v ti With atomic behavior a ti relationship.
[0019] Preferably, S2 constrains the historical data of the small target and the surrounding targets into a behavior set based on the long sequence sampling data.
[0020] Preferably, the specific steps in S2 are as follows:
[0021] On structured roads, according to the vehicle's motion trajectory and speed changes, the speed and turning angle are mapped to the tangent of the curve, achieving the homogenization of the vehicle's behavior between curved and straight segments; in the behavior set, the vehicle's driving state includes 9 types of maneuvering behaviors; these nine maneuvering behaviors can be divided into longitudinal characteristics: acceleration, constant speed and deceleration; longitudinal feature classification is based on the comparison with the vehicle speed at the previous sampling moment: if the vehicle speed drops below 85% of the previous sampling moment, the current driving state is qualitatively defined as deceleration; if the speed increases to 115% of the previous sampling moment, it is defined as acceleration; if the absolute change in speed is less than 15%, it is defined as a constant speed state; for lateral movement, it is classified according to the general time frame of the vehicle's lane change behavior: if all sampling moments have the same lateral speed in the past 10 sampling cycles, it is defined as a lane change behavior, and is further divided into left lane change or right lane change according to the direction of the lateral speed.
[0022] Preferably, the S3 uses a KNN-based classifier to encode behaviors scattered in the BEV space into events.
[0023] Preferably, the specific steps in S3 are as follows:
[0024] First, the states of small targets are divided into 6 types, and the number of KNN classifications is set to 6;
[0025] Then, the priority distribution areas of the six events were obtained by using the behavioral data of the two interaction targets before and after the six events;
[0026] Finally, the distance between the real-time data and the four feature points of each event distribution is calculated to determine the event classification corresponding to the real-time data; the Euclidean distance is used to determine the area adjacent to the vehicle interaction event, while the Manhattan distance is used to search for the closest classification area to the interaction event between pedestrians and other targets;
[0027] After constructing the target behaviors and interaction events, C-AOG is used to represent the visualized flow state under the target knowledge-driven framework. The C-AOG contains the following modules:
[0028] Event trigger: This module consists of events extracted using the KNN algorithm, summarizing the target events and providing a knowledge basis for further inferring the state changes of the video stream;
[0029] And node layer: This layer consists of and nodes, which organize different events to form a knowledge set that affects the target visual features;
[0030] OR node layer: This layer further estimates mutually exclusive or independent knowledge through OR nodes to infer the visual flow state of the target; there are mainly three types of OR nodes: visible to visible, visible to occluded, and occluded to visible.
[0031] Preferably, S4 includes constructing a reasoning graph by fuzzy aggregation of behaviors and mapping the BEV perception sequence to a specific visualization state based on probabilistic reasoning.
[0032] Preferably, the specific steps in S4 are as follows:
[0033] By fuzzy aggregation of behaviors, a C-AOG reasoning graph is constructed to achieve reasoning from observation to target visual state changes; based on probabilistic reasoning, C-AOG maps the BEV perception sequence S to a specific visualization state, that is: for C-AOG, prior knowledge can be used to form an initial graph structure by considering existing events and their impact on the target visual features; or a self-learning method based on Bayesian theorem can be used to transform the construction of the C-AOG graph into a probability maximization problem. Through iterative recursion, the complete C-AOG graph structure can be gradually derived. If the derived graph matches the current example video i, then:
[0034]
[0035] in, represents the derivative graph obtained at the i-th iteration; by maximizing C-AOG can continuously train the events E formed in the perception sequence S. After constructing the complete C-AOG, the visual state of the observed target can be inferred based on the event information. The reasoning process of C-AOG is to maximize the posterior probability. The calculation process is as follows:
[0036]
[0037] Among them, ε(M) represents the similarity between temporally adjacent C-AOG graphs, and ε(I|M) represents the match between each graph and the current BEV data. The optimization problem can be solved by optimizing the reasoning process of C-AOG using the integer linear formula. For the sparse observations caused by occlusion, a disappearance suppression function is designed to compensate for the short-term disappearance of the target caused by observation occlusion or noise:
[0038]
[0039] p vs,t represents the change in the probability of existence of the occluded target. ω, τ, δ, and d are hyperparameters that can be learned from the observed data. If the occlusion leads to the destruction of the pixel coefficient, the vanishing suppression rule is introduced to limit the rapid decrease of its existence probability.
[0040] In order to prevent the sudden appearance of targets caused by false detection, a suppression function is designed to limit the appearance of abnormal targets in non-edge locations:
[0041]
[0042] p ns,t Indicates the suppressed result; p a,t Represents the unreasonable existence probability of a non-existent object. The initial suppression effect of the variable depends on the suppression factor a of the variable.
[0043] Preferably, S5 includes parameter tuning and iterative optimization.
[0044] Compared with the prior art, this application has the following beneficial effects:
[0045] 1. The present invention designs a small target description method based on causal reasoning, which combines causal reasoning with a small target detection algorithm, and uses the probability generated by causal reasoning to judge the actual state of the small target, so that the vehicle can still effectively detect the small target when it is obscured, thereby ensuring the safety of autonomous driving.
[0046] 2. The present invention designs a C-AOG reasoning graph to map the BEV perception sequence to a specific visualization state, and uses prior knowledge to form an initial graph structure by considering existing events and their impact on the target visual features, thereby realizing reasoning from observation to changes in the target visual state. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 Classification diagram of vehicle behavior on structured roads.
[0048] Figure 2 There are six types of small target state changes.
[0049] Figure 3 This is the network structure diagram of C-AOG. DETAILED DESCRIPTION
[0050] The present invention is further described in detail below in conjunction with specific embodiments.
[0051] A small target description method based on causal reasoning includes the following steps:
[0052] S1: Construct atomic knowledge nodes to represent the behavioral characteristics of small targets and surrounding targets;
[0053] The behavioral characteristics of the small target and the surrounding targets are described as:
[0054] s t ={o ti , i=1,…,N t}
[0055] o ti Represents a certain number of perception targets, N t represents the total number of objects perceived in time t, each oti Contains the type of the target c ti , and the position relationship related to the vehicle BEV mapping l ti , visibility v ti With atomic behavior a ti relationship;
[0056] The atomic knowledge node is composed of a sequence of behavioral characteristics of the small target and surrounding targets:
[0057] S={s t , t=1,…,T}
[0058] s t represents the BEV information of each time segment t;
[0059] S2: Constrain the historical data of small targets and surrounding targets into a behavior set based on long sequence sampling data;
[0060] See also Figure 1 In one embodiment, the specific process of multi-scale feature extraction is:
[0061] On structured roads, according to the vehicle's motion trajectory and speed changes, the speed and turning angle are mapped to the tangent of the curve, achieving the homogenization of the vehicle's behavior between curved and straight segments; in the behavior set, the vehicle's driving state includes 9 types of maneuvering behaviors; these nine maneuvering behaviors can be divided into longitudinal characteristics: acceleration, constant speed and deceleration; longitudinal feature classification is based on the comparison with the vehicle speed at the previous sampling moment: if the vehicle speed drops below 85% of the previous sampling moment, the current driving state is qualitatively defined as deceleration; if the speed increases to 115% of the previous sampling moment, it is defined as acceleration; if the absolute change in speed is less than 15%, it is defined as a constant speed state; for lateral movement, it is classified according to the general time frame of the vehicle's lane change behavior: if all sampling moments have the same lateral speed in the past 10 sampling cycles, it is defined as a lane change behavior, and is further divided into left lane change or right lane change according to the direction of the lateral speed.
[0062] S3: Encode behaviors scattered in the BEV space into events using a KNN-based classifier;
[0063] See also Figure 2 The specific process of using the KNN-based classifier to encode the behaviors scattered in the BEV space into events is:
[0064] First, the states of small targets are divided into 6 types, and the number of KNN classifications is set to 6;
[0065] Then, the priority distribution areas of the six events were obtained by using the behavioral data of the two interaction targets before and after the six events;
[0066] Finally, the distance between the real-time data and the four feature points of each event distribution is calculated to determine the event classification corresponding to the real-time data; the Euclidean distance is used to determine the area adjacent to the vehicle interaction event, while the Manhattan distance is used to search for the closest classification area to the interaction event between pedestrians and other targets;
[0067] After constructing the target behaviors and interaction events, C-AOG is used to represent the visualized flow state under the target knowledge-driven framework. The C-AOG contains the following modules:
[0068] Event trigger: This module consists of events extracted using the KNN algorithm, summarizing the target events and providing a knowledge basis for further inferring the state changes of the video stream;
[0069] And node layer: This layer consists of and nodes, which organize different events to form a knowledge set that affects the target visual features;
[0070] OR node layer: This layer further estimates mutually exclusive or independent knowledge through OR nodes to infer the visual flow state of the target; there are mainly three types of OR nodes: visible to visible, visible to occluded, and occluded to visible.
[0071] S4: Construct a reasoning graph by fuzzy aggregation of behaviors and map BEV perception sequences to specific visualization states based on probabilistic reasoning;
[0072] See also Figure 3 The process of fuzzy aggregation of behaviors to construct a reasoning graph and mapping the BEV perception sequence to a specific visualization state based on probabilistic reasoning is as follows:
[0073] By fuzzy aggregation of behaviors, a C-AOG reasoning graph is constructed to achieve reasoning from observation to target visual state changes; based on probabilistic reasoning, C-AOG maps the BEV perception sequence to a specific visualization state, that is: for C-AOG, prior knowledge can be used to form an initial graph structure by considering existing events and their impact on the target visual features; or a self-learning method based on Bayesian theorem can be used to transform the construction of the C-AOG graph into a probability maximization problem. Through iterative recursion, the complete C-AOG graph structure can be gradually derived. If the derived graph matches the current example video I, then:
[0074]
[0075] in, represents the derivative graph obtained at the i-th iteration; by maximizing C-AOG can continuously train the events E formed in the perception sequence S. After constructing the complete C-AOG, the visual state of the observed target can be inferred based on the event information. The reasoning process of C-AOG is to maximize the posterior probability. The calculation process is as follows:
[0076]
[0077] Among them, ε(M) represents the similarity between temporally adjacent C-AOG graphs, and ε(I|M) represents the match between each graph and the current BEV data. The optimization problem can be solved by optimizing the reasoning process of C-AOG using the integer linear formula. For the sparse observations caused by occlusion, a disappearance suppression function is designed to compensate for the short-term disappearance of the target caused by observation occlusion or noise:
[0078]
[0079] p vs,t represents the change in the probability of existence of the occluded target. ω, τ, δ, and d are hyperparameters that can be learned from the observed data. If the occlusion leads to the destruction of the pixel coefficient, the vanishing suppression rule is introduced to limit the rapid decrease of its existence probability.
[0080] In order to prevent the sudden appearance of targets caused by false detection, a suppression function is designed to limit the appearance of abnormal targets in non-edge locations:
[0081]
[0082] p ns,t Indicates the suppressed result; p a,t Represents the unreasonable existence probability of a non-existent object. The initial suppression effect of the variable depends on the suppression factor a of the variable.
[0083] S5: Evaluate and optimize the model:
[0084] The evaluation and optimization include parameter tuning and iterative optimization. Specifically, the parameter tuning includes performing parameter tuning using a parameter tuning tool or algorithm;
[0085] The iterative optimization refers to the feedback of evaluation results and parameter tuning to further improve and optimize the model.
[0086] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0087] The present invention designs a small target description method based on causal reasoning, which combines causal reasoning with a small target detection algorithm, and uses the probability generated by causal reasoning to judge the actual state of the small target, so that the vehicle can still effectively detect the small target when it is obscured, thereby ensuring the safety of autonomous driving.
[0088] The present invention designs a C-AOG reasoning graph to map the BEV perception sequence to a specific visualization state, and uses prior knowledge to form an initial graph structure by considering existing events and their impact on the target visual features, thereby realizing reasoning from observation to target visual state changes.
Claims
1. A small target description method based on causal reasoning, characterized by: The following steps are involved: S1: Construct atomic knowledge nodes through BEV perception sequences to represent the behavioral characteristics of small targets and surrounding targets in the BEV space; S2: constraining the behavior characteristics of the small target and the surrounding targets into a behavior set based on the long sequence sampling data; S3: Encode the behaviors scattered in the behavior set into events using a KNN-based classifier; S4: Construct a reasoning graph by fuzzy aggregation of behaviors and map BEV perception sequences to specific visualization states based on probabilistic reasoning; S5: Evaluate and optimize the model.
2. The small target description method based on causal reasoning according to claim 1 is characterized in that: The specific steps in S1 are as follows: The BEV perception sequence from time 1 to T is defined as: S={s t ,t=1,…,T} Among them, s t ={o ti , i=1,…,N t } represents the BEV information of each time segment t, including a certain number of perception targets O ti ; N t represents the total number of objects perceived in time t, each o ti Contains the type of the target c ti , and the position relationship related to the vehicle BEV mapping l ti , visibility v ti With atomic behavior a ti relationship.
3. The small target description method based on causal reasoning according to claim 1 is characterized in that: The specific steps in S2 are as follows: On structured roads, according to the vehicle's motion trajectory and speed changes, the speed and turning angle are mapped to the tangent of the curve, achieving uniform behavior between curved and straight segments. In the behavior set, the vehicle's driving state includes 9 types of maneuvering behaviors; these nine types of maneuvering behaviors are divided into longitudinal characteristics: acceleration, constant speed, and deceleration. The longitudinal feature classification is based on the comparison with the vehicle speed at the previous sampling moment: if the vehicle speed drops below 85% of the previous sampling moment, the current driving state is qualitatively defined as deceleration; if the speed increases to 115% of the previous sampling moment, it is defined as acceleration; if the absolute change in speed is less than 15%, it is defined as a constant speed state; for lateral movement, classification is performed according to the general time frame of the vehicle's lane changing behavior: if all sampling moments have the same directional lateral speed in the past 10 sampling cycles, it is defined as a lane changing behavior, and is further divided into left lane change or right lane change according to the direction of the lateral speed.
4. The small target description method based on causal reasoning according to claim 1 is characterized in that: The specific steps in S3 are as follows: First, the states of small targets are divided into 6 types, and the number of KNN classifications is set to 6; Then, the priority distribution areas of the six events were obtained by using the behavioral data of the two interaction targets before and after the six events; Finally, the distance between the real-time data and the four feature points of each event distribution is calculated to determine the event classification corresponding to the real-time data; the Euclidean distance is used to determine the area adjacent to the vehicle interaction event, while the Manhattan distance is used to search for the closest classification area to the interaction event between pedestrians and other targets; After constructing the target behaviors and interaction events, C-AOG is used to represent the visualized flow state under the target knowledge-driven framework. The C-AOG contains the following modules: Event triggers: consist of events extracted using the KNN algorithm; And node layer: It consists of and nodes, which organize different events to form a knowledge set that affects the target visual features; OR node layer: This layer further estimates mutually exclusive or independent knowledge through OR nodes to infer the visual flow state of the target; there are three types of OR nodes: visible to visible, visible to occluded, and occluded to visible.
5. The small target description method based on causal reasoning according to claim 1 is characterized in that: The specific steps in S4 include: By fuzzy aggregation of behaviors, a C-AOG reasoning graph is constructed to achieve reasoning from observation to target visual state changes; based on probabilistic reasoning, C-AOG maps the BEV perception sequence to a specific visualization state, that is: for C-AOG, the initial graph structure is formed by considering existing events and their impact on the target visual features using prior knowledge; or a self-learning method based on Bayesian theorem is used to transform the construction of the C-AOG graph into a probability maximization problem; through iterative recursion, the complete C-AOG graph structure is gradually derived. If the derived graph matches the current example video I, then: in, represents the derivative graph obtained at the i-th iteration; by maximizing C-AOG continuously trains the event E formed in the perception sequence S; after constructing the complete C-AOG, the visual state of the observed target is inferred based on the event information. The reasoning process of C-AOG is to maximize the posterior probability. The calculation process is as follows: Among them, ε(M) represents the similarity between temporally adjacent C-AOG graphs, ε(I|M) represents the match between each graph and the current BEV data, and the integer linear formula is used to optimize the reasoning process of C-AOG to solve the optimization problem; for sparse observations caused by occlusion, the disappearance suppression function is used to compensate for the short-term disappearance of the target caused by observation occlusion or noise: p vs,t represents the change in the probability of existence of the occluded target. ω, τ, δ, and d are hyperparameters learned from the observed data. If the occlusion leads to the destruction of the pixel coefficient, the vanishing suppression rule is introduced to limit the rapid decrease of its existence probability. In order to prevent the sudden appearance of targets caused by false detection, a suppression function is used to limit the appearance of abnormal targets in non-edge positions: p ns,t Indicates the suppressed result; p a,t Represents the unreasonable existence probability of a non-existent object. The initial suppression effect of the variable depends on the suppression factor a of the variable.
6. The small target description method based on causal reasoning according to claim 1 is characterized in that: The S5 includes parameter tuning and iterative optimization.