Method, device and equipment for determining perception risk and storage medium

Abstract

The application discloses a kind of perception risk determination method, device and equipment and storage medium, method includes: based on multiple historical trajectory sequences to initial model is trained, obtain target model;Based on multiple historical trajectory sequences to initial model is trained;To multiple historical trajectory sequences are first processed, obtain multiple soft probability and first vector;To multiple historical trajectory sequences are second processed, obtain multiple matching degrees;To the scene sequence characteristics corresponding to multiple historical trajectory sequences are third processed, obtain second vector;Based on first vector and second vector, output multiple scene feature consistency results;Based on corresponding scene feature consistency result, matching degree and soft probability, output historical risk trajectory sequence set;Multiple to-be-determined trajectory sequences are input to target model, output risk trajectory sequence set.The technical scheme of the application constructs target model with multi-tower structure, which is beneficial to the judgment of perception risk.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to a method, apparatus, device and storage medium for determining perceived risks. Background Technology

[0002] In autonomous driving scenarios, the perception system typically only estimates potential risks based on perception data. After receiving the estimated risks from the perception system, the autonomous driving system directly controls the target vehicle by combining the user's driving behavior.

[0003] However, because the perception system itself has poor accuracy in perceiving the environment around the target vehicle, if the autonomous driving system directly uses the data fed back by the perception system to judge the risk, it will often lead to a poor confidence level in the estimated risk, which will affect the safety of autonomous driving and user satisfaction. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one object of this invention is to provide a method, apparatus, device, and storage medium for determining perceived risk.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide the following technical solutions:

[0006] A method for determining perceived risk, comprising:

[0007] The initial model is trained based on multiple historical trajectory sequences to obtain a target model; the training of the initial model based on multiple historical trajectory sequences includes: training a first initial tower model to obtain a first sub-tower model; training a second initial tower model to obtain a second sub-tower model; and training a third initial tower model to obtain a third sub-tower model.

[0008] The training of the first initial tower model includes: performing a first process on multiple historical trajectory sequences to obtain multiple soft probabilities and a first vector;

[0009] The training of the second initial tower model includes: performing a second processing on multiple historical trajectory sequences to obtain multiple matching degrees;

[0010] The training of the third initial tower model includes: performing a third processing on multiple historical trajectory sequences to obtain a second vector; outputting multiple scene feature consistency results based on the first vector and the second vector; and outputting a set of historical risk trajectory sequences based on the corresponding scene feature consistency results, matching degree, and soft probability.

[0011] Multiple trajectory sequences to be determined are input into the target model, and a risk trajectory sequence set is output.

[0012] Optional, also includes:

[0013] A preset reference length is used to characterize the length of each reference road segment.

[0014] Based on multiple historical trajectory sequences, the historical target path is determined;

[0015] Based on the reference length and the reference road segment, multiple target lengths are calculated.

[0016] The historical target path is divided multiple times based on the multiple target lengths to obtain multiple target road segments.

[0017] Optionally, the first processing of the multiple historical trajectory sequences to obtain multiple soft probabilities includes:

[0018] Obtain multiple historical trajectory sequences for each target road segment;

[0019] For each target road segment, multiple sub-historical trajectory sequences are represented by vectors to obtain multiple first vectors;

[0020] Clustering is performed on multiple first vectors for each target road segment to obtain regular clustering results and the soft probabilities of the regular clustering results;

[0021] For each target road segment, multiple first vectors, regular clustering results, and the soft probabilities corresponding to the regular clustering results are output.

[0022] Optionally, the second processing of the multiple historical trajectory sequences to obtain multiple matching degrees includes:

[0023] Based on multiple historical trajectory sequences, the initial lane line and the original trajectory sequence corresponding to the initial lane line are obtained for each target road segment.

[0024] The initial lane lines of each target road segment are cut according to preset rules to obtain multiple sub-lane lines; at the same time, the original trajectory sequence of each target road segment is cut according to the preset rules to obtain multiple sub-trajectory sequences.

[0025] For each of the target road segments, each of the sub-lane lines and the corresponding sub-trajectory sequences are matched to obtain the matching degree and the first sub-loss function value.

[0026] Optionally, the third processing of the scene sequence features corresponding to multiple historical trajectory sequences to obtain the second vector includes:

[0027] Based on the scene sequence features, sub-scene sequence features for each target road segment are obtained;

[0028] Based on the feature labels, feature extraction and feature fusion are performed on the sub-scene sequence features of each target road segment to obtain the sub-scene feature sequence;

[0029] The feature sequence of the sub-scene is represented by a vector to obtain the second vector for each target road segment.

[0030] Optionally, based on the second vector, the output of multiple scene feature consistency results includes:

[0031] The second vector of each target road segment is matched with each corresponding first vector, and the scene feature consistency result and the second sub-loss function value of each first vector are output.

[0032] Specifically, the loss function value of the initial model is calculated based on the second sub-loss function and the value of the first sub-loss function.

[0033] Optionally, the step of outputting a historical risk trajectory sequence set based on the corresponding scene feature consistency results, matching degree, and soft probability includes:

[0034] Obtain the scene feature consistency result, matching degree, and soft probability for each of the sub-historical trajectory sequences of each target road segment;

[0035] The scenario feature consistency result, matching degree, and soft probability of each sub-historical trajectory sequence for each target road segment are calculated to obtain the risk probability of each sub-historical trajectory sequence for each target road segment.

[0036] The risk probability is compared with a risk probability threshold. If the risk probability is greater than the risk probability threshold, the sub-historical trajectory sequence corresponding to the risk probability is determined as a historical risk trajectory sequence.

[0037] Embodiments of the present invention also provide a risk perception determination device, comprising:

[0038] The first training module is used to train an initial model based on multiple historical trajectory sequences to obtain a target model; the training of the initial model based on the multiple historical trajectory sequences includes: training a first initial tower model to obtain a first sub-tower model; training a second initial tower model to obtain a second sub-tower model; and training a third initial tower model to obtain a third sub-tower model.

[0039] The second training module is used to train the first initial tower model, including: performing a first process on multiple historical trajectory sequences to obtain multiple soft probabilities;

[0040] The third training module is used to train the second initial tower model, including: performing a second processing on multiple historical trajectory sequences to obtain multiple matching degrees;

[0041] The fourth training module, used for training the third initial tower model, includes: performing a third processing on the scene sequence features corresponding to multiple historical trajectory sequences to obtain a second vector; outputting multiple scene feature consistency results based on the first vector and the second vector; and outputting a set of historical risk trajectory sequences based on the corresponding scene feature consistency results, matching degree, and soft probability.

[0042] The output module is used to input multiple trajectory sequences to be determined into the target model and output a risk trajectory sequence set.

[0043] Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method described above.

[0044] Embodiments of the present invention also provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls the device containing the computer-readable storage medium to perform the method described above.

[0045] The embodiments of the present invention have the following technical effects:

[0046] The above-mentioned technical solution of the present invention 1) constructs an initial model based on the model building unit, then obtains multiple historical trajectory sequences based on the perception unit, map unit and scene recognition unit, and trains the initial model based on the multiple historical trajectory sequences to obtain a target model; after inputting multiple trajectory sequences to be determined into the target model, a risk trajectory sequence set is output, thereby realizing the prediction of risk trajectory sequences based on the target model and real-time data collected by the perception unit, etc., and improving the safety of autonomous driving.

[0047] 2) A target model with a multi-tower structure was constructed. Based on the first sub-tower model, all first vectors of historical trajectory sequences for each target road segment are output, along with the regular clustering results and corresponding soft probabilities obtained from all first vectors. Based on the second sub-tower model, the matching degree of each undetermined sub-trajectory sequence for each target road segment is output. Based on the third sub-tower model, the sub-scene sequence features of the scene sequence are output for each target road segment. The sub-scene sequence features of each target road segment are represented by vectors to obtain the second vector. Based on the first and second vectors, the scene feature consistency result of each undetermined sub-trajectory sequence for each target road segment is determined. Based on the scene feature consistency result, matching degree, and corresponding soft probability of the target road segment for each undetermined sub-trajectory sequence, the risk probability is determined. This achieves multi-level processing of multiple undetermined trajectory sequences, obtaining multi-level data, and judging whether multiple undetermined trajectory sequences are risky trajectory sequences from multiple dimensions, thus improving the accuracy of risk perception.

[0048] 3) Information was extracted from the determined trajectory sequence at different levels and stages, which expanded the information capacity of the target model, further improved the accuracy of risk perception judgment, and enhanced the safety of autonomous driving and user satisfaction.

[0049] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the structure of a risk perception and determination system provided in an embodiment of the present invention;

[0051] Figure 2 This is a flowchart illustrating a method for determining perceived risk provided by an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the structure of a risk perception and determination device provided in an embodiment of the present invention. Detailed Implementation

[0053] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0054] To facilitate understanding of the embodiments by those skilled in the art, some terms are explained below:

[0055] (1) Transformer: A deep learning model based on self-attention mechanism.

[0056] like Figure 1 As shown, an embodiment of the present invention provides a risk perception determination system, comprising:

[0057] Perception unit, map unit, scene recognition unit, and model building unit;

[0058] The model building unit communicates with the perception unit, map unit, and scene recognition unit via a network.

[0059] In embodiments of the present invention, historical operating status data of a target vehicle is obtained, and multiple historical trajectory sequences are obtained based on the historical operating status data of the target vehicle.

[0060] Specifically, the perception unit sends the perception data of obstacles to the model building unit via the network;

[0061] The map unit sends map data to the model building unit via the network;

[0062] The scene recognition unit identifies the scene labels carried in each frame of the image set obtained by the perception unit, and sends the data carrying the scene labels to the model building unit via the network.

[0063] The model building unit first constructs an initial model, then trains the initial model multiple times based on the obtained historical trajectory sequences to obtain the target model;

[0064] Input multiple determined trajectory sequences into the target model and output a risk trajectory sequence set.

[0065] In an embodiment of the present invention, an initial model is constructed based on a model building unit. Then, multiple historical trajectory sequences are obtained based on a perception unit, a map unit, and a scene recognition unit. The initial model is trained based on these multiple historical trajectory sequences to obtain a target model. After inputting the data to be determined into the target model, a risk trajectory sequence is output. This enables the prediction of risk trajectory sequences based on the target model using real-time data collected by the perception unit and other components, thereby improving the safety of autonomous driving.

[0066] like Figure 2 As shown, embodiments of the present invention also provide a method for determining perceived risk, applied to the above-described system, comprising:

[0067] Step S1: Train the initial model based on multiple historical trajectory sequences to obtain the target model; the training of the initial model based on multiple historical trajectory sequences includes: training the first initial tower model to obtain the first sub-tower model; training the second initial tower model to obtain the second sub-tower model; and training the third initial tower model to obtain the third sub-tower model.

[0068] In an embodiment of the present invention, an initial model comprising a three-layer tower structure is constructed, and each layer of the tower structure is trained separately; wherein, the initial model comprises a first initial tower model, a second initial tower model, and a third initial tower model.

[0069] For multiple historical trajectory sequences, the historical driving data of the target vehicle and the historical driving data of other vehicles are included, specifically the coordinates of obstacles (other vehicles) returned by the perception unit, the coordinates of the target vehicle, and the perception image set obtained based on the perception unit; among them, the obstacle trajectory sequence is obtained based on the obstacle coordinates; and the target vehicle trajectory sequence is obtained based on the target vehicle coordinates.

[0070] In embodiments of the present invention, samples are constructed using random negative sampling. Generally, when a target vehicle is driving normally, the trajectory sequence of the target vehicle is considered to match the lane line or road edge, which is a positive sample. However, in embodiments of the present invention, multiple negative samples are randomly constructed. Specifically, based on multiple historical trajectory sequences, multiple sets of matching trajectory sequences and lane lines or road edges can be obtained, for example: (first trajectory sequence, first lane line), (second trajectory sequence, second lane line), (third trajectory sequence, third lane line)...(Nth trajectory sequence, Nth lane line).

[0071] Then, the multiple sets of matching data are shuffled and regrouped randomly. For example, we can obtain: (first trajectory sequence, second lane line), (second trajectory sequence, sixth lane line), (third trajectory sequence, second lane line)...(Nth trajectory sequence, tenth lane line).

[0072] Therefore, based on the above method, multiple sets of negative samples are randomly constructed and used by subsequent algorithms in the embodiments of the present invention;

[0073] In an embodiment of the present invention, the first initial tower model, the second initial tower model, and the third initial tower model are trained based on multiple sets of negative samples obtained from multiple historical trajectory sequences, thereby obtaining the first sub-tower model, the second sub-tower model, and the third sub-tower model in sequence.

[0074] Step S2: Training the first initial tower model includes: performing a first processing on multiple historical trajectory sequences to obtain multiple soft probabilities and a first vector;

[0075] In embodiments of the present invention, during the training of the initial model, it is necessary to divide the historical target path into multiple road segments, and process multiple historical trajectory sequences using road segments as basic units. However, the length of each road segment cannot be too long or too short, otherwise it will affect the accuracy of the mined risk trajectory sequences. To solve this technical problem, embodiments of the present invention provide the following technical solution:

[0076] A preset reference length is used to characterize the length of each reference road segment.

[0077] Based on multiple historical trajectory sequences, the historical target path is determined;

[0078] Based on the reference length and the reference road segment, multiple target lengths are calculated.

[0079] The historical target path is divided multiple times based on the multiple target lengths to obtain multiple target road segments.

[0080] In one embodiment of the present invention, multiple target lengths are calculated based on the reference length and the reference road segment, including:

[0081] Calculate the complexity of the first reference road segment; wherein, the calculation of the complexity of the first reference road segment includes:

[0082] Multiple preset parameters;

[0083] The priority and initial value of each parameter are preset;

[0084] Determine the parameters of the first reference road segment;

[0085] The actual value of each parameter is determined sequentially according to the priority; based on the initial value and the actual value of each parameter, the first complexity of the first reference road segment is calculated.

[0086] Based on the first complexity and the baseline length, the first target length is calculated, and the historical target path is first segmented based on the first target length to obtain the first target road segment.

[0087] Following this process, based on the above steps, the second complexity and the second target length can be calculated. Based on the second target length, the historical target path is divided a second time to obtain the second target road segment. Specifically, after the first division of the historical target path, the second reference road segment is determined based on the division position and the reference length, and the second complexity of the second reference road segment is calculated.

[0088] ...

[0089] Based on the length of the Nth target, the historical target path is divided into Nth segments to obtain the Nth target road segment; where N≥1 and N is an integer.

[0090] It should be noted that the first reference road segment, the second reference road segment, and the Nth reference road segment have the same length, which are all reference lengths. The only difference is that the starting and ending positions are different for the historical target path.

[0091] The parameters may include curvature, road width, number of lanes, and tunnels, etc.

[0092] It also presets the priority and initial value of each parameter;

[0093] For example, taking the above four parameters as an example:

[0094] Priority: Curvature > Road width > Number of lanes > Tunnel.

[0095] The initial value for curvature is 200; the initial value for road width is 100; the initial value for the number of lanes is 100; the initial value for tunnels is 100.

[0096] The baseline length of the benchmark road segment can be preset to 100 meters;

[0097] Calculate the complexity of each benchmark road segment and determine the target length of each benchmark road segment based on the complexity;

[0098] For example, taking the first benchmark road segment as an example, based on the benchmark length and historical target path, the start and end positions of the first benchmark road segment are determined, and then the parameters of the first benchmark road segment are determined according to the road conditions and other information of the sub-historical target path between the start and end positions.

[0099] Specifically, first, it is determined whether the first reference road segment has curvature. If it does, the actual value of the curvature of the first reference road segment is calculated; if it does, the actual value of the curvature is calculated to be 150.

[0100] The actual value of the road width of the first benchmark road segment is 50.

[0101] The actual number of lane lines for the first benchmark road segment is 50;

[0102] Determine whether the first reference road segment is a tunnel. If it is, calculate the actual value of the tunnel. If not, the actual value of the tunnel parameter is 0.

[0103] Based on the above judgments and calculations, the complexity of the first benchmark road segment can be calculated:

[0104] First complexity = (150 + 50 + 50) / (200 + 100 + 100 + 100) = 0.5

[0105] In the embodiments of the present invention, 0 < complexity < 1; the larger the value of complexity, the more complex the corresponding baseline road segment.

[0106] Then, based on the following calculation formula, the length of the first target is calculated:

[0107] The first target length = baseline length / (1 / complexity) = 200. That is, starting from the starting position of the historical target path, the first segment is obtained by making the first division at a position 200 meters away from the starting position.

[0108] ...

[0109] By repeating the above steps, we can calculate that the length of the second target is 150 meters, the length of the third target is 200 meters, the length of the fourth target is 120 meters, and so on. After dividing the historical target path multiple times, we can obtain multiple target road segments of 200 meters, 150 meters, 200 meters, 120 meters, and so on.

[0110] Therefore, based on the above steps, multiple target lengths can be calculated. Based on all target lengths, the historical target path can be divided multiple times to obtain multiple target road segments.

[0111] In an embodiment of the present invention, after obtaining multiple target road segments, multiple historical trajectory sequences are processed based on the target road segments as the basic unit to obtain the historical trajectory sequence of all obstacles in each target road segment.

[0112] Specifically, the first processing of the multiple historical trajectory sequences to obtain multiple soft probabilities includes:

[0113] Obtain multiple historical trajectory sequences for each target road segment;

[0114] For each target road segment, multiple sub-historical trajectory sequences are represented by vectors to obtain multiple first vectors;

[0115] Clustering is performed on multiple first vectors for each target road segment to obtain regular clustering results and the soft probabilities of the regular clustering results;

[0116] For each target road segment, multiple first vectors, regular clustering results, and the soft probabilities corresponding to the regular clustering results are output.

[0117] In an embodiment of the present invention, if multiple obstacles are detected in a target road segment, a sub-historical trajectory sequence of each obstacle in the target road segment is obtained; for example, if 30 obstacles are detected in a first target road segment, then the first target road segment includes 30 sub-historical trajectory sequences.

[0118] Encode each of the 30 sub-historical trajectory sequences (e.g., use a transformer to represent each sub-historical trajectory sequence as a vector), and then input all the first vectors into the clustering model. The clustering model is based on the consistency of each first vector and its corresponding lane line or road edge, such as whether it is turning or going straight at a certain moment.

[0119] If, at the same time, the first vector and its corresponding lane line or road edge are both turning or going straight, then they are considered consistent; if, at the same time, the first vector and its corresponding lane line or road edge are either turning or going straight, then they are considered inconsistent, and so on. The clustering model classifies all the input first vectors based on this judgment method and outputs: regular clustering results (i.e., consistent clustering results, e.g., 10 results), special clustering results (e.g., 10 results), and suspicious clustering results (e.g., 10 results).

[0120] Similarly, each target road segment can obtain a regular clustering result, which serves as a set of normal or safe trajectory sequences within that target road segment. Each regular clustering result corresponds to a soft probability.

[0121] Furthermore, the consistency score of each sub-historical trajectory sequence of each target road segment can be determined based on the soft probability corresponding to each regular clustering result. A preset calculation rule can be used to calculate the consistency score of each sub-historical trajectory sequence based on the soft probability corresponding to each regular clustering result and the preset calculation rule. The embodiments of the present invention do not specifically limit the preset calculation rule.

[0122] Step S3: Training the second initial tower model includes: performing a second processing on multiple historical trajectory sequences to obtain multiple matching degrees;

[0123] Specifically, the second processing of multiple historical trajectory sequences to obtain multiple matching degrees includes:

[0124] Based on multiple historical trajectory sequences, the initial lane line and the original trajectory sequence corresponding to the initial lane line are obtained for each target road segment.

[0125] The initial lane lines of each target road segment are cut according to preset rules to obtain multiple sub-lane lines; at the same time, the original trajectory sequence of each target road segment is cut according to the preset rules to obtain multiple sub-trajectory sequences.

[0126] For each of the target road segments, each of the sub-lane lines and the corresponding sub-trajectory sequences are matched to obtain the matching degree and the first sub-loss function value.

[0127] In embodiments of the present invention, based on data returned by map units, lane coordinates and trajectory coordinates within each target road segment are obtained; based on the lane coordinates, an initial lane line is formed; based on the trajectory coordinates, an original trajectory sequence is formed; and multiple curvatures of the initial lane line are calculated; wherein, the curvatures can be directly obtained based on the data returned by map units.

[0128] Furthermore, the preset rule is to divide the initial lane line according to the magnitude of curvature; for example, when the curvature of the initial lane line at a certain time or position reaches the curvature threshold (based on automatic generation or manual input), the initial lane line is divided at this point; and so on, to complete the division of the entire initial lane line and obtain multiple sub-lane lines.

[0129] The original trajectory sequence within the initial lane line is divided in the same way according to the preset rules to obtain multiple sub-trajectory sequences;

[0130] Among them, multiple sub-lane lines and multiple sub-trajectory sequences are encoded and represented based on transformer;

[0131] Then, each sub-lane line and the corresponding sub-trajectory sequence of each target road segment are matched to obtain the matching degree and the first sub-loss function value.

[0132] The matching degree can be calculated based on cosine similarity.

[0133] The value of the first sub-loss function can be obtained directly from the output of the second initial tower model.

[0134] Step S4: Training the third initial tower model includes: performing third processing on the scene sequence features corresponding to multiple historical trajectory sequences to obtain a second vector; outputting multiple scene feature consistency results based on the first vector and the second vector; and outputting a historical risk trajectory sequence set based on the corresponding scene feature consistency results, matching degree, and soft probability.

[0135] In the embodiments of the present invention, information is extracted at multiple levels and different stages from the determined trajectory sequence, which expands the information capacity of the target model, further improves the accuracy of risk perception judgment, and enhances the safety of autonomous driving and user satisfaction.

[0136] Specifically, the third processing of the scene sequence features corresponding to multiple historical trajectory sequences to obtain the second vector includes:

[0137] Based on the scene sequence features, sub-scene sequence features for each target road segment are obtained;

[0138] Based on the feature labels, feature extraction and feature fusion are performed on the sub-scene sequence features of each target road segment to obtain the sub-scene feature sequence;

[0139] The feature sequence of the sub-scene is represented by a vector to obtain the second vector for each target road segment.

[0140] In embodiments of the present invention, feature labels may include scene labels and behavior labels, etc.; wherein, scene labels may include pre-labeled weather, congestion level, behavior actions (overtaking, lane changing, etc.), etc., and behavior labels may include safety or danger; for example, based on these scene labels, feature extraction is performed on the sub-scene sequence features of each target road segment, and then feature fusion is performed to obtain a sub-scene feature sequence; then, the sub-scene feature sequence can be vectorized based on a deep learning algorithm, and a second vector is output.

[0141] Furthermore, based on the second vector, the output of multiple scene feature consistency results includes:

[0142] The second vector of each target road segment is matched with each corresponding first vector, and the scene feature consistency result and the second sub-loss function value of each first vector are output.

[0143] Specifically, the loss function value of the initial model is calculated based on the second sub-loss function and the value of the first sub-loss function.

[0144] For example: There are 30 first vectors in total. Each first vector is matched with a second vector to obtain 30 scene feature consistency results.

[0145] The scene feature consistency result of the first first vector is 0.8; the scene feature consistency result of the second first vector is 0.9... The scene feature consistency result of each sub-historical trajectory sequence can be calculated.

[0146] Furthermore, in order to improve the robustness of the target model, embodiments of the present invention perform multiple iterations of training on the initial model;

[0147] In this embodiment of the invention, the robustness of the target model is determined based on a loss function. Specifically, the loss function value can be calculated based on the following formula:

[0148] Loss function value = First sub-loss function value + Soft probability * Second sub-loss function value;

[0149] That is, in the embodiments of the present invention, based on the relevant data obtained during the initial model training process, the loss function value of the current model can be calculated, and the current model can be determined as the target model based on the loss function value. The second sub-loss function value can be obtained directly by the output of the third initial tower model. For example, the loss function value can be minimized by setting a threshold for the loss function or setting a descent gradient threshold for the loss function, so as to improve the accuracy of the target model.

[0150] Furthermore, the step of outputting a historical risk trajectory sequence set based on the corresponding scene feature consistency results, matching degree, and soft probability includes:

[0151] Obtain the scene feature consistency result, matching degree, and soft probability for each of the sub-historical trajectory sequences of each target road segment;

[0152] The scenario feature consistency result, matching degree, and soft probability of each sub-historical trajectory sequence for each target road segment are calculated to obtain the risk probability of each sub-historical trajectory sequence for each target road segment.

[0153] The risk probability is compared with a risk probability threshold. If the risk probability is greater than the risk probability threshold, the sub-historical trajectory sequence corresponding to the risk probability is determined as a historical risk trajectory sequence.

[0154] Specifically, the risk probability of each sub-historical trajectory sequence for each target road segment can be calculated based on the following formula:

[0155] Risk probability = (matching degree + soft probability * scene feature consistency result) / 2

[0156] For example, if the matching degree of the first vector corresponding to a certain sub-historical trajectory sequence is 0.6, the soft probability is 0.9, and the scene feature consistency result is 0.8, then the risk probability corresponding to that sub-historical trajectory sequence is 0.66.

[0157] In an embodiment of the present invention, during the training process, the initial model outputs the risk prediction results of each sub-historical trajectory sequence in the form of classification results, for example: 0 and 1, where 1 represents risk, that is, the probability corresponding to 1 is the risk probability;

[0158] Correspondingly, the risk prediction results or model output results corresponding to the sub-historical trajectory sequences in the above embodiments are 0 (0.34) and 1 (0.66).

[0159] After obtaining the classification results, the risk probability and the risk probability threshold are compared to determine whether the sub-historical trajectory sequence is a historical risk trajectory sequence.

[0160] In an optional embodiment of the present invention, images corresponding to the historical risk trajectory sequence are extracted, and the extracted images are connected based on time sequence to obtain a key frame sequence for risk perception.

[0161] In embodiments of the present invention, a target model with a multi-tower structure is constructed to determine whether the trajectory sequence obtained based on the perception unit is a risk trajectory sequence from multiple dimensions, thereby improving the accuracy of the perceived risk judgment. In addition, embodiments of the present invention extract information from the data to be determined at different levels and stages, expanding the information capacity of the target model, further improving the accuracy of the perceived risk judgment, and enhancing the safety of autonomous driving and user satisfaction.

[0162] Step S5: Input multiple trajectory sequences to be determined into the target model and output a risk trajectory sequence set.

[0163] In embodiments of the present invention, a risk probability threshold of 0.8 can be preset. Then, when the target model classifies multiple first vectors based on multiple undetermined sub-trajectory sequences corresponding to a certain target road segment, the classification results are as follows:

[0164] First vector: 0 (0.1), 1 (0.9);

[0165] Second first vector: 0 (0.19), 1 (0.81);

[0166] Third first vector: 0 (0.33), 1 (0.67);

[0167] Fourth first vector: 0 (0.22), 1 (0.78);

[0168] By comparing the risk probabilities of the first vectors corresponding to the multiple undetermined sub-trajectory sequences with 0.8, it can be seen that the risk probabilities of the first and second first vectors are both greater than the risk probability threshold. Therefore, the undetermined sub-trajectory sequences corresponding to the first and second first vectors are both risky trajectory sequences.

[0169] In addition, by extracting the images corresponding to each risk trajectory sequence and connecting the extracted images based on time sequence, a key frame sequence for risk perception can be obtained.

[0170] In embodiments of this invention, a target model with a multi-tower structure is constructed. Based on a first sub-tower model, all first vectors of historical trajectory sequences for each target road segment are output, along with the regular clustering results obtained from all the first vectors and the corresponding soft probabilities. Based on a second sub-tower model, the matching degree of each undetermined sub-trajectory sequence for each target road segment is output. Based on a third sub-tower model, the sub-scene sequence features of the scene sequence features for each target road segment are output. The sub-scene sequence features of each target road segment are represented by vectors to obtain a second vector. Based on the first and second vectors, the scene feature consistency result of each undetermined sub-trajectory sequence for each target road segment is determined. Based on the scene feature consistency result, matching degree, and corresponding soft probability of the target road segment for each undetermined sub-trajectory sequence, the risk probability is determined. This achieves multi-level processing of multiple undetermined trajectory sequences, obtaining multi-level data, and judging whether multiple undetermined trajectory sequences are risky trajectory sequences from multiple dimensions, thus improving the accuracy of risk perception.

[0171] like Figure 3 As shown, embodiments of the present invention also provide a risk perception determination device 300, comprising:

[0172] The first training module 301 is used to train an initial model based on multiple historical trajectory sequences to obtain a target model; the training of the initial model based on the multiple historical trajectory sequences includes: training a first initial tower model to obtain a first sub-tower model; training a second initial tower model to obtain a second sub-tower model; and training a third initial tower model to obtain a third sub-tower model.

[0173] The second training module 302 is used to train the first initial tower model, including: performing a first processing on multiple historical trajectory sequences to obtain multiple soft probabilities and a first vector;

[0174] The third training module 303 is used to train the second initial tower model, including: performing a second processing on multiple historical trajectory sequences to obtain multiple matching degrees;

[0175] The fourth training module 304 is used to train the third initial tower model, including: performing a third processing on the scene sequence features corresponding to multiple historical trajectory sequences to obtain a second vector; outputting multiple scene feature consistency results based on the first vector and the second vector; and outputting a set of historical risk trajectory sequences based on the corresponding scene feature consistency results, matching degree, and soft probability.

[0176] The output module 305 is used to input multiple trajectory sequences to be determined into the target model and output a risk trajectory sequence set.

[0177] Optional, also includes:

[0178] A preset reference length is used to characterize the length of each reference road segment.

[0179] Based on multiple historical trajectory sequences, the historical target path is determined;

[0180] Based on the reference length and the reference road segment, multiple target lengths are calculated.

[0181] The historical target path is divided multiple times based on the multiple target lengths to obtain multiple target road segments.

[0182] Optionally, the first processing of the multiple historical trajectory sequences to obtain multiple soft probabilities includes:

[0183] Obtain multiple historical trajectory sequences for each target road segment;

[0184] For each target road segment, multiple sub-historical trajectory sequences are represented by vectors to obtain multiple first vectors;

[0185] Clustering is performed on multiple first vectors for each target road segment to obtain regular clustering results and the soft probabilities of the regular clustering results;

[0186] For each target road segment, multiple first vectors, regular clustering results, and the soft probabilities corresponding to the regular clustering results are output.

[0187] Optionally, the second processing of the multiple historical trajectory sequences to obtain multiple matching degrees includes:

[0188] Based on multiple historical trajectory sequences, the initial lane line and the original trajectory sequence corresponding to the initial lane line are obtained for each target road segment.

[0189] The initial lane lines of each target road segment are cut according to preset rules to obtain multiple sub-lane lines; at the same time, the original trajectory sequence of each target road segment is cut according to the preset rules to obtain multiple sub-trajectory sequences.

[0190] For each of the target road segments, each of the sub-lane lines and the corresponding sub-trajectory sequences are matched to obtain the matching degree and the first sub-loss function value.

[0191] Optionally, the third processing of the scene sequence features corresponding to multiple historical trajectory sequences to obtain the second vector includes:

[0192] Based on the scene sequence features, sub-scene sequence features for each target road segment are obtained;

[0193] Based on the feature labels, feature extraction and feature fusion are performed on the sub-scene sequence features of each target road segment to obtain the sub-scene feature sequence;

[0194] The feature sequence of the sub-scene is represented by a vector to obtain the second vector for each target road segment.

[0195] Optionally, based on the second vector, the output of multiple scene feature consistency results includes:

[0196] The second vector of each target road segment is matched with each corresponding first vector, and the scene feature consistency result and the second sub-loss function value of each first vector are output.

[0197] Specifically, the loss function value of the initial model is calculated based on the second sub-loss function and the value of the first sub-loss function.

[0198] Optionally, the step of outputting a historical risk trajectory sequence set based on the corresponding scene feature consistency results, matching degree, and soft probability includes:

[0199] Obtain the scene feature consistency result, matching degree, and soft probability for each of the sub-historical trajectory sequences of each target road segment;

[0200] The scenario feature consistency result, matching degree, and soft probability of each sub-historical trajectory sequence for each target road segment are calculated to obtain the risk probability of each sub-historical trajectory sequence for each target road segment.

[0201] The risk probability is compared with a risk probability threshold. If the risk probability is greater than the risk probability threshold, the sub-historical trajectory sequence corresponding to the risk probability is determined as a historical risk trajectory sequence.

[0202] Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method described above.

[0203] Embodiments of the present invention also provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls the device containing the computer-readable storage medium to perform the method described above.

[0204] Furthermore, other configurations and functions of the apparatus in the embodiments of the present invention are known to those skilled in the art, and will not be described in detail here to reduce redundancy.

[0205] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0206] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0207] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0208] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0209] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0210] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0211] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0212] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for determining perceived risk, characterized in that, include: The initial model is trained based on multiple historical trajectory sequences to obtain the target model; The step of training the initial model based on multiple historical trajectory sequences includes: training a first initial tower model to obtain a first sub-tower model; training a second initial tower model to obtain a second sub-tower model; and training a third initial tower model to obtain a third sub-tower model. The training of the first initial tower model includes: performing a first process on multiple historical trajectory sequences to obtain multiple soft probabilities and a first vector; The training of the second initial tower model includes: performing a second processing on multiple historical trajectory sequences to obtain multiple matching degrees; The training of the third initial tower model includes: performing a third processing on the scene sequence features corresponding to multiple historical trajectory sequences to obtain a second vector; outputting multiple scene feature consistency results based on the first vector and the second vector; and outputting a set of historical risk trajectory sequences based on the corresponding scene feature consistency results, matching degree, and soft probability. Multiple trajectory sequences to be determined are input into the target model, and a risk trajectory sequence set is output. Also includes: A preset reference length is used to characterize the length of each reference road segment. Based on multiple historical trajectory sequences, the historical target path is determined; Based on the reference length and the reference road segment, multiple target lengths are calculated. The historical target path is segmented multiple times based on multiple target lengths to obtain multiple target road segments; The first processing of multiple historical trajectory sequences to obtain multiple soft probabilities includes: Obtain multiple historical trajectory sequences for each target road segment; For each target road segment, multiple sub-historical trajectory sequences are represented by vectors to obtain multiple first vectors; Clustering is performed on multiple first vectors for each target road segment to obtain regular clustering results and the soft probabilities of the regular clustering results; For each target road segment, output multiple first vectors, regular clustering results, and the soft probabilities corresponding to the regular clustering results; The second processing of multiple historical trajectory sequences to obtain multiple matching degrees includes: Based on multiple historical trajectory sequences, the initial lane line and the original trajectory sequence corresponding to the initial lane line are obtained for each target road segment. The initial lane lines of each target road segment are cut according to preset rules to obtain multiple sub-lane lines; at the same time, the original trajectory sequence of each target road segment is cut according to the preset rules to obtain multiple sub-trajectory sequences. For each of the target road segments, each of the sub-lane lines and the corresponding sub-trajectory sequences are matched to obtain the matching degree and the first sub-loss function value.

2. The method according to claim 1, characterized in that, The third processing of the scene sequence features corresponding to multiple historical trajectory sequences to obtain the second vector includes: Based on the scene sequence features, sub-scene sequence features for each target road segment are obtained; Based on the feature labels, feature extraction and feature fusion are performed on the sub-scene sequence features of each target road segment to obtain the sub-scene feature sequence; The feature sequence of the sub-scene is represented by a vector to obtain the second vector for each target road segment.

3. The method according to claim 2, characterized in that, Based on the second vector, the output of multiple scene feature consistency results includes: The second vector of each target road segment is matched with each corresponding first vector, and the scene feature consistency result and the second sub-loss function value of each first vector are output. Specifically, the loss function value of the initial model is calculated based on the second sub-loss function and the value of the first sub-loss function.

4. The method according to claim 3, characterized in that, Based on the corresponding scene feature consistency results, matching degree, and soft probability, a historical risk trajectory sequence set is output, including: Obtain the scene feature consistency result, matching degree, and soft probability for each of the sub-historical trajectory sequences of each target road segment; The scenario feature consistency result, matching degree, and soft probability of each sub-historical trajectory sequence for each target road segment are calculated to obtain the risk probability of each sub-historical trajectory sequence for each target road segment. The risk probability is compared with a risk probability threshold. If the risk probability is greater than the risk probability threshold, the sub-historical trajectory sequence corresponding to the risk probability is determined as a historical risk trajectory sequence.

5. A device for determining perceived risk, characterized in that, include: The first training module is used to train the initial model based on multiple historical trajectory sequences to obtain the target model; The step of training the initial model based on multiple historical trajectory sequences includes: training a first initial tower model to obtain a first sub-tower model; training a second initial tower model to obtain a second sub-tower model; and training a third initial tower model to obtain a third sub-tower model. The second training module is used to train the first initial tower model, including: performing a first processing on multiple historical trajectory sequences to obtain multiple soft probabilities and a first vector; The third training module is used to train the second initial tower model, including: performing a second processing on multiple historical trajectory sequences to obtain multiple matching degrees; The fourth training module, used for training the third initial tower model, includes: performing a third processing on the scene sequence features corresponding to multiple historical trajectory sequences to obtain a second vector; outputting multiple scene feature consistency results based on the first vector and the second vector; and outputting a set of historical risk trajectory sequences based on the corresponding scene feature consistency results, matching degree, and soft probability. The output module is used to input multiple trajectory sequences to be determined into the target model and output a risk trajectory sequence set; It also includes: a preset baseline length; the baseline length is used to characterize the length of each baseline road segment; a historical target path is determined based on multiple historical trajectory sequences; multiple target lengths are calculated based on the baseline length and the baseline road segments; the historical target path is divided multiple times based on the multiple target lengths to obtain multiple target road segments; The second training module is specifically used for: acquiring multiple historical trajectory sequences for each target road segment; representing the multiple sub-historical trajectory sequences of each target road segment into vectors to obtain multiple first vectors; clustering the multiple first vectors of each target road segment to obtain a regular clustering result and the soft probability of the regular clustering result; and outputting the multiple first vectors, the regular clustering result, and the soft probability corresponding to the regular clustering result for each target road segment. The second training module is further configured to: obtain the initial lane line of each target road segment and the original trajectory sequence corresponding to the initial lane line based on multiple historical trajectory sequences; cut the initial lane line of each target road segment according to a preset rule to obtain multiple sub-lane lines; simultaneously, cut the original trajectory sequence of each target road segment according to the preset rule to obtain multiple sub-trajectory sequences; and match each sub-lane line and the corresponding sub-trajectory sequence of each target road segment to obtain a matching degree and a first sub-loss function value.

6. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 4.

Images