Driving environment detection method, device, equipment and storage medium

Abstract

The present invention relates to the field of automatic driving control, and discloses a method, device, equipment and storage medium for detecting a driving environment. The environmental data at the current moment and the first detection result at a preset moment in the past are obtained, and the environmental data and the first detection result are preprocessed to obtain a data set to be detected; based on a preset deep learning network, a direct perception analysis of the driving environment is performed on the data set to be detected to obtain approximate perception data; and the approximate perception data is used to generate a second detection result of the driving environment at the current moment. The present invention reduces the delay of the unmanned vehicle in detecting obstacles in the driving environment, quickly detects obstacles that suddenly appear, and improves the driving safety of the unmanned vehicle.

Description

Technical Field

[0001] The present invention relates to the field of automatic driving control, and in particular to a driving environment detection method, device, equipment and storage medium. Background Art

[0002] In the field of vehicle autonomous driving, autonomous driving algorithms can be divided into three parts: scene recognition, route planning and vehicle control. After obtaining the environmental data collected by the sensor, the perception algorithm can be used to identify obstacles during driving, perceive the location, size, category, direction, motion trajectory, speed, etc. of the obstacles, obtain the results of scene recognition, and then input them into the planning module for route planning and vehicle control.

[0003] In the scene recognition part, the conventional non-direct perception framework is usually used for recognition, such as Figure 1 As shown, in the conventional non-direct connection perception framework, after the environmental data is collected by the sensors on the autonomous driving vehicle, obstacle perception such as obstacle existence detection, category classification, shape estimation, tracking, and quick-freeze acceleration estimation are required as the result of scene recognition for the route planning of the autonomous driving vehicle. In order to ensure that the unmanned driving can safely and stably detect surrounding obstacles, the existing commonly used perception algorithms often require a computing time of about 100ms when the on-board computing power is limited. In some cases, a long obstacle detection delay will result in the unmanned vehicle being unable to respond to a person who suddenly appears in a timely manner, thereby posing a potential safety risk. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem of high delay in obstacle detection in existing autonomous driving technology.

[0005] A first aspect of the present invention provides a method for detecting a driving environment, comprising: acquiring environmental data at a current moment and a first detection result at a preset moment in the past, and preprocessing the environmental data and the first detection result to obtain a data set to be detected; based on a preset deep learning network, performing a direct perception analysis of the driving environment on the data set to be detected to obtain approximate perception data; and using the approximate perception data to generate a second detection result of the driving environment at the current moment.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, after obtaining the environmental data at the current moment and the first detection result at a past preset moment, it also includes: performing segmentation detection processing on the environmental data at the current moment to obtain obstacle information of the driving environment at the current moment; performing association tracking processing on the obstacle information to obtain the motion trajectory of the obstacle in the driving environment at the current moment, and performing object category perception of the obstacles in the driving environment at the current moment based on the motion trajectory; performing scene perception on the driving environment at the current moment based on the result of the object category perception to obtain actual perception data, and using the actual perception data to generate a third detection result of the driving environment at the current moment.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, after using approximate perception data to generate a second detection result of the driving environment at the current moment, it also includes: calculating a first delay based on the second detection result, and calculating a second delay based on a third detection result, and judging the size of the first delay and the second delay; if the first delay is greater than the second delay, using the second detection result to plan the driving route at the current moment; if the first delay is less than the second delay, using the third detection result to plan the driving route at the current moment.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, based on a preset deep learning network, a direct perception analysis of the driving environment is performed on the data set to be detected to obtain approximate perception data, including: extracting multi-dimensional environmental features in the data set to be detected through a preset deep neural network, and combining the multi-dimensional environmental features with a preset global coordinate system to obtain global environmental features; based on the global environmental features, obstacle detection is performed on the driving environment through a deep neural network, and approximate perception data in the driving environment is generated based on the results of the obstacle detection.

[0009] Optionally, in a fourth implementation manner of the first aspect of the present invention, obstacle detection is performed on the driving environment through a deep neural network based on global environmental characteristics, including: determining, according to the global environmental characteristics, perception parameters of various dimensions of obstacles in the driving environment at past preset moments, and local environmental characteristics of obstacles in the driving environment at the current moment; using the perception parameters of various dimensions to set the initial output results corresponding to the deep neural network, and based on the local environmental characteristics, performing obstacle detection on the driving environment at the current moment through the set deep neural network to update the initial results.

[0010] Optionally, in a fifth implementation manner of the first aspect of the present invention, generating approximate perception data in the driving environment based on the results of obstacle detection includes: judging whether there are new obstacles in the driving environment based on the results of obstacle detection; if so, extracting the position information of the new obstacles from the results of obstacle detection, and using the position information to generate approximate perception data; if not, using the result of no new obstacles as the approximate perception data.

[0011] The second aspect of the present invention provides a driving environment detection device, including: a preprocessing module, used to obtain environmental data at a current moment and a first detection result at a preset moment in the past, and preprocess the environmental data and the first detection result to obtain a data set to be detected; a direct perception module, used to perform a direct perception analysis of the driving environment on the data set to be detected based on a preset deep learning network to obtain approximate perception data; a generation module, used to use the approximate perception data to generate a second detection result of the driving environment at the current moment.

[0012] Optionally, in a first implementation method of the second aspect of the present invention, the driving environment detection device also includes a non-direct connection perception module, which is used to: perform segmentation detection processing on the environmental data at the current moment to obtain obstacle information of the driving environment at the current moment; perform association tracking processing on the obstacle information to obtain the motion trajectory of the obstacle in the driving environment at the current moment, and perform object category perception of the obstacles in the driving environment at the current moment based on the motion trajectory; perform scene perception on the driving environment at the current moment according to the result of the object category perception to obtain actual perception data, and use the actual perception data to generate a third detection result of the driving environment at the current moment.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the driving environment detection device also includes a planning judgment module, which is used to: calculate the first delay based on the second detection result, and calculate the second delay based on the third detection result, and judge the size of the first delay and the second delay; if the first delay is greater than the second delay, the second detection result is used to plan the driving route at the current moment; if the first delay is less than the second delay, the third detection result is used to plan the driving route at the current moment.

[0014] Optionally, in a third implementation of the second aspect of the present invention, the direct perception module includes: an extraction unit, used to extract multidimensional environmental features in the data set to be detected through a preset deep neural network, and combine the multidimensional environmental features with a preset global coordinate system to obtain global environmental features; a detection unit, used to perform obstacle detection on the driving environment through a deep neural network based on the global environmental features, and generate approximate perception data in the driving environment based on the results of the obstacle detection.

[0015] Optionally, in a fourth implementation method of the second aspect of the present invention, the detection unit is also used to: determine, based on global environmental characteristics, perception parameters of various dimensions of obstacles in the driving environment at past preset moments, and local environmental characteristics of obstacles in the driving environment at the current moment; use the perception parameters of various dimensions to set the initial output results corresponding to the deep neural network, and based on the local environmental characteristics, perform obstacle detection on the driving environment at the current moment through the set deep neural network to update the initial results.

[0016] Optionally, in a fifth implementation of the second aspect of the present invention, the detection unit is also used to: determine whether there are new obstacles in the driving environment based on the results of obstacle detection; if so, extract the position information of the new obstacles from the results of obstacle detection, and use the position information to generate approximate perception data; if not, use the result of no new obstacles as the approximate perception data.

[0017] The third aspect of the present invention provides a driving environment detection device, comprising: a memory and at least one processor, wherein instructions are stored in the memory; at least one processor calls the instructions in the memory so that the driving environment detection device executes the above-mentioned driving environment detection method.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned driving environment detection method.

[0019] In the technical solution provided by the present invention, by obtaining the current environmental data and the first detection result of the past preset time, and using the preset deep learning network, the environmental data and the data set to be detected corresponding to the first detection result are directly connected to the perception analysis of the driving environment, and the approximate perception data is obtained to generate the second detection result. The direct perception analysis method is to establish a fast channel between the environmental data collected by the sensor and the planning module output by the final second detection result, so as to achieve ultra-low latency detection of new obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of a non-direct connection perception framework in the prior art;

[0021] Figure 2 Schematic diagram of a direct connection perception framework in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of an embodiment of a method for detecting a driving environment in an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of another embodiment of a method for detecting a driving environment in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of an embodiment of a driving environment detection device in an embodiment of the present invention;

[0025] Figure 6 It is a schematic diagram of another embodiment of the driving environment detection device in the embodiment of the present invention;

[0026] Figure 7 It is a schematic diagram of an embodiment of a driving environment detection device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to solve the problem that the non-direct connection perception framework used in the scene recognition process of vehicle autonomous driving has a long obstacle detection delay and brings potential safety risks, the driving environment detection method provided by the present invention is as follows: Figure 2 As shown in the figure, based on the conventional non-direct perception framework, obstacle motion tracking is added between the two processes of sensor environment collection and route planning to form a new direct perception framework. In the new direct perception framework, obstacle motion tracking can directly obtain the current environmental data from the sensor, as well as the obstacle perception results at a preset time in the past, perform direct perception analysis and generate direct detection results as the result of scene recognition for route planning. Compared with obstacle perception, obstacle motion tracking only needs to perform one or several calculations of obstacle existence detection, category classification, shape estimation, tracking, and quick-freeze acceleration estimation according to specific business needs, and then combined with the results of past obstacle perception, it can obtain the approximate obstacle perception results at the current moment, ensure the accuracy of the driving environment detection results, and also reduce the calculation delay between the sensor collecting environmental data and performing route planning.

[0028] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 3, an embodiment of the method for detecting the driving environment in the embodiment of the present invention includes:

[0030] 301. Obtaining environmental data at a current moment and a first detection result at a preset moment in the past, and preprocessing the environmental data and the first detection result to obtain a data set to be detected;

[0031] It is understandable that the execution subject of the present invention may be a driving environment detection device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0032] In this embodiment, environmental data is collected by sensors installed on the car, where the sensor can be a millimeter wave radar, an ultrasonic radar, a lidar, a camera, or a combination of a radar and a camera. The corresponding environmental data can be in the form of radar point cloud data and / or image data, which is not specifically limited here.

[0033] In this embodiment, a motion tracking module is added between the sensor and the planning module to directly obtain the environmental data collected at the current time t and the first detection result at the preset time tk from the sensor, and combine the two as the input of the motion tracking module for direct perception analysis. Here, it means using the perception results of the perception module in the traditional perception framework (specifically the first detection result at the preset time tk) and the latest input sensor data to be processed (that is, the environmental data at the current moment) to perform scene recognition on the driving environment at the current moment, rather than directly inputting it into the perception module for scene recognition. Among them, the first detection result refers to the detection result of the driving environment at the preset time in the past using the non-direct perception framework in the prior art.

[0034] In addition, when preprocessing the environmental data and the first detection result, the environmental data and each row of data in the first detection result are classified separately to obtain multiple category dimension data, which are specifically represented by vectors or matrices; then the missing data in each category dimension data is completed and feature normalization is performed, the environmental data is stored as a first data set, and the first detection result is stored as a second data set; finally, the data set to be detected is generated from the first data set and the second data set.

[0035] 302. Based on the preset deep learning network, perform direct perception analysis of the driving environment on the detection data set to obtain approximate perception data;

[0036] In this embodiment, the dimensional information of each category corresponding to the first detection result in the detection data set is set as the initial value of the deep learning network learning; then, the deep learning network is used to detect the information of one or more dimensions of the obstacles in the driving environment at the current time t based on the environmental data in the data set to be detected, and the initial value of the corresponding dimension that has been set is updated according to the detection result, and the updated value and the initial value are combined to generate approximate perception data.

[0037] For example, the data set to be detected includes the initial values ​​of the type, shape, position, speed, acceleration and other dimensions of each obstacle in the driving environment at the past preset time tk. The deep learning network detects the position, speed and acceleration of each obstacle based on the environmental data in the data set to be detected; the updated values ​​of the position, speed and acceleration of the obstacle and the initial values ​​of the type and shape of the obstacle that have not been updated are combined as the complete dimensional information of each obstacle in the driving environment of the current frame to obtain approximate perception data.

[0038] Here, the dimension of obstacles detected by the deep learning network is set according to specific business needs. The more dimensions there are, the more complex the deep learning network is, and the more information is obtained, but the detection speed is slower. For example, in a "ghosting" driving environment, it is necessary to quickly respond to obstacles in the driving environment. Therefore, the deep learning network can analyze the environmental data based on the dimension of obstacle existence detection without considering the type and shape of the obstacle at the same time.

[0039] 303. Generate a second detection result of the current driving environment using the approximate perception data.

[0040] In this embodiment, the approximate perception data includes the updated value obtained by the detection of the category dimension that needs to be detected and the initial value of the category dimension that does not need to be detected. After data type conversion is performed and written into the preset data table template, the second detection result can be obtained for input into the planning module to perform post-route planning according to the current driving environment. Among them, the second detection result refers to the detection result of the direct perception framework of the embodiment of the present invention on the driving environment at the current moment.

[0041] In the embodiment of the present invention, by obtaining the current environmental data and the first detection result of the preset time in the past, and using the preset deep learning network, the environmental data and the data set to be detected corresponding to the first detection result are directly connected to the perception analysis of the driving environment, and the approximate perception data is obtained to generate the second detection result. The direct perception analysis method is to establish a fast channel between the environmental data collected by the sensor and the planning module output by the final second detection result, so as to achieve ultra-low latency detection of new obstacles.

[0042] See also Figure 4Another embodiment of the method for detecting a driving environment in the embodiment of the present invention includes:

[0043] 401. Obtaining environmental data at a current moment and a first detection result at a preset moment in the past, and preprocessing the environmental data and the first detection result to obtain a data set to be detected;

[0044] 402. Extract multi-dimensional environmental features in the to-be-detected data set through a preset deep neural network, and combine the multi-dimensional environmental features with a preset global coordinate system to obtain global environmental features;

[0045] 403. Based on the global environment features, obstacle detection is performed on the driving environment through a deep neural network, and approximate perception data in the driving environment is generated based on the result of the obstacle detection;

[0046] In this embodiment, the data set to be detected includes a first data set corresponding to the environmental data and a second data set corresponding to the first detection result. The detection results of each dimension of the obstacle corresponding to the second data set are set as the initial values ​​of each detection dimension and non-detection dimension of the deep neural network, and are set based on the corresponding position of the initial value in the preset global coordinate system to obtain the initial scene coordinate system. Then, the preset multi-dimensional environmental features in the data set to be detected are extracted through the deep neural network, that is, the environmental features of the detection dimension corresponding to the aforementioned initial value, and the global environmental features are combined with the initial scene coordinate system obtained above to obtain the global environmental features, so as to use the deep neural network to detect obstacles in the driving environment, as shown below:

[0047] (1) Determine the perception parameters of various dimensions of obstacles in the driving environment at a preset time in the past and the local environmental characteristics of obstacles in the driving environment at the current time based on the global environmental characteristics;

[0048] (2) The initial output results corresponding to the deep neural network are set using the perception parameters of each dimension, and based on the local environmental characteristics, the obstacle detection is performed on the current driving environment through the set deep neural network to update the initial results.

[0049] In this embodiment, after the global environmental features are obtained by combination, the dimensional information corresponding to the local environmental features of the obstacles is detected according to the positions of the multi-dimensional environmental features in the global environmental features and the initial values ​​corresponding to the positions. For example, for the local environmental features for obstacle existence detection, the position information and shape information of the obstacles existing at the past preset time tk in the global environmental features are detected according to the initial output results, and then the position information and shape information of the obstacles existing at the current time t are detected according to the local environmental features. In combination with the shape and position information, it can be determined whether there are new obstacles, and the obstacle existence detection results are obtained. If there are new obstacles, the position information and shape information of the new obstacles are used as approximate perception data in the formal environment. The specific generation steps are as follows:

[0050] (1) Based on the results of obstacle detection, determine whether there are new obstacles in the driving environment;

[0051] (2) If it exists, extract the location information of the newly added obstacle from the obstacle detection result and use the location information to generate approximate perception data;

[0052] (3) If it does not exist, the result without the newly added obstacle is used as the approximate perception data.

[0053] In this embodiment, direct perception analysis can be used to set the detection dimension of the driving environment that requires a relatively short delay. For obstacles that suddenly appear in the driving environment such as "ghosts" (which may be people, cars, traffic lights, or other objects or life forms, etc.), only obstacle existence detection and position detection can be set; obstacle shape analysis, category analysis, tracking, speed and acceleration analysis, etc. are reduced, which can greatly reduce the delay of formal environment detection, making the subsequent route planning and vehicle control response faster to cope with sudden changes in the driving environment and increase the safety of autonomous driving vehicles.

[0054] Specifically, the existence of an obstacle is determined based on the obstacle detection result obtained by the deep neural network. The corresponding obstacle detection result includes the position information and shape information of the newly appeared obstacle. For example, if the global coordinate information (x, y) of the newly appeared obstacle exists, the corresponding global coordinate information (x, y) is extracted to generate approximate perception data; if not, it is determined that no new obstacle has appeared in the driving environment at the current time t compared with the past preset time tk.

[0055] 404. Generate a second detection result of the current driving environment using the approximate perception data;

[0056] 405. Perform segmentation detection processing on the current environmental data to obtain obstacle information of the current driving environment;

[0057] 406. Performing association tracking processing on the obstacle information to obtain the motion trajectory of the obstacle in the driving environment at the current moment, and performing object category perception on the obstacle in the driving environment at the current moment based on the motion trajectory;

[0058] 407. Perform scene perception on the current driving environment according to the result of object category perception to obtain actual perception data, and use the actual perception data to generate a third detection result of the current driving environment;

[0059] In this embodiment, for the environmental data at the current moment, it is necessary to further perform driving environment detection of all detection dimensions through the perception module to obtain a third detection result, so as to be used as the first detection result for the detection of the driving environment for direct perception analysis at a subsequent moment. For example, the third detection result generated at the current moment t will be directly perceived and analyzed at the subsequent moment t+j in combination with the environmental data collected at the moment t+j and the third detection result, and finally the second detection result at the moment t+j will be obtained, and so on. Among them, the third detection result refers to the detection result of the driving environment at the current moment by the non-direct perception framework in the prior art.

[0060] Specifically, when segmenting and detecting the environmental data at the current moment, if the environmental data in the form of images is collected by the camera as a sensor, the occupancy grid map of the dynamic environment can be first constructed through the environmental data. Then, the inverse sensor model is used to estimate the occupancy probability of each unit in the grid map based on the relevant three-dimensional points; the unit is divided into several segments according to the regional distance between each unit. Finally, the interactive multiple model (IMM) method is used to perform association tracking processing on the obstacles to obtain the movement trajectory of the obstacles in the driving environment at the current moment.

[0061] When segmenting and detecting the current environmental data, a 3D local occupancy grid map based on an octree can also be used, which divides the driving environment into occupied, free, and unknown voxels. After constructing the local grid map, the obstacle information of each dimension, i.e., obstacle information, can be detected based on the inconsistency between the free space and the occupied space in the local grid map.

[0062] 408. Calculate a first delay based on the second detection result, calculate a second delay based on the third detection result, and determine the magnitude of the first delay and the second delay;

[0063] 409. If the first delay is greater than the second delay, the second detection result is used to plan the driving route at the current moment;

[0064] 410. If the first delay is less than the second delay, use the third detection result to plan the driving route at the current moment.

[0065] In this embodiment, the second detection result of the motion tracking module and the third detection result of the perception module can also be used to calculate the delay length, and the second detection result or the third detection result with a shorter delay can be selected and input into the planning module for subsequent form route planning at the current moment.

[0066] Specifically, if the second delay corresponding to the third detection result is A seconds, the frequency of the sensor outputting environmental data is N Hz, and the first delay corresponding to the second detection result is B seconds, then the delay in the direct connection perception framework is min(B, A - 1 / N + B). When B < 1 / N, B < A, and this scheme can achieve a smaller delay than the original one. Usually, N = 10, which means that the delay of the second detection result obtained under the direct connection detection framework compared to the third detection result obtained under the traditional non-direct connection detection framework can be reduced from 0.1 second to 0.02 second.

[0067] In addition to comparing the delays of the second detection result and the third detection result to select the non-direct connection detection route and the direct connection detection route with a shorter delay, different detection routes can also be selected according to other conditions, as follows:

[0068] In one embodiment, the number and type of obstacles in the driving environment can be detected. When a large number of dynamically moving obstacles are detected, it is determined that there are many dynamic obstacles such as people or vehicles in the driving environment at the current moment. At this time, the level of detail of the information about the surrounding obstacles is more important than the length of the driving environment detection calculation delay. Therefore, the non-direct connection detection route can be used to detect the driving environment at the current moment, and the corresponding third detection result can be used for subsequent route planning. Through more dimensional obstacle information, the route can be planned more reasonably to improve the safety of vehicle autonomous driving;

[0069] In one embodiment, the existence of obstacles in the driving environment can be detected. When a new obstacle appears, the "ghost probe" situation may occur. At this time, the driving environment detection calculation delay is more important than the level of detail of the obstacle information. Therefore, the direct connection detection route can be used to detect the driving environment at the current moment to deal with suddenly appearing obstacles.

[0070] In one embodiment, the driving route can be identified. When there is no need to use the direct connection detection route, when the vehicle is driving straight, the non-direct connection detection route can be used to detect the obstacles in the current driving environment. When the vehicle is turning, reversing, changing lanes, etc., the direct connection detection route can be used to detect the obstacles in the current driving environment.

[0071] In an embodiment of the present invention, through direct perception analysis, the detection dimension of the driving environment that requires a relatively short delay is set. For obstacles that suddenly appear in the driving environment such as "ghosts" (which may be people, cars, traffic lights, or other objects or life forms, etc.), only the existence detection and position detection of the obstacles can be set; the obstacle shape analysis, category analysis, tracking, speed and acceleration analysis, etc. are reduced, which can greatly reduce the delay of formal environment detection, making the subsequent route planning and vehicle control reaction faster to cope with sudden changes in the driving environment and increase the safety of autonomous driving vehicles.

[0072] The above describes the method for detecting the driving environment in the embodiment of the present invention. The following describes the device for detecting the driving environment in the embodiment of the present invention. Figure 5 , an embodiment of a driving environment detection device in an embodiment of the present invention includes:

[0073] The preprocessing module 501 is used to obtain the current environmental data and the first detection result at a preset time in the past, and preprocess the environmental data and the first detection result to obtain a data set to be detected;

[0074] A direct perception module 502, configured to perform direct perception analysis of the driving environment on the detection data set based on a preset deep learning network to obtain approximate perception data;

[0075] The generating module 503 is used to generate a second detection result of the driving environment at the current moment by using the approximate perception data.

[0076] In the embodiment of the present invention, by obtaining the current environmental data and the first detection result of the preset time in the past, and using the preset deep learning network, the environmental data and the data set to be detected corresponding to the first detection result are directly connected to the perception analysis of the driving environment, and the approximate perception data is obtained to generate the second detection result. The direct perception analysis method is to establish a fast channel between the environmental data collected by the sensor and the planning module output by the final second detection result, so as to achieve ultra-low latency detection of new obstacles.

[0077] See also Figure 6 Another embodiment of the driving environment detection device in the embodiment of the present invention includes:

[0078] The preprocessing module 501 is used to obtain the current environmental data and the first detection result at a preset time in the past, and preprocess the environmental data and the first detection result to obtain a data set to be detected;

[0079] A direct perception module 502, configured to perform direct perception analysis of the driving environment on the detection data set based on a preset deep learning network to obtain approximate perception data;

[0080] The generating module 503 is used to generate a second detection result of the driving environment at the current moment by using the approximate perception data.

[0081] Specifically, the driving environment detection device further includes a non-direct connection perception module 504, which is used to:

[0082] Perform segmentation detection processing on the current environmental data to obtain obstacle information of the current driving environment;

[0083] Performing association tracking processing on obstacle information to obtain the motion trajectory of obstacles in the driving environment at the current moment, and perceiving the object category of obstacles in the driving environment at the current moment based on the motion trajectory;

[0084] According to the result of object category perception, scene perception is performed on the driving environment at the current moment to obtain actual perception data, and the actual perception data is used to generate a third detection result of the driving environment at the current moment.

[0085] Specifically, the driving environment detection device further includes a planning determination module 505, which is used to:

[0086] Calculating a first delay based on the second detection result, calculating a second delay based on the third detection result, and determining the magnitude of the first delay and the second delay;

[0087] If the first delay is greater than the second delay, the second detection result is used to plan the driving route at the current moment;

[0088] If the first delay is less than the second delay, the third detection result is used to plan the driving route at the current moment.

[0089] Specifically, the direct connection perception module 502 includes:

[0090] An extraction unit 5021 is used to extract multi-dimensional environmental features in the data set to be detected through a preset deep neural network, and combine the multi-dimensional environmental features with a preset global coordinate system to obtain global environmental features;

[0091] The detection unit 5022 is used to detect obstacles in the driving environment through a deep neural network based on global environmental features, and generate approximate perception data in the driving environment based on the results of the obstacle detection.

[0092] Optionally, in a fourth implementation of the second aspect of the present invention, the detection unit 5022 is further configured to:

[0093] Determine the various dimensional perception parameters of obstacles in the driving environment at a preset time in the past and the local environmental characteristics of obstacles in the driving environment at the current time according to the global environmental characteristics;

[0094] The initial output results corresponding to the deep neural network are set using the perception parameters of each dimension, and based on the local environmental characteristics, the obstacle detection is performed on the current driving environment through the set deep neural network to update the initial results.

[0095] Specifically, the detection unit 5022 is further used for:

[0096] Based on the results of obstacle detection, determine whether there are new obstacles in the driving environment;

[0097] If it exists, the location information of the newly added obstacle in the obstacle detection result is extracted, and the location information is used to generate approximate perception data;

[0098] If not, the result without the newly added obstacle is used as the approximate perception data.

[0099] In an embodiment of the present invention, through direct perception analysis, the detection dimension of the driving environment that requires a relatively short delay is set. For obstacles that suddenly appear in the driving environment such as "ghosts" (which may be people, cars, traffic lights, or other objects or life forms, etc.), only the existence detection and position detection of the obstacles can be set; the obstacle shape analysis, category analysis, tracking, speed and acceleration analysis, etc. are reduced, which can greatly reduce the delay of formal environment detection, making the subsequent route planning and vehicle control reaction faster to cope with sudden changes in the driving environment and increase the safety of autonomous driving vehicles.

[0100] above Figure 5 and Figure 6 The driving environment detection device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The driving environment detection device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0101] Figure 7: is a schematic diagram of the structure of a driving environment detection device provided by an embodiment of the present invention. The driving environment detection device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the driving environment detection device 700. Furthermore, the processor 710 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the driving environment detection device 700.

[0102] The driving environment detection device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 760, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 7 The structure of the driving environment detection device shown does not constitute a limitation on the driving environment detection device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0103] The present invention also provides a driving environment detection device, the computer device includes a memory and a processor, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the driving environment detection method in the above-mentioned embodiments.

[0104] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the driving environment detection method.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0107] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting a driving environment, characterized in that: The driving environment detection method includes: Acquire environmental data at a current moment and a first detection result at a preset moment in the past, and preprocess the environmental data and the first detection result to obtain a data set to be detected; Based on a preset deep learning network, a direct perception analysis of the driving environment is performed on the data set to be detected to obtain approximate perception data; Using the approximate perception data to generate a second detection result of the driving environment at the current moment; The method of performing direct perception analysis of the driving environment on the data set to be detected based on a preset deep learning network to obtain approximate perception data includes: extracting multi-dimensional environmental features in the data set to be detected through a preset deep neural network, and combining the multi-dimensional environmental features with a preset global coordinate system to obtain global environmental features; based on the global environmental features, performing obstacle detection on the driving environment through the deep neural network, and generating approximate perception data in the driving environment based on the result of the obstacle detection.

2. The method for detecting the driving environment according to claim 1, characterized in that: After acquiring the environmental data at the current moment and the first detection result at a preset moment in the past, the method further includes: Perform segmentation detection processing on the current environmental data to obtain obstacle information of the current driving environment; Performing association tracking processing on the obstacle information to obtain a motion trajectory of the obstacle in the driving environment at the current moment, and performing object category perception on the obstacle in the driving environment at the current moment based on the motion trajectory; According to the result of object category perception, scene perception is performed on the driving environment at the current moment to obtain actual perception data, and the actual perception data is used to generate a third detection result of the driving environment at the current moment.

3. The method for detecting the driving environment according to claim 2, characterized in that: After the second detection result of the driving environment at the current moment is generated by using the approximate perception data, the method further includes: Calculating a first delay based on the second detection result, calculating a second delay based on the third detection result, and determining the magnitude of the first delay and the second delay; If the first delay is greater than the second delay, the second detection result is used to plan the driving route at the current moment; If the first delay is less than the second delay, the third detection result is used to plan the driving route at the current moment.

4. The method for detecting the driving environment according to claim 1, characterized in that: The performing obstacle detection on the driving environment by using the deep neural network based on the global environment features includes: Determine, based on the global environmental features, the perception parameters of various dimensions of obstacles in the driving environment at a preset time in the past and the local environmental features of obstacles in the driving environment at a current time; The perception parameters of each dimension are used to set the initial output result corresponding to the deep neural network, and based on the local environmental characteristics, the obstacle detection is performed on the driving environment at the current moment through the set deep neural network to update the initial result.

5. The method for detecting the driving environment according to claim 4, characterized in that: The generating of approximate perception data in the driving environment based on the obstacle detection result comprises: Based on the results of obstacle detection, determine whether there are new obstacles in the driving environment; If so, extracting the location information of the newly added obstacle from the obstacle detection result, and using the location information to generate approximate perception data; If not, the result without the newly added obstacle is used as the approximate perception data.

6. A driving environment detection device, characterized in that: The driving environment detection device comprises: A preprocessing module, used to obtain environmental data at a current moment and a first detection result at a preset moment in the past, and preprocess the environmental data and the first detection result to obtain a data set to be detected; A direct perception module, used to perform direct perception analysis of the driving environment on the data set to be detected based on a preset deep learning network to obtain approximate perception data; A generating module, configured to generate a second detection result of the driving environment at a current moment by using the approximate sensing data; The direct-connect perception module includes: an extraction unit, which is used to extract multi-dimensional environmental features in the data set to be detected through a preset deep neural network, and combine the multi-dimensional environmental features with a preset global coordinate system to obtain global environmental features; a detection unit, which is used to perform obstacle detection on the driving environment through a deep neural network based on the global environmental features, and generate approximate perception data in the driving environment based on the results of the obstacle detection.

7. The driving environment detection device according to claim 6, characterized in that: The driving environment detection device further includes a non-direct connection sensing module, which is used to: Perform segmentation detection processing on the current environmental data to obtain obstacle information of the current driving environment; Performing association tracking processing on the obstacle information to obtain a motion trajectory of the obstacle in the driving environment at the current moment, and performing object category perception on the obstacle in the driving environment at the current moment based on the motion trajectory; According to the result of object category perception, scene perception is performed on the driving environment at the current moment to obtain actual perception data, and the actual perception data is used to generate a third detection result of the driving environment at the current moment.

8. A driving environment detection device, characterized in that: The driving environment detection device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the driving environment detection device to perform the steps of the driving environment detection method as described in any one of claims 1-5.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the driving environment detection method as described in any one of claims 1-5 are implemented.

Images