Method and apparatus for environment sensing in aided driving of vehicle

Abstract

The present disclosure provides a method for environmental perception in vehicle assisted driving, comprising: obtaining original radar data from a vehicle-mounted radar, the original radar data comprising a plurality of location points associated with an object detected by the vehicle-mounted radar in an environment; generating scene feature data based on the raw radar data; and providing the scene feature data to a deep learning model to obtain a model prediction value indicating whether the vehicle is in the target driving scene.

Description

Technical Field

[0001] The present disclosure generally relates to the field of vehicles, and more particularly to a method and apparatus for environmental perception in vehicle assisted driving. Background Art

[0002] In recent years, with the rise of artificial intelligence technology, intelligent vehicle technology represented by Advanced Driving Assistance System (ADAS) has received widespread attention from all walks of life and has become an important strategic direction for the development of the automotive industry. As an important branch of intelligent vehicle technology, environmental perception is the basis for realizing back-end functions such as planning, decision-making and control. Through environmental perception, multi-level perception capabilities of the vehicle's environment can be provided, including target perception, small-scale scene perception, and large-scale scene spatiotemporal domain perception. This multi-level perception capability helps to achieve a full understanding of the environment, so that the vehicle can be accurately positioned, and then help the back-end decision-making function module to make accurate planning decisions. Therefore, environmental perception capabilities are crucial to the safety, robustness and stability of smart cars.

[0003] Some existing ADAS systems achieve environmental perception based on visual data. At the same time, in view of the problem that the quality of visual data is easily affected by various factors such as light and weather, it is proposed that environmental perception technology based on radar data or the fusion of radar data and visual data can be used instead. However, it has been noted that in the face of some special driving scenarios (for example, underground garages, tunnels, etc.), the current environmental perception technology often cannot meet the requirements of automatic driving / assisted driving for the accuracy of target recognition. Specifically, in these special driving scenarios, false targets that do not actually exist may be mistakenly identified due to the multipath effect of the radar, so that the back-end decision-making function module may make wrong decisions (for example, wrong braking) for the false targets. This will lead to an unsatisfactory user experience.

[0004] Therefore, there is a need for an improved method of environmental perception in vehicle assisted driving to assist the back-end decision-making function module in adjusting its decision-making strategy by providing prediction results for the recognition of target driving scenarios. Summary of the invention

[0005] It is desirable to provide an improved method for environmental perception in vehicle assisted driving, which can use a deep learning model based on radar data to obtain a model prediction value indicating whether the vehicle is in a target driving scene (for example, a special driving scene such as an underground garage or tunnel that is prone to multipath effects). The model prediction value can be provided to a back-end decision function module so that the back-end decision function module can adjust its decision-making strategy to avoid false triggering and improve the user experience.

[0006] According to one aspect of the present disclosure, a method for environmental perception in vehicle assisted driving is provided, comprising: obtaining raw radar data from a vehicle-mounted radar, the raw radar data comprising a plurality of position points associated with objects detected by the vehicle-mounted radar in an environment; generating scene feature data based on the raw radar data; and providing the scene feature data to a deep learning model to obtain a model prediction value indicating whether the vehicle is in a target driving scene.

[0007] According to another aspect of the present disclosure, a device for environment perception in vehicle assisted driving is provided, comprising: a memory and a processor. The processor is coupled to the memory and configured to execute any one of the methods in the various embodiments of the present disclosure.

[0008] According to still another aspect of the present disclosure, a computer-readable medium is provided, which stores a computer program including instructions, and when the instructions are executed by a processor, the processor is configured to perform a method according to any one of the various embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Various embodiments of the claimed subject matter will now be described by way of example with reference to the accompanying drawings. In the different drawings, the same reference numerals are used to denote the same or similar components.

[0010] Figure 1 A schematic diagram showing an example process of a method for environment perception in vehicle assisted driving according to an example embodiment of the present disclosure is shown.

[0011] Figure 2 A schematic diagram showing various parameters included in raw radar data according to an example embodiment of the present disclosure is shown.

[0012] Figure 3a-3c A schematic diagram of an operation of generating feature scene data through a maximum pooling operation according to an example embodiment of the present disclosure is shown.

[0013] Figure 4 A schematic diagram showing the structure of a deep learning model according to an example embodiment of the present disclosure.

[0014] Figure 5 An operational flow chart of deriving and applying a probability compensation value according to an example embodiment of the present disclosure is shown.

[0015] Figure 6 An operational flow chart of a method for environmental perception in vehicle assisted driving according to an example embodiment of the present disclosure is shown.

[0016] Figure 7A block diagram of an apparatus 700 according to an example embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0017] In the following description, many specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure. However, those skilled in the relevant art will recognize that the present disclosure can be practiced without one or more of the specific details, or alternative methods, components, etc. can be used to practice the present disclosure. In some instances, well-known structures and operations are not shown or described in detail to avoid unnecessarily obscuring the present disclosure.

[0018] In the field of vehicle assisted driving, environmental perception is the basis for realizing back-end functions such as planning, decision-making and control, and is crucial to the safety, robustness and stability of smart cars. Some existing ADAS systems realize environmental perception based on visual data captured by on-board cameras. However, the visual data captured by the camera is easily affected by factors such as light and weather (rain, snow, fog and other weather that affect visibility). At the same time, in some special driving scenarios with poor lighting (for example, underground garages, tunnels, etc.), environmental perception through visual data often cannot obtain recognition results that meet the accuracy requirements.

[0019] On the other hand, environmental perception technology based on radar data can make up for the shortcomings of visual data that are easily affected by factors such as light. At the same time, compared with visual data, radar can also provide more accurate distance and speed information. Therefore, environmental perception technology based on radar data or multi-sensor environmental perception technology based on radar and camera fusion has been widely used in the field of vehicle assisted driving.

[0020] However, when the vehicle is driving in some special driving scenarios, the radar performance may be affected and cannot provide accurate environmental perception results. Such special driving scenarios may be closed or semi-closed scenes with buildings on the side or above, or scenes in which there may be a large number of objects that interfere with the radar signal, such as underground garages and tunnels where a large number of vehicles are parked, etc. In the above special driving scenarios, radar data is typically susceptible to multipath effects / multipath reflections. The multipath effect describes a phenomenon in which electromagnetic waves emitted from the transmitter reach the receiver through multiple paths. For example, signals reflected from buildings or other objects are received together with directly transmitted signals. These signals may overlap with each other and cause interference, causing the original signal to be distorted or errors to occur. In other words, due to the influence of multipath effects in special driving scenarios, it may lead to the identification of non-existent reflection points (or false targets) based on radar data. Moreover, even if radar data is combined with visual data through sensor fusion technology, the above problems cannot be completely avoided. This is because, due to the influence of dim light in special driving scenarios, the possibility of identifying false targets based on visual data is greatly increased. When a false target identified in the visual data overlaps with a false target identified based on the radar data, the fused sensor data will also indicate the presence of the false target. As a result, the backend decision-making function module may make wrong decisions for the false target, such as wrong braking, thereby reducing the user experience.

[0021] In order to solve the above problems, the present disclosure proposes an improved method for environmental perception in vehicle assisted driving, which can use a deep learning model based on radar data to obtain a model prediction value indicating whether the vehicle is in a target driving scene (for example, special driving scenes such as underground garages and tunnels that are prone to multipath effects). The model prediction value can be provided to a back-end decision function module so that the back-end decision function module can adjust its decision-making strategy to avoid false triggering and improve the user experience.

[0022] Figure 1 A schematic diagram showing an example process of a method for environment perception in vehicle assisted driving according to an example embodiment of the present disclosure is shown.

[0023] like Figure 1As shown, the raw radar data 102 from the vehicle-mounted radar can be first obtained. The vehicle-mounted radar is usually a front millimeter wave radar installed at the front of the vehicle, which can detect objects in the environment located in front of the vehicle (including the front and side front) and above the vehicle, and provide detection data. The detection data provided by the vehicle-mounted radar (referred to as raw radar data in this disclosure) may include multiple position points, as shown in the various points in the raw radar data 102. These position points correspond to the reflection points of the electromagnetic waves and characterize the objects detected by the vehicle-mounted radar in the environment, including buildings, vehicles, pedestrians, and the like.

[0024] Each of the above-mentioned multiple position points may include a set of parameters for the detected object. Specifically, the set of parameters may include a radar cross section (RCS). RCS is a physical quantity used to measure the scattering characteristics of a target. The main factors affecting RCS generally include the structure and surface medium of the target object, radar frequency (wavelength), polarization mode, radar line of sight (target attitude angle / incident direction of electromagnetic waves), the positional relationship between the target and the radar, and so on. Therefore, when the above-mentioned influencing factors related to the properties of the radar itself are determined / fixed, the distinction and identification of different target objects can be achieved based on the value of RCS. In addition, the set of parameters may also include values ​​indicating the position attributes of the detected target object, such as radial distance, radial velocity, azimuth, and pitch angle and one or more of other position attributes. In one example, the set of parameters may also include other values, including but not limited to quality metrics for position attributes. The following will focus on Figure 2 The various parameters contained in the raw radar data are further described in detail.

[0025] Since the amount of raw radar data is relatively large, for example, including hundreds to thousands of location points, and each location point may contain the above-mentioned RCS, location attributes and values ​​of quality metrics thereof. Therefore, before using a deep learning model to analyze the radar data to obtain a model prediction value indicating whether the vehicle is in a target driving scene, the raw radar data may be processed to reduce the amount of data to be analyzed and further extract data features related to target driving scene recognition. Thus, the calculation accuracy and efficiency may be improved without increasing the hardware load.

[0026] like Figure 1As shown, based on the original radar data 102, scene feature data 104 for inputting a deep learning model can be generated. First, in order to reduce the amount of data to be analyzed in order to save hardware resources, multiple location points can be filtered to filter out one or more location points among the multiple location points. Points corresponding to target objects of no interest in the multiple location points can be filtered out based on RCS. As discussed above, the distinction and identification of different target objects can be achieved based on the value of RCS. In one example, the RCS values ​​of some typical targets in the application scenario of vehicle-mounted millimeter-wave radar can be: a car corresponds to an RCS value of about 10dBsm, a motorcycle corresponds to an RCS value of about 5dBsm, a bicycle corresponds to an RCS value of about -5dBsm, a pedestrian corresponds to an RCS value of about -7dBsm, and so on. Please note that the above specific values ​​are only used as examples to understand that different target objects can be identified based on different RCS values. In contrast, the RCS of a false target that does not actually exist and is identified due to, for example, the multipath effect usually has a typical lower value. Thus, an RCS threshold may be set, and those location points with RCS values ​​lower than the RCS threshold may be screened out from the plurality of location points, while only location points reaching or exceeding the RCS threshold may be retained.

[0027] In addition, the plurality of position points may also be filtered based on the above-mentioned quality metrics (e.g., the quality metric of radial distance, the quality metric of radial velocity, the quality metric of azimuth angle, and the quality metric of pitch angle) to filter out one or more position points that fail to meet the quality requirements and may affect the accuracy of subsequent calculations. Alternatively, RCS and quality metrics may be used as filtering conditions at the same time. It should be noted that the above-mentioned operation of filtering the plurality of position points based on RCS and / or quality metrics is optional, so the raw radar data may be directly processed in the subsequent process without performing the filtering operation.

[0028] In one example, in order to generate scene feature data 104, the original radar data (which may be filtered as described above) may be subjected to coordinate transformation and maximum pooling operations. Since the original radar data is represented in a polar coordinate system established with the vehicle-mounted radar as the origin, in order to facilitate subsequent processing, the data in the polar coordinate system needs to be converted to a Cartesian coordinate system through a coordinate transformation operation, which can be achieved by converting the radial distance, radial velocity, azimuth, and pitch angle. Next, the radar data subjected to coordinate transformation may be subjected to a maximum pooling operation to obtain scene feature data 104. The following will refer to Figure 3a-3c The specific process of the maximum pooling operation is further described in detail.

[0029] Next, the obtained scene feature data 104 may be provided to a deep learning model 106. The deep learning model 106 may analyze the scene feature data 104 to obtain a model prediction value 108 indicating whether the vehicle is in a target driving scene. The deep learning model 106 may include, but is not limited to, any type of neural network model that may be used to perform target driving scene recognition, such as a convolutional neural network, a recursive neural network, a deep belief network, a stacked autoencoder, and the like.

[0030] In one example, the deep learning model 106 may implement a regression algorithm, and the obtained model prediction value may include a predicted probability that the vehicle is in a target driving scene. For example, the model prediction value 108 may indicate that the predicted probability that the vehicle is in an underground garage is 0.8. In another example, the deep learning model 106 may implement a classification algorithm, and the obtained model prediction value 108 may include a classification prediction result for one or more target driving scenes. For example, the model prediction value 108 may indicate that the vehicle is in a first target driving scene (such as an underground garage), the vehicle is in a second target driving scene (such as a tunnel), and so on. Figure 4 A schematic diagram describing an example structure of deep learning model 106 is further described in detail.

[0031] In order to ensure the prediction accuracy of the deep learning model 106, it can be trained based on the training data set. The training phase refers to the process of iteratively inputting the training data set into the deep learning model and obtaining the optimal model parameters by adjusting various model parameters (e.g., weights, biases). The training data set can be annotated or unannotated to achieve supervised learning and unsupervised learning, respectively. After completing the training phase, the trained deep learning model 106 can be deployed to the execution unit of the vehicle (e.g., vehicle control unit (VCU), electronic control unit (ECU), etc.) to perform the target driving scene recognition task.

[0032] The model prediction value 108 generated by the deep learning model 106 may be provided to the back-end decision function module 110. The back-end decision function module 110 may adjust its decision-making strategy based on the model prediction value 108. Specifically, due to the influence of the multipath effect in the target driving scene, a false target that does not actually exist may be identified. The back-end decision function module 110 may set the conditions for screening the target object of interest to be more stringent based on the model prediction value 108 so that the false target will not be used as the target object of interest. In one example, the back-end decision function module may include an automatic emergency braking (Autonomous Emergency Braking, AEB) system, which is used to automatically take emergency braking measures when a potential danger of collision with a target object is detected and the driver fails to take braking measures in time. Through the method disclosed in the present invention, the AEB system can indicate that the vehicle is in the target driving scene based on the model prediction value 108, and avoid taking certain reflection points that are more likely to be false targets as target objects, thereby avoiding false triggering of automatic emergency braking, so as to improve user comfort and improve user experience.

[0033] In one example, the model prediction value 108 generated by the deep learning model 106 indicating whether the vehicle is in the target driving scenario is not directly provided to the backend decision function module 110. Instead, the model prediction value 108 can be post-processed to further improve the prediction accuracy. Such post-processing is particularly suitable for the case where the model prediction value 108 includes the predicted probability of the vehicle being in the target driving scenario, as discussed below.

[0034] On the one hand, post-processing may include filtering the prediction probability generated by the deep learning model based on historical prediction data from one or more previous prediction cycles. Imagine a scenario where, in the previous prediction cycle, the prediction probability that the vehicle is in the target driving scene can be calculated to be 0.8, and AEB can identify a target object as an uninteresting target object based on relatively strict screening conditions (because it is more likely to be a false target caused by the multipath effect), so it is ignored and no braking is taken. In the prediction of the current cycle, the prediction probability that the vehicle is in the target driving scene can be calculated to jump to 0.2, and AEB can identify the target object as an interesting target object based on less strict screening conditions, so it decides to take automatic emergency braking. In other words, this jump in prediction probability may cause the back-end decision function module to make completely opposite decisions, which may lead to an unsatisfactory user experience. Therefore, the prediction probability of the current cycle can be adjusted based on the historical prediction data of one or more previous prediction cycles to avoid jumps or jitters in the prediction probability. For example, based on the prediction probability obtained in the previous prediction cycle of 0.8 and the prediction probability obtained in the current prediction cycle of 0.2, the prediction probability finally output to the back-end decision function module can be appropriately increased. The filtering based on the historical prediction data can be implemented by using a first-order low-pass filter or an IR filter, etc.

[0035] In another aspect, post-processing may include applying a probability compensation value to the predicted probability generated by the deep learning model. Figure 5 A flowchart of the operations for deriving and applying probability compensation values ​​is further described.

[0036] Through the above post-processing operations, the final prediction probability can be obtained to be provided to the back-end decision function module 110. The back-end decision function module 110 can further adjust its decision-making strategy based on the final prediction probability.

[0037] Figure 2 FIG. 2 is a schematic diagram showing various parameters included in raw radar data according to an example embodiment of the present disclosure. Figure 2 As shown, for each of the plurality of location points, the following parameters may be included: RCS 202 and location attribute 204. Figure 1As discussed, RCS 202 can be used to distinguish and identify different target objects, and in particular, can be used to filter out position points corresponding to false targets. In addition, the position attribute 204 is used to characterize the position information of the electromagnetic wave reflection point corresponding to the position point relative to the vehicle-mounted radar, and can include: radial distance 204-1, radial velocity 204-2, azimuth 204-3, and pitch angle 204-4. It should be noted that the above-mentioned position attributes are particularly applicable to the method of the present disclosure, at least because the method of the present disclosure is intended to identify a target driving scene in which a large number of stationary points are arranged on the side and / or above the vehicle. Specifically, radial distance 204-1 intuitively represents the straight-line distance of the reflection point relative to the vehicle-mounted radar. Radial velocity 204-2 can be used to identify whether the reflection point corresponds to a moving object or a stationary object. For example, a speed threshold can be set, and if the radial velocity of a certain reflection point is higher than the speed threshold, the reflection point is identified as a moving point, otherwise the reflection point is identified as a stationary point. Azimuth 204-3 can be used to identify whether the reflection point is located on the left or right side of the vehicle-mounted radar. Specifically, the horizontal direction from the rear axle to the front axle of the vehicle can be set to be equal to an azimuth of 0 degrees, and the left side of the axis is a positive azimuth and the right side is a negative azimuth. Thus, it is possible to identify whether the reflection point is located on the left or right side of the vehicle by simply judging the sign of the azimuth. The pitch angle 204-4 can be used to identify whether the reflection point is located above the vehicle.

[0038] In addition, the above parameters may also include quality metrics 206 for the position attribute 204, for example, a quality metric 206-1 for the radial distance, a quality metric 206-2 for the radial velocity, a quality metric 206-3 for the azimuth angle, and a quality metric 206-4 for the elevation angle. The quality metric 206 may be used to characterize the confidence of the corresponding value of the position attribute, which may be calculated based on the spectrum analysis of the radar signal. As described above with reference to Figure 1 As discussed, the quality metric 206 can be used to filter the raw radar data alone or in combination with the RCS 202. In addition, the above parameters can also include other types of parameter values, such as variance 208 for the position attribute 204, which can include the variance of radial distance, the variance of radial velocity, the variance of azimuth angle, and the variance of pitch angle, etc.

[0039] Figure 3a-3c A schematic diagram of an operation of generating feature scene data through a maximum pooling operation according to an example embodiment of the present disclosure is shown. Specifically, in assisted driving technology, a vehicle-mounted radar can detect a target object within its field of view (FOV), wherein the FOV defines the coverage of the azimuth and elevation angles of the vehicle-mounted radar and is generally a cone-shaped or fan-shaped area. Figure 3aAs shown, an example of the FOV of the vehicle-mounted radar is shown by a triangle with gray shading. Based on the FOV, a bounding rectangle can be generated, which can cover the FOV, such as Figure 3a Note that before the maximum pooling operation, the original radar data may have been converted from the polar coordinate system to the Cartesian coordinate system. Therefore, the dotted rectangular box can be established on the xy axis plane, as shown in FIG. Figure 3b as shown by the grey shaded rectangle in the .

[0040] Next, the bounding rectangle may be subdivided into a plurality of blocks. For example, the bounding rectangle may be divided into m*n blocks based on a preset or adjustable resolution (e.g., 1 meter). Then, an m*n feature matrix corresponding to the divided plurality of blocks may be constructed, such as Figure 3c As shown in the blank matrix on the left. The constructed feature matrix can then be filled. Specifically, for each of the divided multiple blocks, it can be determined whether the block includes any position point among the multiple position points associated with the object detected by the vehicle-mounted radar. In response to the block not including any position point, a zero value can be filled at the position corresponding to the block in the feature matrix. On the contrary, in response to the block including at least one position point among the multiple position points, a non-zero value can be filled at the position corresponding to the block in the feature matrix. The non-zero value can correspond to a set of parameters of a position point with the maximum RCS among the at least one position point contained in the block (for example, the above reference Figure 1 and Figure 2 discussed RCS and location attributes, etc.).

[0041] In one example, to further simplify the processing, the non-zero value filled in the feature matrix may be the height value of a position point having the maximum RCS among the at least one position point included in the block (which may be determined based on the pitch angle), such as Figure 3c The filled feature matrix on the right is shown. It is reasonable to fill only the height value as feature data into the feature matrix, because the filled position in the feature matrix has already reflected the relative position of the corresponding position point in multiple position points in terms of the x-axis and y-axis to some extent. Figure 3b It is intuitively shown in , where the above maximum pooling operation can be understood as using Figure 3b The raw radar data is represented by a plurality of small cubes as shown, wherein the height of each small cube is determined by the height value of a position point with the maximum RCS among at least one position point contained in the above-mentioned divided blocks. Finally, the filled feature matrix can be provided to the deep learning model as scene feature data to derive a model prediction value indicating whether the vehicle is in the target driving scene.

[0042] Figure 4A schematic diagram showing the structure of a deep learning model according to an example embodiment of the present disclosure is shown. Figure 4 As shown, the deep learning model may include a convolutional neural network (CNN) layer 402, a fully connected layer 404, and a logistic regression layer 406. Among them, the CNN layer 402, the fully connected layer 404, and the logistic regression layer 406 are connected in sequence. The CNN layer 402 can adopt a CNN network of any type or structure. For example, it can be based on any known or developing model architecture, including U-Net, VGG, Resnet, and the like. The logistic regression layer 406 can be implemented using a sigmoid function to normalize the model prediction value to a value range of (0, 1). Thus, the deep learning model can analyze the input feature scene data to obtain the predicted probability that the vehicle is in the target driving scene.

[0043] Figure 5 An operational flow chart of deriving and applying a probability compensation value according to an example embodiment of the present disclosure is shown.

[0044] At S502, a first percentage of stationary points associated with a stationary object detected in a first direction of the vehicle among a plurality of position points may be calculated based on the position attribute. In one example, the first direction may be the side of the vehicle, including the left and right sides. Specifically, it may be determined whether the position point is located on the left or right side of the vehicle based on the azimuth of the corresponding position point. Next, at S504, the first percentage may be compared with a first threshold to determine whether the first percentage is greater than the first threshold. In response to the first percentage being greater than the first threshold (as shown in branch "Y"), it may proceed to S506 to increase the predicted probability generated by the deep learning model by a first probability compensation value. The first probability compensation value may be calculated based on the first percentage and a first scaling factor. For example, the above process may be specifically described as: multiplying the first percentage by the first scaling factor, and adding the calculated product as the first probability compensation value to the predicted probability generated by the deep learning model to derive the predicted probability that is ultimately passed to the back-end decision function module.

[0045] In addition, if it is determined at S504 that the first percentage is less than or equal to the first threshold (as shown in branch "N"), it can proceed to S508. At S508, the first percentage can be compared with the second threshold to determine whether the first percentage is less than the second threshold, wherein the second threshold is less than the first threshold. In response to the first percentage being less than the second threshold (as shown in branch "Y"), the predicted probability generated by the deep learning model can be reduced by a second probability compensation value. The second probability compensation value can be calculated based on the first percentage and the second scaling factor. For example, the above process can be specifically described as: multiplying the first percentage by the second scaling factor, and using the calculated product as the second probability compensation value, and subtracting the second probability compensation value from the predicted probability generated by the deep learning model to derive the predicted probability that is ultimately passed to the back-end decision function module.

[0046] In addition, if it is determined at S508 that the first percentage is greater than or equal to the second threshold and less than or equal to the first threshold (as shown in branch "N"), it can proceed to S512. At S512, a second percentage of stationary points associated with a stationary object detected in a second direction of the vehicle can be calculated among a plurality of position points based on the position attribute. In one example, the second direction can be above the vehicle. Specifically, it can be determined whether the point is located above the vehicle based on the pitch angle of the corresponding position point. Next, at S514, the second percentage can be compared with a third threshold to determine whether the second percentage is greater than the third threshold. The third threshold can have a value that is the same as or different from the first threshold and the second threshold. In response to the second percentage being greater than the third threshold (as shown in branch "Y"), it can proceed to S516. At S516, the predicted probability generated by the deep learning model can be increased by a third probability compensation value. The third probability compensation value can be calculated based on the second percentage and the third scaling factor. For example, the above process can be specifically described as: multiplying the second percentage by the third scaling factor, and adding the calculated product as the third probability compensation value to the predicted probability generated by the deep learning model to derive the predicted probability that is finally passed to the back-end decision function module.

[0047] The above-mentioned first scaling factor, second scaling factor, and third scaling factor can take values ​​in the range of (0, 1), respectively, and their values ​​can be the same or different. In one example, the values ​​of the first scaling factor, the second scaling factor, and the third scaling factor can be fixed, such as preset values ​​derived based on laboratory simulation or statistical data. In another example, the first scaling factor, the second scaling factor, and the third scaling factor can be adjustable. For example, the value of at least one of the first scaling factor, the second scaling factor, and the third scaling factor can be reduced to reduce the degree of influence of the post-processing operation based on the probability compensation value on the predicted probability ultimately delivered to the back-end decision function module. Accordingly, the value of at least one of the first scaling factor, the second scaling factor, and the third scaling factor can be increased to increase the degree of influence of the post-processing operation based on the probability compensation value on the predicted probability ultimately delivered to the back-end decision function module.

[0048] In the above example, the first direction corresponds to the side of the vehicle, and the second direction corresponds to the top of the vehicle. In another example, the first direction may correspond to the top of the vehicle, and the second direction may correspond to the side of the vehicle. In this case, the first percentage may be determined based on the pitch angle of the corresponding position point, and the second percentage may be determined based on the azimuth angle of the corresponding position point.

[0049] Figure 6 An operational flow chart of a method for environmental perception in vehicle assisted driving according to an example embodiment of the present disclosure is shown.

[0050] At S602, raw radar data from a vehicle-mounted radar may be obtained, and the raw radar data may include a plurality of position points associated with objects detected by the vehicle-mounted radar in an environment, as described above with reference to Figure 1 The raw radar data 102 and Figure 2 At S604, scene feature data may be generated based on the raw radar data, as described above with reference to Figure 1 The scene feature data 104 and Figure 3a-3c At S606, the scene feature data may be provided to a deep learning model to obtain a model prediction value indicating whether the vehicle is in a target driving scene, as described above with reference to Figure 1 The deep learning model 104, the model prediction value 108 and Figure 4 and Figure 5 The model prediction value can be provided to the backend decision function module so that the backend decision function module can adjust its decision-making strategy to avoid false triggering and improve the user experience.

[0051] It should be noted that, in one example, multi-sensor fusion environmental perception technology can be used to combine the model prediction value obtained by processing the radar data using the deep learning model mentioned above with the prediction result based on visual data to further improve the recognition accuracy.

[0052] Figure 7 A block diagram of an apparatus 700 according to an exemplary embodiment of the present disclosure is shown. The apparatus 700 may be used to implement the above-mentioned Figure 1-Figure 6 The method for environmental perception in vehicle assisted driving discussed above. In one example, the apparatus 700 may include a control unit of a vehicle, such as the VCU, ECU, etc. discussed above.

[0053] The example device 700 includes a processor 704 connected to an internal communication bus 702, and the processor 704 is used to execute instructions in a memory 706 to implement the method for environmental perception in vehicle assisted driving described in detail above. Examples of processor 704 may include a central processing unit (CPU), a microcontroller, and the like. The memory 706 suitable for tangibly embodying computer program instructions and data includes various forms of memory, such as EPROM, EEPROM, and flash memory devices, and the like. The device 700 may also include an input interface 708 and an output interface 710. The input interface 708 is used to receive input signals and data, including, for example, the raw radar data discussed above, and the like. The output interface 710 is used to send output signals and data, for example, a braking command output based on a model prediction value generated by a deep learning model indicating whether the vehicle is in a target driving scene, and the like.

[0054] The computer program may include instructions executable by a computer, which are used to cause the processor 704 of the device 700 to perform the method for environmental perception in vehicle assisted driving disclosed in the present invention. The program can be recorded on any data storage medium including a memory. For example, the program can be implemented in a digital electronic circuit, or in computer hardware, firmware, software, or a combination thereof. The process / method steps described in the present disclosure can be performed by a programmable processor that executes program instructions to perform methods, steps, operations by operating on input data and generating outputs.

[0055] In addition to what is described herein, various modifications may be made to the disclosed embodiments and implementations of the present invention without departing from the scope of the disclosed embodiments and implementations of the present invention. Therefore, the descriptions and examples herein should be interpreted as illustrative rather than limiting. The scope of the present invention should be measured only by reference to the claims.

Claims

1. A method for environmental perception in vehicle assisted driving, comprising: Obtaining raw radar data from a vehicle-mounted radar, the raw radar data comprising a plurality of position points associated with objects detected by the vehicle-mounted radar in an environment; generating scene feature data based on the raw radar data; The scene feature data is provided to a deep learning model to obtain a model prediction value indicating whether the vehicle is in a target driving scene.

2. The method according to claim 1, in, Each location point in the plurality of location points includes a set of parameters, and the set of parameters includes a radar cross section (RCS) and location attributes; and The location attributes include one or more of the following: Radial distance; Radial velocity; Azimuth; and Pitch angle.

3. The method according to claim 2, in, The set of parameters further includes a quality metric for the location attribute; and The generating of scene feature data based on the original radar data includes: filtering the multiple location points based on the RCS and / or the quality metric to filter out one or more location points among the multiple location points.

4. The method according to claim 1, wherein: The model prediction value includes a predicted probability that the vehicle is in the target driving scenario.

5. The method according to claim 4, in, The deep learning model includes a convolutional neural network (CNN) layer, a fully connected layer, and a logistic regression layer; and Wherein, the logistic regression layer is implemented using the sigmoid function.

6. The method according to claim 4, wherein: Generating scene feature data based on the raw radar data includes: Converting the raw radar data from a polar coordinate system with the vehicle-mounted radar as the origin to a Cartesian coordinate system to obtain coordinate-converted radar data; and A maximum pooling operation is performed on the coordinate-converted radar data to generate the scene feature data.

7. The method according to claim 6, wherein: Performing a maximum pooling operation on the coordinate-converted radar data to generate the scene feature data includes: generating a bounding rectangle based on a field of view (FOV) of the vehicle-mounted radar, wherein the bounding rectangle covers the FOV; subdividing the bounding rectangle into a plurality of blocks; constructing a feature matrix corresponding to the plurality of blocks; and The feature matrix is ​​filled based on the position points contained in each of the multiple blocks to generate the scene feature data.

8. The method according to claim 7, wherein: Filling the feature matrix includes performing the following operations on each of the plurality of blocks: In response to the block not including any position point in the plurality of position points, filling a position in the feature matrix corresponding to the block with a zero value; or In response to the block including at least one of the multiple location points, filling a non-zero value at a position corresponding to the block in the feature matrix, wherein the non-zero value corresponds to the set of parameters of a location point having a maximum RCS among the at least one location point.

9. The method according to claim 4, further comprising: filtering the predicted probabilities based on historical prediction data from one or more previous prediction periods; and / or A probability offset value is applied to the predicted probability.

10. The method according to claim 9, wherein: Applying a probability compensation value to the predicted probability includes: calculating, based on the location attributes, a first percentage of stationary points in the plurality of location points that are associated with stationary objects detected in a first direction of the vehicle; comparing the first percentage to a first threshold; and In response to the first percentage being greater than the first threshold, increasing the predicted probability by a first probability compensation value, wherein the first probability compensation value is calculated based on the first percentage and a first scaling factor.

11. The method according to claim 10, wherein: Applying a probability compensation value to the predicted probability further comprises: In response to the first percentage being less than or equal to the first threshold, comparing the first percentage to a second threshold, wherein the second threshold is lower than the first threshold; and In response to the first percentage being less than the second threshold, reducing the predicted probability by a second probability compensation value, wherein the second probability compensation value is calculated based on the first percentage and a second scaling factor.

12. The method according to claim 11, wherein: Applying a probability compensation value to the predicted probability further comprises: In response to the first percentage being greater than or equal to the second threshold and less than or equal to the first threshold, calculating, based on the location attribute, a second percentage of stationary points in the plurality of location points that are associated with stationary objects detected in a second direction of the vehicle; comparing the second percentage to a third threshold; and In response to the second percentage being greater than the third threshold, increasing the predicted probability by a third compensation value, wherein the third compensation value is calculated based on the second percentage and a third scaling factor.

13. The method according to claim 12, in, The first direction corresponds to a side of the vehicle, and the second direction corresponds to an upper side of the vehicle; or The first direction corresponds to an upper side of the vehicle, and the second direction corresponds to a side of the vehicle.

14. A device for environmental perception in vehicle assisted driving, comprising: Memory; A processor is coupled to the memory, the processor being configured to execute the method according to any one of claims 1-13.

15. A computer readable medium storing a computer program comprising instructions which, when executed by a processor, cause the processor to be configured to perform the method according to any one of claims 1 to 13.

Images