Method for blindness detection of millimeter wave radar, auxiliary driving system and computer program product

Abstract

The present application provides a method for blindness detection of a millimeter wave radar (12), the method comprising: sensing a target object (2) in a surrounding environment with another type of in-vehicle sensor (11) different from the millimeter wave radar and obtaining sensing information; a blindness confidence of the millimeter wave radar (12) is adjusted based on the sensing information. The invention further provides an auxiliary driving system and a computer program product. According to the method and the device, the blindness confidence of the millimeter-wave radar can be adjusted by using sensing information of other types of vehicle-mounted sensors, so that the blindness fault judgment result of the blindness detection algorithm of the millimeter-wave radar is subjected to efficient and accurate credibility detection, misjudgment of blindness detection of the millimeter-wave radar is avoided as much as possible, and the detection accuracy of the millimeter-wave radar is improved. The robustness of the detection result of the millimeter wave radar is improved, and a foundation is laid for reliable implementation of an automatic driving function and / or an auxiliary driving function of the vehicle.

Description

Technical Field

[0001] The present application relates to the field of vehicle-mounted sensors, and in particular to a method for blindness detection of millimeter-wave radar, an assisted driving system, and a computer program product for at least assisting in implementing the steps of the method according to the present application. Background Art

[0002] Millimeter-wave radar is used to detect the positioning data of targets in the vehicle's surrounding environment, such as distance, relative speed, direction angle, etc. These positioning data are the basis for executing various driving assistance functions, such as deviation warning function, collision warning function and / or adaptive cruise function. When the vehicle is driving in harsh working conditions such as open factory areas, wastelands or open areas covered with ice and snow, the amount of point cloud data received from the field of view of the millimeter-wave radar is limited and evenly distributed, which makes the blindness detection algorithm of the millimeter-wave radar prone to misjudgment, which in turn affects the reliable implementation of driving assistance functions.

[0003] Therefore, how to improve the reliability of blindness detection of millimeter-wave radar has become a technical problem that needs to be solved. Summary of the invention

[0004] The purpose of the present application is to provide a method for blindness detection of millimeter wave radar, an assisted driving system, and a computer program product to at least partially solve the problems in the prior art.

[0005] According to a first aspect of the present application, a method for blindness detection of a millimeter wave radar is provided, and the method may include:

[0006] - using another type of vehicle-mounted sensor other than millimeter-wave radar to sense targets in the surrounding environment and obtain sensing information; and

[0007] -Adjusting the blindness confidence of the millimeter wave radar based on the sensing information.

[0008] The core concept of the present application includes at least: constructing a surrounding environment model based on environmental sensing information from other types of on-board sensors to evaluate the openness of the vehicle's current environment - which can reflect the difficulty of the millimeter-wave radar's blindness detection algorithm to misjudge blindness failures in the current surrounding environment. The blindness confidence of the millimeter-wave radar can be adjusted using sensing information from other types of on-board sensors to efficiently and accurately verify the credibility of the blindness failure judgment results of the millimeter-wave radar's blindness detection algorithm, avoid misjudgment of the millimeter-wave radar's blindness detection as much as possible, improve the robustness of the millimeter-wave radar's detection results, and lay the foundation for the reliable implementation of the vehicle's automatic driving function and / or assisted driving function.

[0009] According to a second aspect of the present application, an assisted driving system is provided, and the assisted driving system may include the following components:

[0010] -Sensors for sensing the environment, including: millimeter-wave radar and another type of vehicle-mounted sensor other than millimeter-wave radar;

[0011] - A control unit configured to perform the method according to the application.

[0012] According to a third aspect of the present application, a computer program product, such as a computer-readable program carrier, is provided, comprising computer program instructions, which, when executed by a processor, at least assist in implementing the steps of the method described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention will be described in more detail below with reference to the accompanying drawings, so that the principles, features and advantages of the present invention can be better understood. The accompanying drawings include:

[0014] Figure 1 A schematic diagram of a driving scene according to an exemplary embodiment of the present application is shown;

[0015] Figure 2 A flowchart of a method for blindness detection of a millimeter wave radar according to an exemplary embodiment of the present application is shown;

[0016] Figure 3 A flowchart showing a method for blindness detection of a millimeter wave radar according to another exemplary embodiment of the present application is shown; and

[0017] Figure 4 A schematic block diagram of an assisted driving system according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the scope of protection of the present application.

[0019] like Figure 1 The schematic diagram of a driving scene according to an exemplary embodiment of the present application is shown, in which a vehicle 1 is equipped with a millimeter wave radar 12, through which the positioning data of a target object 2 in the surrounding environment of the vehicle 1 can be detected. The detected positioning data includes, for example, the distance between the target object 2 and the millimeter wave radar 12, the relative speed and direction angle of the target object 2 relative to the millimeter wave radar 12, etc. For the sake of clarity, Figure 1Only one target object 2 is shown as an example, and the target object 2 may include other vehicles, pedestrians, obstacles and other objects with identifiable features in the surrounding environment of the vehicle 1. When the vehicle 1 is traveling in an area with harsh working conditions such as an open factory area, wasteland or an open area covered with ice and snow, the number of point cloud data reflected back from the field of view of the millimeter-wave radar 12 and received by the millimeter-wave radar 12 is limited and evenly distributed, which makes the blindness detection algorithm of the millimeter-wave radar 12 prone to misjudgment. The method according to the present application can effectively improve the credibility of the blindness fault judgment result of the millimeter-wave radar 12.

[0020] Figure 2 A flowchart of a method for blindness detection of a millimeter wave radar according to an exemplary embodiment of the present application is shown. The following exemplary embodiments describe the method according to the present application in more detail.

[0021] like Figure 2 As shown, the method may include steps S1 and S2. In step S1, a vehicle-mounted sensor 11 of another type different from the millimeter-wave radar senses a target object 2 in the surrounding environment and obtains sensing information. In the current embodiment of the present application, the vehicle 1 is equipped with a vehicle-mounted sensor 11 of a sensor type different from the millimeter-wave radar, which may be, for example, a vehicle-mounted camera, or a laser radar, etc.

[0022] The sensing information may include the sensed existence of the target object 2 by the on-board sensor 11. Here, "sensed existence" is used to characterize whether the existence of the target object 2 can be reliably detected by the on-board sensor. When the existence of the target object 2 is detected by the on-board sensor 11, the target object 2 actually exists in the current surrounding environment of the vehicle 1 with a certain probability, and its probability value depends on the operating parameters of the on-board sensor 11. The detection principle and detection accuracy of the on-board sensor 11 are different from those of the millimeter wave radar 12. By collecting the detection results of various types of on-board sensors, as many information sources as possible about the surrounding environment of the vehicle can be provided, thereby constructing a surrounding environment model of the vehicle 1 that is as accurate as possible.

[0023] The sensing information may also include position and size information about the target object 2 sensed by the vehicle-mounted sensor 11, wherein the position and size information may include not only the distance d from the target object 2 to the vehicle-mounted sensor 11, but also the size and quantity of the target object 2. For example, when the vehicle-mounted sensor 11 is configured as a binocular vehicle-mounted camera, the position and size information of the target object 2 may be determined by means of the stereo imaging effect of the two camera lenses. The two camera lenses will form two two-dimensional images when shooting the same target object 2. Due to the baseline distance between the two cameras, the position of the same target object 2 in the two two-dimensional images is offset, and this offset is the parallax. The position and size information of the target object 2 can be determined by calculating the offset of the corresponding pixel points of the target object 2 in the two two-dimensional images. When the vehicle-mounted sensor 11 is configured as a monocular vehicle-mounted camera, for example, the position and size information of the target object 2 may be determined by means of a motion recovery structure algorithm. During the movement of the vehicle 1 relative to the target object 2, multiple two-dimensional images of the target object 2 may be shot by a monocular camera, and the position and size information of the target object 2 may be determined by using the feature matching and geometric relationship between these two-dimensional images.

[0024] In addition, the sensing information may also include texture gradient information of the sensing image of the target object 2 obtained by the vehicle-mounted camera. In the meaning of the present application, "texture gradient information" refers to the change in texture density between different image areas of the sensing image of the target object, which can be obtained by calculating the gradient of the pixel values ​​in the sensing image and transmitted through the CAN bus of the vehicle. The texture gradient information of the sensing image of the target object 2 is related to the position and size information of the target object 2. For example, when the size of the target object 2 increases, the texture gradient value of the sensing image of the target object will increase; when the number of the target objects 2 increases, the texture gradient value of the sensing image of the target object will increase; when the distance d between the target object 2 and the vehicle-mounted camera decreases, the texture gradient value of the sensing image of the target object 2 will increase.

[0025] In step S2, the blindness confidence of the millimeter wave radar 12 can be adjusted based on the sensing information. In the meaning of the present application, "blindness confidence" refers to the credibility assessment of the blindness fault detection result of the millimeter wave radar 12, that is, the determination result of whether the millimeter wave radar 12 has a blindness fault in the current surrounding environment of the vehicle 1 using the blindness detection algorithm, wherein the higher the blindness confidence, the higher the credibility of the blindness fault detection result of the millimeter wave radar 12 using the blindness detection algorithm. When the vehicle 1, especially a heavy truck, is traveling in an area with harsh working conditions such as an open factory area, wasteland, or an open area covered with ice and snow, the millimeter-wave radar 12 is prone to misjudgment of blindness detection. Since the detection principle of the on-board sensor 11 is different from that of the millimeter-wave radar 12, the sensing information of the on-board camera and / or lidar can be used to construct a surrounding environment model of the vehicle 1. Based on the constructed surrounding environment model, the openness of the surrounding environment of the vehicle 1 can be evaluated, which can reflect the difficulty of the blindness detection algorithm of the millimeter-wave radar 12 to misjudge blindness failure in the current surrounding environment, thereby enabling the determination result of the blindness detection algorithm of the millimeter-wave radar 12 to be credibility tested.

[0026] Here, the blindness confidence of the millimeter-wave radar 12 can be adjusted based at least on the sensed existence of the target object 2 by the onboard sensor 11. When the existence of the target object 2 is sensed by the onboard sensor 11, this means that the target object 2 actually exists in the current surrounding environment of the vehicle 1 with a high probability, the degree of openness of the current surrounding environment of the vehicle 1 decreases, and the blindness detection algorithm of the millimeter-wave radar 12 is less likely to misjudge blindness in the surrounding environment, which can improve the blindness confidence of the millimeter-wave radar 12; when the existence of the target object 2 is not sensed by the onboard sensor 11, this means that the target object 2 actually does not exist in the current surrounding environment of the vehicle 1 with a high probability, the degree of openness of the current surrounding environment of the vehicle 1 increases, and the blindness detection algorithm of the millimeter-wave radar 12 is more likely to misjudge blindness in the surrounding environment, which can reduce the blindness confidence of the millimeter-wave radar 12.

[0027] Optionally, the blindness confidence of the millimeter wave radar 12 can be adjusted based on at least the position size information about the target object 2 sensed by the vehicle-mounted sensor 11. When the size of the target object 2 increases, it means that the degree of openness of the current surrounding environment of the vehicle 1 decreases, and the blindness detection algorithm of the millimeter wave radar 12 is less likely to misjudge blindness failure in the surrounding environment, which can improve the blindness confidence of the millimeter wave radar 12; when the number of the target objects 2 increases, it also means that the degree of openness of the current surrounding environment of the vehicle 1 decreases, and the blindness detection algorithm of the millimeter wave radar 12 is less likely to misjudge blindness failure in the surrounding environment, which can improve the blindness confidence of the millimeter wave radar 12; when the distance from the target object 2 to the vehicle-mounted sensor 11 decreases, it also means that the degree of openness of the current surrounding environment of the vehicle 1 decreases, and the blindness detection algorithm of the millimeter wave radar 12 is less likely to misjudge blindness failure in the surrounding environment, which can improve the blindness confidence of the millimeter wave radar 12.

[0028] Optionally, the blindness confidence of the millimeter wave radar 12 can also be adjusted based on at least the texture gradient information of the sensing image of the target object 2 obtained by the vehicle-mounted camera. When the texture gradient value is higher than the preset gradient threshold, it means that the size of the target object 2 is large, the number is large, and / or the distance between the target object 2 and the vehicle 1 is close, and the degree of openness of the current surrounding environment of the vehicle 1 is low. The blindness detection algorithm of the millimeter wave radar 12 is less likely to misjudge the blindness fault in the surrounding environment, and the blindness confidence of the millimeter wave radar 12 can be improved; when the texture gradient value is lower than or equal to the preset gradient threshold, it means that the size of the target object 2 is small, the number is small, and / or the distance between the target object 2 and the vehicle 1 is far, and the degree of openness of the current surrounding environment of the vehicle 1 is high. The blindness detection algorithm of the millimeter wave radar 12 is more likely to misjudge the blindness fault in the surrounding environment, and the blindness confidence of the millimeter wave radar 12 can be reduced.

[0029] According to the above-mentioned embodiments of the present application, a vehicle's surrounding environment model is constructed based on the environmental sensing information of other types of vehicle-mounted sensors to evaluate the openness of the vehicle's current environment - which can reflect the difficulty of the blindness detection algorithm of the millimeter-wave radar 12 to misjudge a blindness fault in the current surrounding environment. Therefore, the blindness confidence of the millimeter-wave radar can be adjusted using the sensing information of other types of vehicle-mounted sensors to efficiently and accurately verify the credibility of the blindness fault judgment results of the blindness detection algorithm of the millimeter-wave radar, avoid misjudgment of the blindness detection of the millimeter-wave radar as much as possible, improve the robustness of the detection results of the millimeter-wave radar, and lay the foundation for the reliable implementation of the vehicle's automatic driving function and / or assisted driving function.

[0030] Figure 3FIG. 1 shows a flowchart of a method for blindness detection of a millimeter wave radar according to another exemplary embodiment of the present application. Figure 2 The differences between the embodiments shown in FIG. 1 and FIG. 2 are as follows, and the same steps are not described repeatedly for the sake of brevity.

[0031] like Figure 3 As shown, the method may further include steps S3 and S4. In step S3, it may be determined whether the blindness confidence of the millimeter-wave radar 12 is higher than a preset confidence threshold. In the case where the blindness confidence of the millimeter-wave radar 12 is higher than the preset confidence threshold, notification information about the blindness failure of the millimeter-wave radar 12 may be sent in step S4. Here, acoustic notification information may be sent to the driver through, for example, an on-board voice device, and / or optical notification information may be sent to the driver through an on-board display device such as an instrument panel, a head-up display, or a central control display screen, to notify the driver that the millimeter-wave radar has a blindness failure, and optionally, the driver may be prompted to take over vehicle control, and / or prompted to drive carefully.

[0032] In addition, notification information about the blindness failure of the millimeter-wave radar 12 can also be sent to the auxiliary driving system or automatic driving system of the vehicle 1, for example, via the CAN bus, so that the auxiliary driving system or the automatic driving system no longer uses the detection information of the millimeter-wave radar 12 as a control factor, thereby avoiding the detection information of the millimeter-wave radar 12 affecting the decision-making control of the auxiliary driving system or the automatic driving system.

[0033] In addition, it should be noted that the step numbers described herein do not necessarily represent a chronological order, but are merely a figure mark. The order can be changed according to specific circumstances as long as the technical purpose of the present application can be achieved.

[0034] Figure 4 A schematic block diagram of an auxiliary driving system 10 according to an exemplary embodiment of the present application is shown.

[0035] like Figure 4 As shown, the auxiliary driving system 10 may include the following components:

[0036] - sensors for sensing the environment, including: a millimeter wave radar 12 and another type of vehicle-mounted sensor 11 different from the millimeter wave radar, wherein the vehicle-mounted sensor 11 includes, for example, a vehicle-mounted camera and / or a laser radar, etc.; and

[0037] - A control unit 13 configured to carry out the method according to the present application.

[0038] Optionally, the system 10 may further include a notification unit 14, which is configured to send notification information about the blindness failure of the millimeter-wave radar 12 when the blindness confidence of the millimeter-wave radar 12 is higher than a preset confidence threshold. The notification unit 14 includes, for example, an in-vehicle voice device, an instrument panel, a head-up display, and / or a central control display screen.

[0039] It should be understood that, in this document, the expressions "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance, nor should they be understood as implicitly indicating the number of the indicated technical features.

[0040] If an embodiment includes an “and / or” relationship between a first feature and a second feature, it should be interpreted as follows: according to one embodiment, the embodiment has both the first feature and the second feature, and according to another embodiment, the embodiment has either only the first feature or only the second feature.

[0041] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to specific features. The feature examples provided in the present disclosure are intended to be illustrative, not restrictive, unless otherwise stated. In specific implementations, multiple features may be combined with each other, where technically feasible, according to actual needs. Various substitutions, changes, and modifications may also be conceived without departing from the spirit and scope of the present application.

Claims

1. A method for blindness detection of a millimeter wave radar (12), the method comprising: Using another type of vehicle-mounted sensor (11) different from the millimeter wave radar to sense a target object (2) in the surrounding environment and obtain sensing information; and The blindness confidence of the millimeter wave radar (12) is adjusted based on the sensing information.

2. The method according to claim 1, wherein: The sensing information includes the sensed existence of the target object (2) by the on-board sensor (11), wherein when the existence of the target object (2) is sensed by the on-board sensor (11), the blindness confidence of the millimeter-wave radar (12) is increased; when the existence of the target object (2) is not sensed by the on-board sensor (11), the blindness confidence of the millimeter-wave radar (12) is reduced.

3. The method according to claim 1, wherein: The sensing information includes position and size information about the target object (2) sensed by the vehicle-mounted sensor (11), wherein the position and size information includes the distance from the target object (2) to the vehicle-mounted sensor (11), and the size and number of the target object (2).

4. The method according to claim 3, wherein: The blindness confidence of the millimeter wave radar (12) is adjusted based at least on the position and size information of the target object (2) sensed by the vehicle-mounted sensor (11), wherein when the size of the target object (2) increases, the blindness confidence of the millimeter wave radar (12) is increased; when the number of the target objects (2) increases, the blindness confidence of the millimeter wave radar (12) is increased; when the distance from the target object (2) to the vehicle-mounted sensor (11) decreases, the blindness confidence of the millimeter wave radar (12) is increased.

5. The method according to claim 1, wherein: The sensing information comprises texture gradient information of a sensing image of a target object (2) obtained by a vehicle-mounted camera, wherein the texture gradient information is related to the position and size information of the target object (2), wherein when the size of the target object (2) increases, the texture gradient value of the sensing image of the target object increases; when the number of the target objects (2) increases, the texture gradient value of the sensing image of the target object increases; and when the distance between the target object (2) and the vehicle-mounted camera decreases, the texture gradient value of the sensing image of the target object (2) increases.

6. The method according to claim 5, wherein: The blindness confidence of a millimeter wave radar (12) is adjusted based on at least texture gradient information of a sensing image of a target object (2) obtained by a vehicle-mounted camera, wherein when the texture gradient value is higher than a preset gradient threshold, the blindness confidence of the millimeter wave radar (12) is increased; and when the texture gradient value is lower than or equal to the preset gradient threshold, the blindness confidence of the millimeter wave radar (12) is reduced.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: When the blindness confidence of the millimeter wave radar (12) is higher than a preset confidence threshold, notification information about the blindness failure of the millimeter wave radar (12) is sent.

8. The method according to any one of claims 1 to 6, wherein: The vehicle-mounted sensor (11) includes a vehicle-mounted camera and / or a laser radar.

9. An assisted driving system (10), the assisted driving system (10) comprising: A sensor for sensing an environment, comprising: a millimeter wave radar (12) and another type of vehicle-mounted sensor (11) different from the millimeter wave radar; and A control unit (13) configured to execute the method according to any one of claims 1 to 8.

10. The driving assistance system (10) according to claim 9, wherein: The vehicle-mounted sensor (11) includes a vehicle-mounted camera and / or a laser radar.

11. The system (10) according to claim 9 or 10, wherein: The system (10) further comprises a notification unit (14) configured to send notification information about a blindness failure of the millimeter-wave radar (12) when the blindness confidence of the millimeter-wave radar (12) is higher than a preset confidence threshold. 12 . A computer program product, such as a computer-readable program carrier, comprising computer program instructions which, when executed by a processor, at least assist in implementing the steps of the method according to claim 1 .

Images