Adaptive perception fusion driving method and device, storage medium and vehicle

Abstract

The application provides an adaptive perception fusion driving method and device, a storage medium and a vehicle. The adaptive perception fusion driving method comprises: obtaining multiple target detection information based on vehicle sensor data and V2X data for a target to be detected; performing information fusion on the multiple target detection information based on weights obtained through current adaptive adjustment to obtain an information fusion result; determining a target detection result according to a preset judgment rule based on the information fusion result; and generating a vehicle passing scheme based on the target detection result. The method fully extracts effective information in multiple types of data by performing information fusion on multiple target detection information obtained through vehicle sensor data and V2X data, and can timely discover dangerous situations in non-line-of-sight or non-detection ranges, thereby improving driving safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicles, and more particularly to an adaptive perception fusion driving method and device, a storage medium and a vehicle. BACKGROUND

[0002] The advanced driving assistance system is to use various sensors installed on the vehicle to sense the surrounding environment at any time during the driving of the vehicle, collect data, identify, detect and track static and dynamic objects, and combine navigation map data to perform system operation and analysis, so as to better help the driver to understand the external environment of the vehicle, and enable some applications such as auxiliary parking and vehicle speed control.

[0003] At present, the advanced driving assistance system usually uses a camera and a radar to detect the surrounding environment, but only has the detection ability within the visual range, which brings certain safety hazards to the safe driving of the vehicle. With the development of vehicle networking and intelligent networked vehicles, the advantages of vehicle networking (V2X) technology are increasingly obvious. The V2X technology can realize detection, scanning and bidirectional communication between vehicles and road networks within the range beyond the visual range, and can provide basic function support for the driving safety of the vehicle.

[0004] The V2X communication technology includes V2I (Vehicle to Infrastructure) and V2V (Vehicle to Vehicle), which can provide a variety of detection information outside the visual range for the vehicle in driving. In this case, how the vehicle perceives the environment based on the various detection information to obtain correct perception information and improve the safety of driving becomes a technical problem to be solved. SUMMARY

[0005] The present application is proposed in view of the above problems. The present application provides an adaptive perception fusion driving method, device, storage medium and vehicle, which can fully extract the effective information amount existing in various data sources, and can timely discover dangerous situations outside the non-visual or non-detection range, which can greatly improve the safety of driving.

[0006] According to an aspect of the present application, an adaptive perception fusion driving method is provided, which comprises:

[0007] For the target to be detected, a plurality of target detection information obtained based on vehicle-mounted sensor data and V2X data respectively is acquired;

[0008] Based on the weight obtained by the current adaptive adjustment, the plurality of target detection information is informationally fused to obtain an information fusion result;

[0009] determine a target detection result according to a preset judgment rule based on the information fusion result;

[0010] generate a vehicle passing scheme based on the target detection result.

[0011] In an embodiment of the present application, a device collecting the vehicle-mounted sensor data or receiving the V2X data is taken as an information source, and the adaptive adjustment method of the weight comprises:

[0012] For each information source, a current weight value of target detection information obtained based on the vehicle-mounted sensor data or the V2X data is determined, comprising: if the target detection information is detection of a target, the current weight value is a preset value greater than 0 and less than 1; if the target detection information is no detection of a target, the current weight value is 0.

[0013] The current weight value is adjusted based on feedback information of a driver of the vehicle to obtain an adjusted weight value.

[0014] The adjusted weight value is taken as a preset value corresponding to target detection information obtained through the information source in the next information fusion.

[0015] In an embodiment of the present application, the adjustment of the current weight value based on the feedback information of the driver of the vehicle comprises:

[0016] For each information source, when the feedback information is correct detection of a target in the target detection information, the current weight value is increased; when the feedback information is incorrect detection of a target in the target detection information, the current weight value is decreased.

[0017] In an embodiment of the present application, the information fusion of multiple target detection information to obtain an information fusion result comprises:

[0018] A current weight value corresponding to each of the multiple target detection information is obtained.

[0019] The multiple target detection information is normalized and weighted averaged according to the current weight value corresponding to each of the multiple target detection information to obtain an information fusion result.

[0020] In an embodiment of the present application, when the target to be detected is an obstacle avoidance target, the vehicle-mounted sensor data comprises one or more of position data, speed data and video image data, and the V2X data comprises V2V data; when the target to be detected is a driving guide mark, the vehicle-mounted sensor data comprises video image data, and the V2X data comprises V2V data and V2I data.

[0021] In an embodiment of the present application, the preset value of the current weight value corresponding to the V2V data is less than the preset value corresponding to the vehicle-mounted sensor data, and the preset value of the current weight value corresponding to the V2V data is less than the preset value corresponding to the V2I data.

[0022] In an embodiment of the present application, the target detection result is determined according to a preset judgment rule based on the information fusion result, including:

[0023] When the information fusion result is greater than or equal to a first preset threshold, it is determined that the target detection result is that the current target is detected.

[0024] When the information fusion result is less than or equal to a second preset threshold, it is determined that the target detection result is that the current target is not detected.

[0025] When the information fusion result is less than the first preset threshold and greater than the second preset threshold, it is determined that the target detection result is that the current target is in an uncertain state.

[0026] In an embodiment of the present application, the method further includes:

[0027] The target detection result and the vehicle passing scheme are broadcast in the Internet of Vehicles.

[0028] In an embodiment of the present application, the method further includes:

[0029] Obtaining map information, the map information including position information of a driving guide mark on a driving road;

[0030] Determining a distance between the vehicle and a next driving guide mark based on the position information of the driving guide mark;

[0031] Determining a target to be detected based on the distance, including:

[0032] When the distance is greater than a preset distance threshold, an obstacle avoidance target is taken as the target to be detected;

[0033] When the distance is less than or equal to the preset distance threshold, the driving guide mark is taken as the target to be detected.

[0034] In an embodiment of the present application, when the target to be detected is an obstacle avoidance target and the vehicle passing scheme is emergency avoidance, the method further includes:

[0035] Generating alarm triggering information to trigger an alarm device to send an alarm signal, the alarm signal including one or more of a sound signal, a light signal, and a tactile signal.

[0036] According to a second aspect of the present application, there is provided a device for adaptive perception fusion driving based on V2X and vehicle-mounted sensors, the device comprising a memory and a processor, the memory storing a computer program executable by the processor, the computer program causing the device in which the processor is installed to perform the adaptive perception fusion driving method when the computer program is executed by the processor.

[0037] According to a third aspect of the present application, there is provided a storage medium, the storage medium storing a computer program executable on a computer, the computer program causing the computer to perform the adaptive perception fusion driving method when executed.

[0038] According to a fourth aspect of the present application, there is provided a vehicle, the vehicle comprising the adaptive perception fusion driving device or the storage medium.

[0039] The present application can greatly improve the safety of driving by fusing multiple target detection information based on multiple data, fully extracting the effective information amount of multiple data sources, and timely discovering dangerous situations in non-line-of-sight or non-detection range. BRIEF DESCRIPTION OF DRAWINGS

[0040] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures. The accompanying drawings are intended to provide a further understanding of embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and serve to explain the present application, and do not limit the present application. In the drawings, like reference numerals refer to like parts throughout the various figures.

[0041] Figure 1 is a schematic block diagram of an electronic device for implementing the adaptive perception fusion driving method and device according to an embodiment of the present application;

[0042] Figure 2 is a schematic flowchart of the adaptive perception fusion driving method according to an embodiment of the present application;

[0043] Figure 3 is a schematic flowchart of the weight adaptive adjustment method in the intersection traffic light monitoring scenario according to a first embodiment of the present application;

[0044] Figure 4 is a schematic flowchart of the weight adaptive adjustment method in the obstacle avoidance target monitoring scenario according to a second embodiment of the present application;

[0045] Figure 5 is a schematic block diagram of the method for weight adjustment and information fusion of the current weight value based on the feedback information of the driver of the vehicle in the T scenario according to an embodiment of the present application;

[0046] Figure 6 is a schematic flow chart of a method for adjusting a current weight value and fusing information based on feedback information of a driver of a vehicle in a P scenario according to an embodiment of the present application;

[0047] Figure 7 is a schematic flow chart of an adaptive perception fusion driving method according to a third embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the present application more apparent, the following will describe example embodiments of the present application in detail with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present application.

[0049] When a vehicle is driving on an urban road, the driver needs to know various information to ensure safe driving, such as the situation of an obstacle avoidance target such as a pedestrian / bicycle around the vehicle, the situation of a traffic signal lamp in front of the vehicle, and the like. Although the current sensors such as a camera, a millimeter wave radar, and a laser radar can inform the driver of the detected early warning information through sound, image, and the like, there is a safety hazard in the blind area of the vehicle sensor. Based on the shared information between vehicles and between a vehicle and a road through V2X technology, the present application proposes an adaptive perception fusion driving scheme for a vehicle driving on an urban road, performs environment perception based on various detection information, obtains correct perception information, and can timely discover a dangerous situation in a non-line-of-sight or non-detection range, thereby improving the safety of driving.

[0050] First, an example electronic device 100 for implementing an adaptive perception fusion driving method and device according to an embodiment of the present application will be described with reference to Figure 1

[0051] As shown in Figure 1 , the electronic device 100 includes one or more processors 101, one or more storage devices 102, an input device 103, and an output device 104, which are interconnected through a bus system 105 and / or other forms of connection mechanism (not shown). It should be noted that Figure 1 The components and structure of the electronic device 100 shown are only exemplary and are not limiting, and the control device can also have other components and structures as needed.

[0052] ​The processor 101 can be a microcontroller unit (MCU), a central processing unit (CPU), a digital signal processor (DSP), a single-chip computer, and an embedded device or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 100 to perform desired functions.

[0053] The storage 102 can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory, and the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor 101 can run the program instructions to implement the client functions (implemented by the processor) in the embodiments described below and / or other desired functions. Various application programs and various data, such as various data used and / or generated by the application programs, and the like, can also be stored in the computer readable storage medium.

[0054] The input device 103 can be a device used by a user to input instructions, and can include a microphone or a touch screen, among other input devices.

[0055] The output device 104 can output various information (such as images or sounds) to the outside (such as a user), and can include one or more of a display, a speaker, and the like.

[0056] Next, a V2X and vehicle sensor-based adaptive perception fusion driving method according to an embodiment of the present application will be described with reference to Figure 2

[0057] As shown in Figure 2 The adaptive perception fusion driving method provided by the present application includes:

[0058] In step S210, for a target to be detected, a plurality of target detection information obtained based on vehicle sensor data and V2X data, respectively, is acquired.

[0059] ​Here, the to-be-detected target can include an obstacle avoidance target and / or a driving guide mark. The obstacle avoidance target can be a dynamic object or a static object. The dynamic object can be, for example, a pedestrian, a motor vehicle, or a non-motor vehicle. The static object can be, for example, a road edge, a tree, a road sign, a lane, or other obstacles. The driving guide mark can be a traffic signal, a traffic sign, a road marking, or the like.

[0060] The sensor data in the present application can be collected by an in-vehicle sensor. The in-vehicle sensor can include an image sensor, a radar sensor, a positioning sensor, a wind speed sensor, an illumination sensor, an infrared sensor, and the like. The position data can be acquired by the radar sensor and / or the positioning sensor. The speed data can also be acquired by the radar sensor and / or the positioning sensor. The video image data can be acquired by the image sensor. The acquisition manner of the sensor data can be determined according to actual conditions, and the acquisition manner of the sensor data is not limited in the present application.

[0061] Here, the image sensor can be, for example, a vehicle front camera, a vehicle side camera, and a vehicle rear camera. The environmental information collected by the camera is image data within the field of view of the camera. The radar sensor can be, for example, an ultrasonic radar, a millimeter wave radar, a laser radar, or other types of radar sensors, which are not limited herein. The positioning sensor can be a Global Positioning System (GPS), a Global Navigation Satellite System (GNSS), or an Inertial Navigation System (INS), or the like. The environmental information collected by the positioning sensor is the position information of the vehicle. The environmental information collected by the wind speed sensor is the wind speed, the wind direction, and the like. The environmental information collected by the illumination sensor is the illumination intensity.

[0062] Correspondingly, the target detection information is obtained based on the data collected by the sensor according to a preset target detection algorithm. The target detection algorithm can be flexibly set by those skilled in the art according to actual requirements, and is not limited in the present application.

[0063] The V2X data in the present application can include data directly acquired through Vehicle to everything (V2X), or information data of a corresponding vehicle or driving guide mark acquired by the vehicle through Internet of Vehicles (IoV) or Device to Device (D2D) communication.

[0064] For example, when the V2X data is data obtained through vehicle networking, the V2X data can include V2V data and V2I data, and the V2V data and the V2I data can include target detection information and other information. Specifically, the V2X data can be received through a V2X communication module. The V2X communication module includes a V2V module for vehicle-to-vehicle communication with other vehicles, and a V2I module for vehicle-to-infrastructure communication with roadside devices; the V2X communication module transmits and receives in the form of messages, and the message format meets the V2X communication standard protocol.

[0065] In step S220, based on the weight obtained by the current adaptive adjustment, the plurality of target detection information is information fused to obtain an information fusion result.

[0066] It should be understood that when the plurality of target detection information is information fused, the target detection information can be represented by a number convenient for normalization.

[0067] For example, for the obstacle avoidance target as a weak traffic participant (Pedestrian, P) scene (hereinafter referred to as P scene) and the driving guide sign as a current road segment traffic light (Traffic-light, T) scene (hereinafter referred to as T scene), the target to be detected is a weak traffic participant and a traffic light, respectively. In the P scene, 1 represents that there is a weak traffic participant nearby, and 0 represents that there is no weak traffic participant nearby. A value between 0 and 1 represents the probability of having a weak traffic participant nearby. Similarly, in the T scene, 1 represents that the current road segment is a green light, and 0 represents that the current road segment is not a green light. A value between 0 and 1 represents the probability that the current road segment is a green light.

[0068] It should be noted that each target detection information has a corresponding weight obtained by the current adaptive adjustment, and in the adaptive adjustment process, the sum of all weight values can be equal to 1 or not equal to 1.

[0069] In step S230, based on the information fusion result, a target detection result is determined according to a preset judgment rule.

[0070] Specifically, when the target detection information is represented by a number convenient for normalization, the information fusion result is a value between 0 and 1.

[0071] In step S240, a vehicle passing scheme is generated based on the target detection result.

[0072] In this application, the specific method of generating a vehicle passing scheme based on the target detection result can use existing methods, which can be set according to actual conditions, and this application does not make specific limitations.

[0073] In the embodiments of the present application, by information fusion of multiple target detection information obtained based on multiple data, the effective information amount existing in multiple data sources is fully extracted, and when some information is inaccurate or has a large offset, the fusion with other information can effectively offset the influence of the offset; even if some type of information has a problem, the probability of the driver being misled is greatly reduced, thereby greatly improving the safety of driving.

[0074] According to one embodiment of the present application, when the obstacle avoidance target is a vulnerable road user, the vehicle-mounted sensor data includes millimeter wave radar data and video image data, and the V2X data includes V2V data; when the target to be detected is a driving guide sign, the vehicle-mounted sensor data includes video image data, and the V2X data includes V2V data and V2I data.

[0075] Next, the weight adaptive adjustment method according to the embodiments of the present application is described with reference to Figure 3 and Figure 4 .

[0076] The weight adaptive adjustment method includes weight initialization at first use and weight adaptive adjustment at other times.

[0077] Figure 3 is a schematic flow chart of the weight adaptive adjustment method in the intersection traffic light monitoring scene according to the first embodiment of the present application, as Figure 3 shown, in this embodiment, various vehicle-mounted sensor data and V2X data are sent to the program for weight adaptive adjustment in the form of messages, specifically including V2I signal phase and timing (SPAT) messages, messages about vulnerable road users detected based on image processing, and current road segment traffic light V2V messages. The weight adaptive adjustment method of this embodiment includes:

[0078] S301: If the V2I SPAT message (hereinafter referred to as V2I message) filtered according to the current road segment traffic light id from the road side unit exists, the initial message weight thereof can be set to 0.4; otherwise, it is fixed to 0 and does not participate in subsequent adaptive module weight adjustment and normalization calculation. The initial message weight can be configured by the user.

[0079] S302: If the message about the vulnerable road user detected by the vehicle-mounted binocular camera based on image processing (hereinafter referred to as camera message) exists, the initial message weight thereof can be set to 0.4; otherwise, it is fixed to 0 and does not participate in subsequent adaptive module weight adjustment and normalization calculation. The initial message weight can be self-defined configuration. If the camera message has been initialized or updated in the P scene before the T scene initialization, it is initialized to the corresponding latest updated value in the P scene.

[0080] S303: If the V2V message about the traffic light of the current road section from other vehicles nearby (hereinafter referred to as V2V message) exists, the initial message weight thereof can be set to 0.2; otherwise, it is fixed to 0 and does not participate in subsequent adaptive module weight adjustment and normalization calculation. If the V2V message has been initialized or updated in the P scene before the T scene initialization, it is initialized to the corresponding latest updated value in the P scene.

[0081] If the camera message weight and the V2V message weight have not been initialized or updated in the P scene, the initial weight ratio of the three messages is as follows: V2I: camera: V2V = 2:2:1; if the camera message weight and the V2V message weight are updated in the P scene, it may cause the sum of the weights of the three messages in the T scene to be not 1. At this time, it is not necessary to make normalization adjustment in the initialization, and the normalization processing will be done in the subsequent message weight adaptive adjustment module.

[0082] S304: Generating the T scene fusion message m according to the initial value or the weight adjusted after the driver feedback information T The adaptive adjustment of the credibility / weight of the effective message type based on the driver feedback information is the message weight adaptive adjustment module, and the specific steps will be described later.

[0083] Since the adjustment is performed after S310 each time, the initial or current weight value is adjusted based on the information fed back by the driver of the vehicle, and the adjusted weight value will be used as the preset value corresponding to the target detection information obtained by the vehicle-mounted sensor in the next target detection.

[0084] Here, the current weight value refers to the weight value adjusted from the initial value.

[0085] S305, S306: The T scene fusion message m obtained in S304 T determining whether it is less than 0.4. If the result is "yes", go to S306, and the message fusion decision is "normal driving in front of the green light"; if not, go to S307.

[0086] S307, S308: Determine whether the T scenario fusion message obtained in S304 is greater than or equal to 0.6. If the result is "yes", enter S308, and the message fusion decision is "slow down / stop at the front red light"; otherwise, directly enter S309.

[0087] The threshold value used for determination can be specifically set according to the situation, and the embodiment does not specifically limit the size of the threshold value.

[0088] S309: The driving strategy decision obtained in S306, S308 needs to be provided to the driver to let him decide whether to receive the decision and make corresponding actions; for the input of S307, the driver decides whether to slow down / stop.

[0089] S310: Feedback the current all effective message source avoidance decision result to the driver, and the driver judges whether each message source is correct or not, and feeds back to the message weight self-adaptive adjustment module.

[0090] It should be noted that the initial weight value can be specifically set according to the actual situation, and the embodiment does not specifically limit the size of the initial weight value.

[0091] Figure 4 The flowchart of the method for self-adaptive adjustment of the weight of the weak traffic participant monitoring scenario according to the second embodiment of the application is shown in FIG. 8, wherein various vehicle-mounted sensor data and V2X data are sent to the program for self-adaptive adjustment of the weight in the form of messages, and the self-adaptive adjustment method of the weight includes: Figure 4

[0092] S401: If the message about the weak traffic participant obtained from the millimeter wave radar processing (hereinafter referred to as millimeter wave radar message) exists, set its initial message weight to 0.4; otherwise, fix it to 0, and do not participate in the subsequent self-adaptive module weight adjustment and normalization calculation. The initial message weight can be configured by the user.

[0093] S402: If the message about the weak traffic participant detected by the vehicle-mounted binocular camera based on image processing (hereinafter referred to as camera message) exists, set its initial message weight to 0.4; otherwise, fix it to 0, and do not participate in the subsequent self-adaptive module weight adjustment and normalization calculation. The initial message weight can be configured by the user. If the camera message has been initialized or updated in the T scenario before the P scenario initialization, initialize it to the corresponding latest updated value in the T scenario.

[0094] ​S403: The V2V message (hereinafter referred to as V2V message) from other vehicles about vulnerable road users is set to an initial message weight of 0.2 if it exists; otherwise, it is fixed at 0 and does not participate in subsequent adaptive module weight adjustment and normalization calculation. If the V2V message has been initialized or updated in the T scenario before the P scenario initialization, it is initialized to the corresponding latest updated value in the T scenario.

[0095] If the camera message weight and the V2V message weight have not been initialized or updated in the T scenario, the initial weight ratio of the three messages is as follows: millimeter wave radar: camera: V2V = 2:2:1; if the camera message weight and the V2V message weight are updated in the T scenario, it may cause the sum of the three message weights in the P scenario to be not equal to 1. At this time, it is not necessary to make normalization adjustment in the initialization, and the normalization processing will be done in the subsequent message weight adaptive adjustment module.

[0096] S404: Generate P scenario fusion message m according to the initial value or the weight adjusted after the driver feedback information P The adaptive adjustment of the credibility / weight of the effective message type based on the driver feedback information is the message weight adaptive adjustment module, and the specific steps will be described later.

[0097] Since each adjustment is performed after S410, the initial or current weight value is adjusted based on the feedback information of the driver of the vehicle, and the adjusted weight value is obtained as the preset value corresponding to the target detection information obtained by the vehicle sensor in the next target detection.

[0098] Here, the current weight value refers to the weight value adjusted from the initial value.

[0099] S405, S406: P scenario fusion message m obtained in S204 P Determine whether it is less than 0.4. If the result is "yes", go to S206, and the message fusion decision is "normal driving"; otherwise, go to S207.

[0100] S407, S408: Determine whether the P scenario fusion message obtained in S204 is greater than or equal to 0.6. If the result is "yes", go to S208, and the message fusion decision is "slow down and avoid"; otherwise, go directly to S209.

[0101] Here, the threshold for judgment can be specifically set according to the situation, and the embodiment does not specifically limit the size of the threshold.

[0102] S409: The driving strategy decision obtained in S406 and S408 needs to be provided to the driver to let the driver decide whether to accept the decision and make corresponding actions; the input of S207, the driver decides whether to slow down and avoid.

[0103] S410: The current all effective message source avoidance decision result is fed back to the driver, the driver judges whether each message source is correct or not, and feeds back to the message weight self-adaptive adjustment module.

[0104] According to one embodiment of the application, the device for collecting the vehicle-mounted sensor data or receiving the V2X data is taken as an information source, and the method for adjusting the current weight value based on the feedback information of the driver of the vehicle includes the following steps:

[0105] For each information source, when the feedback information is correct target detection in the target detection information, the current weight value is increased; when the feedback information is incorrect target detection in the target detection information, the current weight value is decreased.

[0106] According to one embodiment of the application, the target detection information is fused to obtain the information fusion result, which includes the following steps:

[0107] Obtain the current weight value corresponding to the target detection information;

[0108] The target detection information is normalized and weighted averaged with the corresponding current weight value to obtain the information fusion result.

[0109] Next, the method for adjusting the current weight value based on the feedback of the driver of the vehicle and the method for information fusion according to the embodiments of the application are described with reference to Figure 5 and Figure 6

[0110] As shown in Figure 5 , according to one embodiment of the application, the method for adjusting the current weight value based on the feedback information of the driver of the vehicle and the method for information fusion in the T scenario include the following steps:

[0111] S501, S502, S503: In S501, the driver judges whether the V2I message is accurate or not, if it is accurate, enter S502, adjust the weight of V2I message to w1=w1+0.02; if it is not accurate, enter S503, adjust the weight of V2I message to w1=w1-0.02.

[0112] It should be noted that the adjustment value can be set according to the actual situation, and the embodiment does not make specific limitation on the size of the adjustment value.

[0113] ​S504, S505, S506: In S504, the driver judges whether the camera message is accurate. If it is accurate, the driver proceeds to S505 to adjust the weight of the camera message to w2 = w2 + 0.02. If it is inaccurate, the driver proceeds to S506 to adjust the weight of the camera message to w2 = w2 - 0.02.

[0114] S507, S508, S509: In S507, the driver judges whether the V2V message is accurate. If it is accurate, proceed to S508 and adjust the V2V message weight to w3 = w3 + 0.01; if it is inaccurate, proceed to S509 and adjust the V2V message weight to w3 = w3 - 0.01.

[0115] S510: The target detection information obtained from each information source is normalized and weighted by its corresponding weight value to obtain the information fusion result.

[0116] like Figure 6 As shown, according to one embodiment of this application, a method for adjusting the weight value and fusing information based on feedback information from the driver of the vehicle in scenario P includes:

[0117] S601, S602, S603: In S601, the driver judges whether the camera message is accurate. If it is accurate, the process proceeds to S602, and the weight of the camera message is adjusted to w2 = w2 + 0.02. If it is inaccurate, the process proceeds to S603, and the weight of the camera message is adjusted to w2 = w2 - 0.02.

[0118] S604, S605, S606: In S604, the driver judges whether the millimeter-wave radar message is accurate. If it is accurate, the driver enters S605 to adjust the weight of the millimeter-wave radar message to w4 = w4 + 0.02. If it is inaccurate, the driver enters S406 to adjust the weight of the millimeter-wave radar message to w4 = w4 - 0.02.

[0119] S607, S608, S609: In S607, the driver judges whether the V2V message is accurate. If it is accurate, proceed to S608 and adjust the V2V message weight to w3 = w3 + 0.01; if it is inaccurate, proceed to S609 and adjust the V2V message weight to w3 = w3 - 0.01.

[0120] S610: The target detection information obtained from each information source is normalized and weighted by its corresponding weight value to obtain the information fusion result.

[0121] It should be noted that all adjustment values ​​involved in the above two embodiments can be set according to the actual situation, and this embodiment does not specifically limit the size of the adjustment values.

[0122] All corresponding message sources are evaluated for reliability in two scenarios through driver feedback, and then their corresponding message weights are adjusted step by step, so that the resulting sensor corresponding weight can better adapt to the change of driving environment.

[0123] According to one embodiment of the present application, the adaptive perception fusion driving method based on V2X and vehicle-mounted sensors further comprises: broadcasting the target detection result and the vehicle passing scheme in the Internet of Vehicles.

[0124] Specifically, the vehicle-to-Internet communication can include two communication interfaces, one is a short-distance direct communication interface (PC5) between vehicles, people and roads, and the other is a communication interface (Uu) between terminals and base stations, which can realize reliable communication with long distance and larger range. Preferably, the vehicle broadcasts through the PC5 interface for short-distance to share information with nearby vehicles and provide driving decision reference for surrounding vehicles.

[0125] For example, when the vehicle detects a target, such as a weak traffic participant around the vehicle that needs to be avoided, the driving scheme of the vehicle will affect the driving scheme of the surrounding vehicles, and when the vehicle automatically avoids in an emergency, the operation of the vehicle may cause danger to the surrounding normal vehicles. Therefore, the vehicle state information when the vehicle implements automatic avoidance, the information detected by the vehicle for the weak traffic participant around the vehicle, etc. are broadcast through the PC5 air interface through V2X, so as to complete information sharing with surrounding vehicles and provide driving decision reference for surrounding vehicles.

[0126] Next, the adaptive perception fusion driving method according to the third embodiment of the present application will be described with reference to Figure 7 The embodiment provides a driving method for detecting, warning and avoiding weak traffic participants and intersection traffic lights through sensor and V2X fusion cooperative perception and adaptive adjustment according to driver feedback, which comprises:

[0127] S701: The vehicle obtains map information through vehicle-mounted navigation during driving on a municipal road, and the map information includes traffic light position information at an intersection;

[0128] S702: The distance between the vehicle and the next traffic light intersection is determined through the traffic light position information obtained in step S701, and if the distance is greater than 150 meters, step S703 for detecting weak traffic participants is executed, and when the vehicle drives until the distance to the next traffic light intersection is less than 150 meters, step S709 for detecting and determining the traffic light is executed.

[0129] S703: If the step S702 is determined to be no, this step is executed to detect weak traffic participants such as pedestrians and bicycles;

[0130] S704: The vehicle acquires the target detection information shared by the camera, millimeter wave radar, and surrounding vehicles through V2V. The camera associates the recognized target with the surrounding environment, finds the association between the target and the target, and the target and the environment, obtains the target type, state, distance, etc. The millimeter wave radar receives the electromagnetic wave energy reflected by the target for information processing to extract useful information, performs object recognition and tracking processing to obtain the position, speed, size, type, etc. of the target. The position, distance, etc. of the target detected by the surrounding vehicles to the vehicle visual blind area weak traffic participants are obtained through V2X PC5 air interface;

[0131] S705: The detection results of the above three types of targets are fused, the weights of different message sources are dynamically and adaptively adjusted to increase the reliability of the decision results, and then the vehicle passing scheme is determined through the decision results, i.e. whether the vehicle needs to avoid the target;

[0132] S706: When S705 step is determined by threshold value as non-emergency situation and no avoidance operation is needed for the target, the vehicle continues normal driving or the driving scheme is decided by the driver;

[0133] S707: When S705 step is determined by threshold value that there is a weak traffic participant around the vehicle, and the current driving state will cause collision damage to the target and needs to be avoided urgently, after reasonable decision, the driver is informed or warned through alarm device in multiple ways such as sound, light and touch, and the vehicle is actively intervened in time through control execution device so that the vehicle can automatically avoid urgently;

[0134] S708: Regardless of the vehicle driving state (avoidance or normal driving), the vehicle BSM message of the vehicle passing scheme and the detected surrounding target information are broadcasted through PC5 air interface to provide decision reference information for other surrounding vehicles.

[0135] S709: S702 step is determined to be yes, then this step is executed to detect the state of the intersection traffic light;

[0136] S710: The vehicle fuses the detection information of the camera on the traffic light image, the intersection traffic light information broadcasted by the surrounding vehicles through PC5 air interface, and the SPAT information filtered by the roadside unit according to the intersection id;

[0137] S711: The detection results of the above three types of traffic light information are fused, the weights of different message sources are dynamically and adaptively adjusted to increase the reliability of the decision results, and then the vehicle passing scheme is determined through the decision results, i.e. whether the vehicle needs to wait for the traffic light at the intersection;

[0138] S712: When the S711 step passes the threshold value to determine that the front red light needs to be slowed down and stopped, the driver is informed or warned through the alarm device in a variety of ways such as sound, light and touch after reasonable decision, and the vehicle is actively intervened in time to make the vehicle slow down and stop to wait for the red light through the control execution device;

[0139] S713: When the S711 step passes the threshold value to determine that the front can be normally passed or needs to be determined by the driver, the driver decides to continue normal driving or slow down to pass the red and green light intersection;

[0140] S714: The vehicle BSM information of the passing scheme (normal passing or waiting for the red light) and the information detected by the red and green light are broadcast through the PC5 air interface to provide decision reference information for surrounding vehicles.

[0141] It should be noted that in S705 and S711, the preset value corresponding to the V2V message is less than the preset value corresponding to the vehicle sensor message, and the preset value corresponding to the V2V message is less than the preset value corresponding to the V2I message.

[0142] Preferably, the preset value of the current weight value corresponding to the V2V message is half of the preset value corresponding to other types of messages. Whether in the P scene or the T scene, when initializing or adaptively updating, the credibility weight of the V2V message is reduced by about one time compared to other types of messages, which is more in line with the driving habits of the driver. Because compared with the information provided by other vehicles, the driver naturally trusts the sensor message of the vehicle more; compared with the V2V message, the driver will naturally trust the V2I roadside unit message more.

[0143] The method of the embodiment provides specific information fusion schemes for weak traffic participants and current road traffic lights, which fully extracts the effective information amount of various message sources, that is, the millimeter wave radar, camera, V2V message in the P scene and the V2I, camera, V2V message in the T scene. Moreover, when a certain message is inaccurate or has a large offset, the fusion with other messages can offset the influence of the offset, thereby greatly reducing the probability of misleading the driver and correspondingly reducing the probability of safe driving or illegal risk.

[0144] The embodiment of the application also provides a self-adaptive perception fusion driving device, which comprises a memory and a processor, the memory stores a computer program which is run by the processor, and when the computer program is run by the processor, the device installed with the processor executes the self-adaptive perception fusion driving method as described in any of the above embodiments.

[0145] The embodiment of the present application further provides a storage medium, characterized in that the storage medium has a computer program stored thereon, the computer program runs on a computer, and the computer program makes the computer execute the adaptive perception fusion driving method according to any one of the above embodiments when running.

[0146] The embodiment of the present application further provides a vehicle, which comprises the adaptive perception fusion driving device or the storage medium.

[0147] Although the example embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the above-described example embodiments are merely illustrative and are not intended to limit the scope of the present application. Various changes and modifications can be made thereto by those of ordinary skill in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as defined by the appended claims.

[0148] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0149] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be omitted or not executed.

[0150] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not described in detail in order not to obscure the understanding of the present specification.

[0151] Similarly, it is to be understood that the embodiments of the present application can be alternately grouped together in a single embodiment, figure, or description of embodiments for the purpose of brevity and understanding in the interest of conciseness and didacticism. However, this method of grouping the embodiments of the present application is not to be interpreted as reflecting a desire to claim more than is explicitly claimed in each of the claims. Rather, the inventive point is that the respective technical problem can be solved with less than all the features of a certain disclosed single embodiment, as reflected in the respective claims. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim is a separate embodiment of the present application.

[0152] Those skilled in the art will appreciate that all features described herein (including all accompanying claims, abstract and drawings), and steps or elements of any method or process so described, can be combined in any combination, save for features that are mutually exclusive. Each feature or step of the methods described herein can also be replaced by an alternative feature serving the same, equivalent or a similar purpose, unless any such features are expressly stated as being essential to the correct functioning of the application.

[0153] Furthermore, those skilled in the art will recognize that references to conventional functionality of a skilled in the art and references to conventional methodology for analysis and / or detection are meant to be illustrative only and that modifications to and substitutions of such methods and functionality can be made without departing from the scope of the present application. Moreover, those skilled in the art will appreciate that the features of the different embodiments can be combined in any combination, save for mutually exclusive combinations.

[0154] Embodiments of the various components of the present application can be implemented in hardware, or as software modules running in one or more processors, or combinations thereof. Those skilled in the art will appreciate that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functionality of some of the modules of the item analysis apparatus according to embodiments of the present application. The present application can also be implemented as a program (e.g., computer program and computer program product) for executing any or all of the steps of the methods described herein on a computer system. Such program(s) can be stored on a computer readable medium which can be any medium, or combination of media, used to store information for access by a computer. Such a medium can be available on the Internet, or can be provided on a carrier signal, or in any other form.

[0155] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices or means can be listed, comprising means performing the same function. The use of the word 'a' or 'an' does not exclude the presence of a plurality of such devices or means. The word 'first','second', 'third', and the like in the description do not necessarily have a chronological order.

[0156] The above description is only specific embodiments or specific implementations of the present application, and the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, and all such changes or replacements should be covered within the protection scope of the present application. The protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An adaptive perception fusion driving method, characterized in that, The method comprises: For a target to be detected, obtaining a plurality of target detection information respectively based on vehicle-mounted sensor data and V2X data; Based on the weight obtained by the current adaptive adjustment, the plurality of target detection information is information fused to obtain an information fusion result; Based on the information fusion result, a target detection result is determined according to a preset judgment rule; Based on the target detection result, a vehicle passing scheme is generated; The device for collecting the vehicle-mounted sensor data or receiving the V2X data is taken as an information source, and the adaptive adjustment method of the weight comprises: For each information source, a current weight value of the target detection information based on the vehicle-mounted sensor data or the V2X data is determined, including: if the target detection information is that a target is detected, the current weight value is a preset value greater than 0 and less than 1; if the target detection information is that a target is not detected, the current weight value is 0; Based on the feedback information of the driver of the vehicle, the current weight value is adjusted to obtain an adjusted weight value; The adjusted weight value is taken as a preset value corresponding to the target detection information obtained through the information source in the next information fusion; The weight adjustment of the current weight value based on the feedback information of the driver of the vehicle comprises: For each information source, when the feedback information is that the target detection in the target detection information is correct, the current weight value is increased; when the feedback information is that the target detection in the target detection information is incorrect, the current weight value is decreased.

2. The self-adaptive awareness fusion driving method of claim 1, wherein, The information fusion of the plurality of target detection information to obtain the information fusion result comprises: The current weight value corresponding to each of the plurality of target detection information is obtained; The plurality of target detection information is normalized and weighted averaged according to the current weight value corresponding to each of the plurality of target detection information to obtain the information fusion result.

3. The self-adaptive awareness fusion driving method of claim 1, wherein, When the target to be detected is an obstacle avoidance target, the vehicle-mounted sensor data comprises one or more of position data, speed data, and video image data, and the V2X data comprises V2V data; when the target to be detected is a driving guide mark, the vehicle-mounted sensor data comprises video image data, and the V2X data comprises V2V data and V2I data.

4. The self-adaptive awareness fusion driving method of claim 3, wherein, The preset value of the current weight value corresponding to the V2V data is less than the preset value corresponding to the vehicle-mounted sensor data, and the preset value of the current weight value corresponding to the V2V data is less than the preset value corresponding to the V2I data.

5. The self-adaptive perception fusion driving method of any one of claims 1-4, wherein, The determination of the target detection result based on the information fusion result according to the preset judgment rule comprises: When the information fusion result is greater than or equal to a first preset threshold, it is determined that the target detection result is that a current target is detected; When the information fusion result is less than or equal to a second preset threshold, it is determined that the target detection result is that a current target is not detected; When the information fusion result is less than the first preset threshold and greater than the second preset threshold, it is determined that the target detection result is that the current target is in an uncertain state.

6. The self-adaptive perception fusion driving method of any one of claims 1-4, wherein, The method further comprises: The target detection result and the vehicle passing scheme are broadcasted in a vehicle network.

7. The self-adaptive perception fusion driving method of any one of claims 1-4, wherein, The method further comprises: Obtain map information, the map information including position information of a driving guide mark on a driving road; Determine a distance between the vehicle and a next driving guide mark based on the position information of the driving guide mark; Determine a detection target based on the distance, including: When the distance is greater than a preset distance threshold, an obstacle avoidance target is determined as the detection target; When the distance is less than or equal to the preset distance threshold, the driving guide mark is determined as the detection target.

8. The self-adaptive perception fusion driving method of any one of claims 1-4, wherein, When the detection target is the obstacle avoidance target and the vehicle passing scheme is emergency avoidance, the method further includes: Generating alarm trigger information to trigger an alarm device to issue an alarm signal, the alarm signal including one or more of a sound signal, a light signal, and a tactile signal.

9. An adaptive perception fusion driving device, characterized in that, The device includes a memory and a processor, the memory having a computer program stored thereon for execution by the processor, the computer program, when executed by the processor, causing the device in which the processor is installed to perform the adaptive perception fusion driving method of any one of claims 1-8.

10. A storage medium, characterized by The storage medium has a computer program stored thereon, the computer program being executed on a computer, the computer program, when executed, causing the computer to perform the adaptive perception fusion driving method of any one of claims 1-8.

11. A vehicle characterized by comprising: The vehicle includes the adaptive perception fusion driving device of claim 9 or the storage medium of claim 10.

Images