Special vehicle identification method, device, equipment, medium and program product

Abstract

The invention provides a special vehicle identification method, device and equipment, a medium and a program product. According to the method, after weather data, vehicle body environment audio data, a vehicle body environment image and vehicle body environment radar data at the current moment are obtained, special vehicle recognition processing is conducted on the vehicle body environment image according to the current moment, and an image recognition result is obtained; and performing special vehicle identification processing on the vehicle body environment audio data and the vehicle body environment radar data to obtain an audio identification result and a radar identification result. And determining a target recognition result according to the current moment, the weather data, and the result indication identifier and the recognition confidence in the image recognition result, the audio recognition result and the radar recognition result. According to the scheme, through the current moment, the weather data, the image recognition result, the audio recognition result and the radar recognition result, the target recognition result indicating whether the special vehicle exists in the vehicle body environment or not is jointly determined, and the recognition accuracy is improved.

Description

Technical Field

[0001] This application relates to the technical field of vehicles, and in particular, to a special vehicle recognition method, device, equipment, medium and program product. Background Art

[0002] With the continuous development of technology, there are more and more sensors on vehicles, and the functions that vehicles can achieve are also increasing. Vehicles can sense the surrounding environment of the vehicle body through sensors to achieve assisted driving or autonomous driving functions. During assisted driving and autonomous driving, the vehicle can avoid special vehicles.

[0003] In the prior art, in order to avoid special vehicles, it is necessary to identify special vehicles. Usually, the image information captured by a camera is used, and then target detection processing is performed to determine whether there is a special vehicle in the image, that is, to identify whether there is a special vehicle around the vehicle body.

[0004] In summary, the existing special vehicle recognition method only performs recognition based on images, resulting in low recognition accuracy. Summary of the Invention

[0005] The special vehicle recognition method, device, equipment, medium and program product provided by the embodiments of this application are used to solve the problem that the existing special vehicle recognition method only performs recognition based on images, resulting in low recognition accuracy.

[0006] In a first aspect, an embodiment of this application provides a special vehicle recognition method, including:

[0007] Obtain weather data, vehicle body environment audio data, vehicle body environment images, and vehicle body environment radar data at the current moment;

[0008] Perform special vehicle recognition processing on the vehicle body environment image according to the current moment to obtain an image recognition result;

[0009] Perform special vehicle recognition processing on the vehicle body environment audio data and the vehicle body environment radar data respectively to obtain an audio recognition result and a radar recognition result;

[0010] Determine a target recognition result according to the current moment, the weather data, and the result indication identifier and recognition confidence in the image recognition result, the audio recognition result, and the radar recognition result. Both the result indication identifier and the target recognition result are used to indicate whether there is a special vehicle in the vehicle body environment.

[0011] In a possible implementation manner, the performing special vehicle recognition processing on the vehicle body environment image according to the current moment to obtain an image recognition result includes:

[0012] If the current moment belongs to a preset night time range, perform siren light color recognition processing on the vehicle body environment image to determine whether there is a siren light color and the recognition confidence in the vehicle body environment image;

[0013] If there is a siren light color in the vehicle body environment image, generate a result indication identifier indicating that there is a special vehicle in the vehicle body environment;

[0014] If there is no siren light color in the vehicle body environment image, generate a result indication identifier indicating that there is no special vehicle in the vehicle body environment.

[0015] In a possible implementation manner, the method further includes:

[0016] If the current moment does not belong to the preset night time range, perform vehicle type recognition processing on the vehicle body environment image to obtain the image recognition result.

[0017] In a possible implementation manner, the determining the target recognition result according to the current moment, the weather data, and the result indication identifiers and recognition confidences in the image recognition result, the audio recognition result, and the radar recognition result includes:

[0018] If there are different result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result, determine the initial scores of the image recognition result, the audio recognition result, and the radar recognition result according to the current moment, the weather data, and the corresponding relationship between the preset time range, weather range, and scores;

[0019] Calculate the first weights of the image recognition result, the audio recognition result, and the radar recognition result according to the recognition confidences in the image recognition result, the audio recognition result, and the radar recognition result;

[0020] Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result;

[0021] Use the result indication identifier in the recognition result with the largest target score among the image recognition result, the audio recognition result, and the radar recognition result as the target recognition result.

[0022] In a possible implementation manner, the method further includes:

[0023] If there are no different result indication identifiers among the image recognition result, the audio recognition result, and the radar recognition result, then use the result indication identifier in the image recognition result as the target recognition result.

[0024] In a possible implementation manner, before calculating the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result, the method further includes:

[0025] Determine the second weights of the image recognition result, the audio recognition result, and the radar recognition result according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous preset number of moments of the current moment;

[0026] Calculate the target weights of the image recognition result, the audio recognition result, and the radar recognition result according to the first weights and the second weights of the image recognition result, the audio recognition result, and the radar recognition result;

[0027] The step of calculating the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result includes:

[0028] Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the target weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0029] In a possible implementation manner, the method further includes:

[0030] If the target recognition result indicates that there is a special vehicle in the vehicle body environment, then determine whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment;

[0031] If the special vehicle has an overtaking intention, then control the vehicle to avoid.

[0032] In a possible implementation manner, the step of determining whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment includes:

[0033] If the special vehicle position is included in the image recognition result, determine whether the special vehicle has an overtaking intention according to the image recognition result and the historical image recognition result;

[0034] If the special vehicle position is not included in the image recognition result but is included in the radar recognition result, determine whether the special vehicle has an overtaking intention according to the radar recognition result and the historical radar recognition result;

[0035] If the special vehicle position is not included in both the image recognition result and the radar recognition result but is included in the audio recognition result, determine whether the special vehicle has an overtaking intention according to the audio recognition result and the historical audio recognition result.

[0036] In a second aspect, an embodiment of the present application provides a special vehicle recognition device, including:

[0037] An acquisition module, configured to acquire weather data, body environment audio data, body environment images, and body environment radar data at the current moment;

[0038] An identification module, configured to:

[0039] Perform special vehicle identification processing on the body environment images according to the current moment to obtain an image recognition result;

[0040] Perform special vehicle identification processing on the body environment audio data and the body environment radar data respectively to obtain an audio recognition result and a radar recognition result;

[0041] A fusion module, configured to determine a target recognition result according to the current moment, the weather data, and the result indication identifiers and recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result, where both the result indication identifier and the target recognition result are used to indicate whether there is a special vehicle in the body environment.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0043] A processor, a memory, and a communication interface;

[0044] The memory is configured to store executable instructions of the processor;

[0045] Wherein, the processor is configured to execute the special vehicle recognition method according to any one of the first aspect by executing the executable instructions.

[0046] Fourthly, an embodiment of the present application provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the special vehicle recognition method according to any one of the first aspect is implemented.

[0047] Fifthly, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it is used to implement the special vehicle recognition method according to any one of the first aspect.

[0048] The special vehicle recognition method, device, equipment, medium and program product provided by the embodiments of the present application, after obtaining the weather data, vehicle body environment audio data, vehicle body environment image and vehicle body environment radar data at the current moment, perform special vehicle recognition processing on the vehicle body environment image according to the current moment to obtain an image recognition result; perform special vehicle recognition processing on the vehicle body environment audio data and the vehicle body environment radar data respectively to obtain an audio recognition result and a radar recognition result. Then, according to the current moment, the weather data, and the result indication identifier and recognition confidence in the image recognition result, audio recognition result and radar recognition result, determine the target recognition result. This solution jointly determines the target recognition result indicating whether there is a special vehicle in the vehicle body environment through the current moment, the weather data, and the image recognition result, audio recognition result and radar recognition result, improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present application and used together with the description to explain the principles of the present application.

[0050] Figure 1 Schematic diagram of the microphone array provided by the present application;

[0051] Figure 2 Schematic flowchart of Embodiment 1 of the special vehicle recognition method provided by the present application;

[0052] Figure 3 Schematic flowchart of Embodiment 2 of the special vehicle recognition method provided by the present application;

[0053] Figure 4 Schematic flowchart of Embodiment 3 of the special vehicle recognition method provided by the present application;

[0054] Figure 5 Schematic flowchart of Embodiment 4 of the special vehicle recognition method provided by the present application;

[0055] Figure 6 Schematic structural diagram of the embodiment of the special vehicle recognition device provided by the present application;

[0056] Figure 7A schematic structural diagram of an electronic device provided by the present application.

[0057] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and more detailed descriptions will be provided hereinafter. These drawings and written descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0058] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0059] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these process, method, product or device.

[0060] With the continuous development of assisted driving and autonomous driving technologies, the function of automatically avoiding special vehicles can be realized.

[0061] In the prior art, in order to avoid special vehicles, it is necessary to identify special vehicles. Usually, the image information captured by a camera is used, and then target detection processing is performed to determine whether there is a special vehicle in the image, that is, to identify whether there is a special vehicle around the vehicle body. Since only images are used for identification, it is easily affected by the external environment, resulting in a problem of low recognition accuracy.

[0062] In view of the problems existing in the prior art, during the research on the special vehicle recognition method, the inventor found that in order to improve the recognition accuracy, the body environment audio data, the body environment image, and the body environment radar data can be used for comprehensive recognition. According to the current moment, the special vehicle recognition process is performed on the body environment image to obtain an image recognition result; the special vehicle recognition processes are respectively performed on the body environment audio data and the body environment radar data to obtain an audio recognition result and a radar recognition result. Furthermore, according to the current moment, the weather data, and the result indication identifiers and recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result, the target recognition result is determined. Based on the above inventive concept, the special vehicle recognition solution in this application is designed.

[0063] The execution subject of the special vehicle recognition method in this application can be a vehicle control unit (VCU for short), or other devices such as an in-vehicle terminal, a server, or a computer. This application does not limit it. Hereinafter, the VCU is taken as an example for illustration.

[0064] Next, an example of the application scenario of the special vehicle recognition method provided in this application is described.

[0065] Exemplarily, in this application scenario, an autonomous vehicle is driving on the road, and there is a special vehicle driving behind it, and the two are in the same lane.

[0066] A microphone array, a camera, and an in-vehicle radar are installed in the autonomous vehicle. The in-vehicle radar can be a lidar or a millimeter-wave radar. Exemplarily, Figure 1 is a schematic diagram of the microphone array provided in this application. As Figure 1 shown, 4 microphones are installed on the top of the autonomous vehicle, namely microphone 1 - microphone 4. The 4 microphones form a microphone array.

[0067] During the driving process of the autonomous vehicle, the VCU therein can obtain the weather data, the body environment audio data, the body environment image, and the body environment radar data at the current moment. The body environment audio data is obtained through the microphone array, the body environment image is obtained through the camera, and the body environment radar data is obtained through the in-vehicle radar.

[0068] The VCU then performs the special vehicle recognition process on the body environment image according to the current moment to obtain an image recognition result. The special vehicle recognition processes are respectively performed on the body environment audio data and the body environment radar data to obtain an audio recognition result and a radar recognition result.

[0069] Based on the current time, weather data, and the result indication flags and recognition confidence levels in the image recognition result, audio recognition result, and radar recognition result, the VCU determines the target recognition result. Both the result indication flag and the target recognition result are used to indicate whether there is a special vehicle in the vehicle body environment.

[0070] When the target recognition result determined by the VCU indicates that there is a special vehicle in the vehicle body environment, and it is further determined that the special vehicle has an overtaking intention, the vehicle is controlled to avoid.

[0071] It should be noted that the above scenario is only an example of an application scenario provided by the embodiments of the present application. The embodiments of the present application do not limit the actual forms of various devices included in this scenario, nor do they limit the interaction methods between devices. In the specific application of the solution, it can be set according to actual needs.

[0072] Next, the technical solution of the present application will be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0073] Figure 2 FIG. 13 is a schematic flowchart of the first embodiment of the special vehicle recognition method provided by the present application. The embodiments of the present application describe the situation where the VCU performs special vehicle recognition based on the weather data, vehicle body environment audio data, vehicle body environment image, and vehicle body environment radar data at the current time. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware. As Figure 2 shown, the special vehicle recognition method specifically includes the following steps:

[0074] S201: Obtain the weather data, vehicle body environment audio data, vehicle body environment image, and vehicle body environment radar data at the current time.

[0075] In this step, in order to recognize whether there is a special vehicle in the vehicle body environment, the VCU needs to obtain the weather data, vehicle body environment audio data, vehicle body environment image, and vehicle body environment radar data at the current time.

[0076] The VCU obtains the vehicle body environment audio data through a microphone array, obtains the vehicle body environment image through a camera, and obtains the vehicle body environment radar data through an in-vehicle radar.

[0077] S202: Perform special vehicle recognition processing on the vehicle body environment image according to the current time to obtain an image recognition result.

[0078] In this step, after the VCU obtains the vehicle body environment image, when there is a special vehicle in the vehicle body environment, the special vehicle is clearer in the image captured by the camera during the day, and the color of the alarm light of the special vehicle is clearer in the image captured at night. Therefore, it is necessary to perform special vehicle recognition processing on the vehicle body environment image according to the current time to obtain the image recognition result.

[0079] Specifically, the VCU determines whether the current time belongs to the preset night time range.

[0080] It should be noted that the preset night time range can be 19:00 - 5:00, 16:00 - 6:00, 20:00 - 4:00, etc. The embodiments of the present application do not limit the preset night time range, which can be determined according to the actual situation.

[0081] If the current time belongs to the preset night time range, the VCU performs alarm light color recognition processing on the vehicle body environment image to determine whether there is an alarm light color and the recognition confidence level in the vehicle body environment image. The recognition confidence level is used to represent the recognition accuracy. The higher the recognition confidence level, the higher the recognition accuracy.

[0082] If there is an alarm light color in the vehicle body environment image, a result indication flag indicating that there is a special vehicle in the vehicle body environment is generated; if there is no alarm light color in the vehicle body environment image, a result indication flag indicating that there is no special vehicle in the vehicle body environment is generated. The image recognition result includes the result indication flag and the recognition confidence level.

[0083] If the current time does not belong to the preset night time range, the VCU performs vehicle type recognition processing on the vehicle body environment image to obtain the image recognition result, and the image recognition result includes the result indication flag and the recognition confidence level.

[0084] It should be noted that for the alarm light color recognition processing of the vehicle body environment image and the vehicle type recognition processing of the vehicle body environment image, a pre-trained neural network model can be used for processing.

[0085] S203: Perform special vehicle recognition processing on the vehicle body environment audio data and the vehicle body environment radar data respectively to obtain the audio recognition result and the radar recognition result.

[0086] In this step, after the VCU obtains the vehicle body environment audio data and the vehicle body environment radar data, it performs special vehicle recognition processing on the vehicle body environment audio data and the vehicle body environment radar data respectively to obtain the audio recognition result and the radar recognition result.

[0087] It should be noted that for the identification and processing of special vehicle in vehicle body environment audio data and vehicle body environment radar data, a pre-trained neural network model can be used for processing. Both the audio recognition result and the radar recognition result include a result indication flag and a recognition confidence level. The result indication flag is used to indicate whether there is a special vehicle in the vehicle body environment.

[0088] It should be noted that the vehicle body environment audio data can be denoised and then processed for special vehicle identification.

[0089] It should be noted that for the execution order of step S202 and step S203, it can be to execute step S202 first and then step S203. It can also be: execute step S203 first and then step S202. It can also be: step S202 and step S203 are executed simultaneously. The embodiments of the present application do not limit the execution order of step S202 and step S203, and can be determined according to the actual situation.

[0090] S204: Determine the target recognition result according to the current time, weather data, and the result indication flag and recognition confidence level in the image recognition result, audio recognition result, and radar recognition result.

[0091] In this step, after the VCU obtains the image recognition result, audio recognition result, and radar recognition result, in order to determine whether there is a special vehicle in the vehicle body environment, according to the current time, weather data, and the result indication flag and recognition confidence level in the image recognition result, audio recognition result, and radar recognition result, the target recognition result is determined. The target recognition result is used to indicate whether there is a special vehicle in the vehicle body environment.

[0092] The special vehicle identification method provided in this embodiment, after obtaining the weather data, vehicle body environment audio data, vehicle body environment image, and vehicle body environment radar data at the current time, performs special vehicle identification processing on the vehicle body environment image according to the current time to obtain an image recognition result; performs special vehicle identification processing on the vehicle body environment audio data and vehicle body environment radar data respectively to obtain an audio recognition result and a radar recognition result. Then, according to the current time, weather data, and the result indication flag and recognition confidence level in the image recognition result, audio recognition result, and radar recognition result, the target recognition result is determined. Compared with the prior art that only performs identification through images, this solution jointly determines the target recognition result indicating whether there is a special vehicle in the vehicle body environment through the current time, weather data, and the image recognition result, audio recognition result, and radar recognition result, improving the recognition accuracy.

[0093] Figure 3This is a schematic flowchart of the second embodiment of the special vehicle recognition method provided by this application. On the basis of the above embodiment, this application embodiment describes the situation where the VCU determines the target recognition result according to the current time, weather data, and the image recognition result, audio recognition result, and radar recognition result. As Figure 3 shown, the special vehicle recognition method specifically includes the following steps:

[0094] S301: Determine whether there are different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result. If there are different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result, then execute steps S302 - S305; if there are no different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result, then execute step S306.

[0095] In this step, after the VCU obtains the image recognition result, audio recognition result, and radar recognition result, in order to determine whether there is a conflict among the image recognition result, audio recognition result, and radar recognition result, it is necessary to determine whether there are different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result.

[0096] The existence of different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result means that: among the image recognition result, audio recognition result, and radar recognition result, there are both result indication identifiers indicating the existence of a special vehicle in the vehicle body environment and result indication identifiers indicating the non - existence of a special vehicle in the vehicle body environment.

[0097] The non - existence of different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result means that: the result indication identifiers in the image recognition result, audio recognition result, and radar recognition result all indicate the existence of a special vehicle in the vehicle body environment, or all indicate the non - existence of a special vehicle in the vehicle body environment.

[0098] S302: According to the current time, weather data, and the corresponding relationship between the preset time range, weather range, and score, determine the initial scores of the image recognition result, audio recognition result, and radar recognition result.

[0099] In this step, if the VCU determines that there are different result indication identifiers in the image recognition result, audio recognition result, and radar recognition result, it means that there is a conflict among the image recognition result, audio recognition result, and radar recognition result, and it is necessary to determine the initial scores of the image recognition result, audio recognition result, and radar recognition result according to the current time, weather data, and the corresponding relationship between the preset time range, weather range, and score.

[0100] First, the target time range to which the current moment belongs can be determined, and then the target weather range described by the weather data can be determined. Next, according to the preset corresponding relationship between the time range, weather range, and score, the initial scores of the image recognition result, audio recognition result, and radar recognition result corresponding to the target time range and target weather range are determined.

[0101] Exemplarily, Table 1 is a schematic table of the corresponding relationship between the time range, weather range, and score provided by this application.

[0102] Table 1

[0103]

[0104] When the current moment belongs to the daytime range and the weather data belongs to non-severe weather, the initial scores of the image recognition result, radar recognition result, and audio recognition result decrease in sequence. When the current moment belongs to the daytime range and the weather data belongs to severe weather, the initial scores of the audio recognition result, image recognition result, and radar recognition result decrease in sequence. When the current moment belongs to the nighttime range and the weather data belongs to non-severe weather, the initial scores of the radar recognition result, audio recognition result, and image recognition result decrease in sequence. When the current moment belongs to the nighttime range and the weather data belongs to severe weather, the initial scores of the audio recognition result, radar recognition result, and image recognition result decrease in sequence.

[0105] It should be noted that the nighttime range can be [19:00, 5:00), [16:00, 6:00), [20:00, 4:00), etc., and the daytime range can be [5:00, 19:00), [6:00, 16:00), [4:00, 20:00), etc. Non-severe weather includes sunny days, cloudy days, etc., and severe weather includes cloudy days, rainy days, snowy days, foggy days, etc. The embodiments of this application do not limit the nighttime range, daytime range, severe weather, and non-severe weather, and can be determined according to actual situations.

[0106] It should be noted that Table 1 is only an example of the corresponding relationship between the time range, weather range, and score. The embodiments of this application do not limit the corresponding relationship between the time range, weather range, and score, and can be determined according to actual situations.

[0107] S303: Calculate the first weights of the image recognition result, audio recognition result, and radar recognition result according to the recognition confidence in the image recognition result, audio recognition result, and radar recognition result.

[0108] In this step, when the VCU determines that there are different result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result, it is also necessary to calculate the first weights of the image recognition result, the audio recognition result, and the radar recognition result according to the recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result.

[0109] The recognition confidence level of the image recognition result is , the recognition confidence level of the audio recognition result is , and the recognition confidence level in the radar recognition result is . Then, the first weight of the image recognition result is , the first weight of the audio recognition result is , and the first weight of the radar recognition result is .

[0110] It should be noted that for the execution order of step S302 and step S303, it can be to execute step S302 first and then step S303. It can also be: execute step S303 first and then step S302. It can also be: step S302 and step S303 are executed simultaneously. The embodiments of the present application do not limit the execution order of step S302 and step S303, and can be determined according to the actual situation.

[0111] S304: Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0112] In this step, after the VCU obtains the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result, it calculates the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0113] The product of the initial score of the image recognition result and the first weight of the image recognition result is used as the target score of the image recognition result.

[0114] The product of the initial score of the audio recognition result and the first weight of the audio recognition result is used as the target score of the audio recognition result.

[0115] The product of the initial score of the radar recognition result and the first weight of the radar recognition result is used as the target score of the radar recognition result.

[0116] S305: Use the result indication identifier in the recognition result with the largest target score among the image recognition result, the audio recognition result, and the radar recognition result as the target recognition result.

[0117] In this step, after the VCU obtains the target scores of the image recognition result, the audio recognition result, and the radar recognition result, it uses the result indication identifier in the recognition result with the highest target score among the image recognition result, the audio recognition result, and the radar recognition result as the target recognition result.

[0118] The target score is used to represent the recognition accuracy. The larger the target score, the higher the recognition accuracy.

[0119] S306: Use the result indication identifier in the image recognition result as the target recognition result.

[0120] In this step, if the VCU determines that there are no different result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result, it means that there is no conflict among the image recognition result, the audio recognition result, and the radar recognition result. Then it uses the result indication identifier in the image recognition result as the target recognition result.

[0121] It should be noted that since the result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result are the same, the result indication identifier in the audio recognition result or the radar recognition result can also be used as the target recognition result.

[0122] The special vehicle recognition method provided in this embodiment improves the accuracy of the target recognition result by determining whether there is a conflict among the image recognition result, the audio recognition result, and the radar recognition result, calculating the target score to determine the target recognition result when there is a conflict, and using the result indication identifier in the image recognition result as the target recognition result when there is no conflict.

[0123] Figure 4 This is a schematic flowchart of the third embodiment of the special vehicle recognition method provided by this application. On the basis of the above embodiment, this application embodiment describes the situation where the VCU determines the second weights of the image recognition result, the audio recognition result, and the radar recognition result before calculating the target score, and then combines the first weights to calculate the target weights. As Figure 4 shown, the special vehicle recognition method specifically includes the following steps:

[0124] S401: Determine the second weights of the image recognition result, the audio recognition result, and the radar recognition result according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous preset number of moments of the current moment.

[0125] In this step, after the VCU obtains the first weights of the image recognition result, the audio recognition result, and the radar recognition result, in order to further improve the accuracy of the target score, the second weights of the image recognition result, the audio recognition result, and the radar recognition result can also be determined according to the image recognition result, the audio recognition result, and the radar recognition result, as well as the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous preset number of moments of the current moment.

[0126] Specifically, the ratio of the number of recognition results with the same result indication identifier as the image recognition result among the historical image recognition results at the preset number of moments to the preset number is used as the second weight of the image recognition result.

[0127] The ratio of the number of recognition results with the same result indication identifier as the audio recognition result among the historical audio recognition results at the preset number of moments to the preset number is used as the second weight of the audio recognition result.

[0128] The ratio of the number of recognition results with the same result indication identifier as the radar recognition result among the historical radar recognition results at the preset number of moments to the preset number is used as the second weight of the radar recognition result.

[0129] It should be noted that the preset number can be 3, 5, 7, etc. The embodiments of the present application do not limit the preset number, which can be determined according to the actual situation.

[0130] S402: Calculate the target weights of the image recognition result, the audio recognition result, and the radar recognition result according to the first weights and the second weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0131] In this step, after the VCU obtains the second weights of the image recognition result, the audio recognition result, and the radar recognition result, in combination with the first weights of the image recognition result, the audio recognition result, and the radar recognition result, calculate the target weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0132] The average value of the first weight and the second weight of the image recognition result is used as the target weight of the image recognition result.

[0133] The average value of the first weight and the second weight of the audio recognition result is used as the target weight of the audio recognition result.

[0134] The average value of the first weight and the second weight of the radar recognition result is used as the target weight of the radar recognition result.

[0135] S403: Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result based on the initial scores and the target weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0136] It should be noted that this step is similar to step S304 in Embodiment 2, and will not be elaborated here.

[0137] In the special vehicle recognition method provided in this embodiment, after calculating the second weights of the image recognition result, the audio recognition result, and the radar recognition result, the target scores are determined according to the second weights, the first weights, and the initial scores, further improving the accuracy of the target scores.

[0138] Figure 5 This is a schematic flowchart of Embodiment 4 of the special vehicle recognition method provided by this application. On the basis of the above embodiments, this application embodiment VCU determines whether to avoid when it is determined that there is a special vehicle in the vehicle body environment. As Figure 5 shown, the special vehicle recognition method specifically includes the following steps:

[0139] S501: If the target recognition result indicates that there is a special vehicle in the vehicle body environment, determine whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment.

[0140] In this step, if the VCU determines that the target recognition result indicates that there is a special vehicle in the vehicle body environment, since there is no need to avoid the special vehicle in some scenarios, such as when the special vehicle is parked by the roadside. Only when the special vehicle has an overtaking intention, it is necessary to avoid, so it is necessary to determine whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment.

[0141] Specifically, if the special vehicle position is included in the image recognition result, determine whether the special vehicle has an overtaking intention according to the image recognition result and the historical image recognition result. If the special vehicle position is not included in the historical image recognition result, it is determined that the special vehicle does not have an overtaking intention.

[0142] If the special vehicle position is included in the historical image recognition result, since the vehicle position is the position in the vehicle coordinate system, it can be determined whether the distance between the special vehicle position in the historical image recognition result and the vehicle itself is greater than the distance between the special vehicle position in the image recognition result and the vehicle itself.

[0143] If the distance between the position of the special vehicle in the historical image recognition result and the own vehicle is greater than the distance between the position of the special vehicle in the image recognition result and the own vehicle, it is determined that the special vehicle has an overtaking intention. If the distance between the position of the special vehicle in the historical image recognition result and the own vehicle is less than or equal to the distance between the position of the special vehicle in the image recognition result and the own vehicle, it is determined that the special vehicle does not have an overtaking intention.

[0144] If the image recognition result does not include the position of the special vehicle and the radar recognition result includes the position of the special vehicle, then according to the radar recognition result and the historical radar recognition result, it is determined whether the special vehicle has an overtaking intention. If the historical radar recognition result does not include the position of the special vehicle, it is determined that the special vehicle does not have an overtaking intention.

[0145] If the historical radar recognition result includes the position of the special vehicle, since the vehicle position is in the vehicle coordinate system, it can be determined whether the distance between the position of the special vehicle in the historical radar recognition result and the own vehicle is greater than the distance between the position of the special vehicle in the radar recognition result and the own vehicle.

[0146] If the distance between the position of the special vehicle in the historical radar recognition result and the own vehicle is greater than the distance between the position of the special vehicle in the radar recognition result and the own vehicle, it is determined that the special vehicle has an overtaking intention. If the distance between the position of the special vehicle in the historical radar recognition result and the own vehicle is less than or equal to the distance between the position of the special vehicle in the radar recognition result and the own vehicle, it is determined that the special vehicle does not have an overtaking intention.

[0147] If neither the image recognition result nor the radar recognition result includes the position of the special vehicle and the audio recognition result includes the position of the special vehicle, then according to the audio recognition result and the historical audio recognition result, it is determined whether the special vehicle has an overtaking intention. If the historical audio recognition result does not include the position of the special vehicle, it is determined that the special vehicle does not have an overtaking intention.

[0148] If the historical audio recognition result includes the position of the special vehicle, since the vehicle position is in the vehicle coordinate system, it can be determined whether the distance between the position of the special vehicle in the historical audio recognition result and the own vehicle is greater than the distance between the position of the special vehicle in the audio recognition result and the own vehicle.

[0149] If the distance between the position of the special vehicle in the historical audio recognition result and the own vehicle is greater than the distance between the position of the special vehicle in the audio recognition result and the own vehicle, it is determined that the special vehicle has an overtaking intention. If the distance between the position of the special vehicle in the historical audio recognition result and the own vehicle is less than or equal to the distance between the position of the special vehicle in the audio recognition result and the own vehicle, it is determined that the special vehicle does not have an overtaking intention.

[0150] S502: If the special vehicle has an overtaking intention, control the vehicle to avoid.

[0151] In this step, if the VCU determines that the special vehicle has an overtaking intention, it controls the vehicle to give way. If it determines that the special vehicle has no overtaking intention, it controls the vehicle to drive normally.

[0152] Exemplarily, during the driving process of the vehicle, there is a special vehicle driving behind the vehicle, and the special vehicle has an overtaking intention. The VCU determines that the result indication identifier in the image recognition result and the audio recognition result indicates that there is a special vehicle in the vehicle body environment, and the result indication identifier in the radar recognition result indicates that there is no special vehicle in the vehicle body environment. After calculation, the target score of the audio recognition result is the highest, so it is determined that there is a special vehicle in the vehicle body environment. Furthermore, the VCU determines that the special vehicle has an overtaking intention, and then controls the vehicle to give way.

[0153] It should be noted that when the VCU determines that the special vehicle has an overtaking intention, if it receives other control instructions, it will give way first.

[0154] It should be noted that the ways of giving way can be pulling over to stop, lateral avoidance, longitudinal speed reduction, etc. The embodiments of the present application do not limit the ways of giving way, which can be determined according to the actual situation.

[0155] The special vehicle recognition method provided in this embodiment determines whether the vehicle gives way by determining whether the special vehicle has an overtaking intention, improving the accuracy of giving way.

[0156] The following is the device embodiment of the present application, which can be used to execute the method embodiment of the present application. For the details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0157] Figure 6 It is a schematic structural diagram of the special vehicle recognition device embodiment provided by the present application. As Figure 6 shown, the special vehicle recognition device 60 includes:

[0158] An acquisition module 61, configured to acquire weather data, vehicle body environment audio data, vehicle body environment images, and vehicle body environment radar data at the current moment;

[0159] An identification module 62, configured to:

[0160] Perform special vehicle recognition processing on the vehicle body environment image according to the current moment to obtain an image recognition result;

[0161] Perform special vehicle recognition processing on the vehicle body environment audio data and the vehicle body environment radar data respectively to obtain an audio recognition result and a radar recognition result;

[0162] A fusion module 63, configured to determine a target recognition result according to the current moment, the weather data, and the result indication flag and recognition confidence in the image recognition result, the audio recognition result, and the radar recognition result, where both the result indication flag and the target recognition result are used to indicate whether there is a special vehicle in the vehicle body environment.

[0163] Further, the recognition module 62 is specifically configured to:

[0164] If the current moment belongs to a preset night time range, perform an alarm light color recognition process on the vehicle body environment image to determine whether there is an alarm light color and recognition confidence in the vehicle body environment image;

[0165] If there is an alarm light color in the vehicle body environment image, generate a result indication flag indicating that there is a special vehicle in the vehicle body environment;

[0166] If there is no alarm light color in the vehicle body environment image, generate a result indication flag indicating that there is no special vehicle in the vehicle body environment.

[0167] Further, the recognition module 62 is specifically further configured to:

[0168] If the current moment does not belong to the preset night time range, perform a vehicle type recognition process on the vehicle body environment image to obtain the image recognition result.

[0169] Further, the fusion module 63 is specifically configured to:

[0170] If there are different result indication flags in the image recognition result, the audio recognition result, and the radar recognition result, determine the initial scores of the image recognition result, the audio recognition result, and the radar recognition result according to the current moment, the weather data, and the corresponding relationship between the preset time range, weather range, and score;

[0171] Calculate the first weights of the image recognition result, the audio recognition result, and the radar recognition result according to the recognition confidence in the image recognition result, the audio recognition result, and the radar recognition result;

[0172] Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result;

[0173] Use the result indication flag in the recognition result with the maximum target score among the image recognition result, the audio recognition result, and the radar recognition result as the target recognition result.

[0174] Further, the fusion module 63 is specifically further configured to:

[0175] If there are no different result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result, then use the result indication identifier in the image recognition result as the target recognition result.

[0176] Further, before calculating the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result, the fusion module 63 is further configured to:

[0177] Determine the second weights of the image recognition result, the audio recognition result, and the radar recognition result according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous preset number of moments of the current moment;

[0178] Calculate the target weights of the image recognition result, the audio recognition result, and the radar recognition result according to the first weights and the second weights of the image recognition result, the audio recognition result, and the radar recognition result;

[0179] Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the target weights of the image recognition result, the audio recognition result, and the radar recognition result.

[0180] The processing module 64 is configured to:

[0181] If the target recognition result indicates that there is a special vehicle in the vehicle body environment, then determine whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment;

[0182] If the special vehicle has an overtaking intention, then control the vehicle to avoid.

[0183] Further, the processing module 64 is specifically configured to:

[0184] If the image recognition result includes the position of the special vehicle, then determine whether the special vehicle has an overtaking intention according to the image recognition result and the historical image recognition result;

[0185] If the special vehicle position is not included in the image recognition result and is included in the radar recognition result, determine whether the special vehicle has an overtaking intention according to the radar recognition result and the historical radar recognition result;

[0186] If the special vehicle position is not included in both the image recognition result and the radar recognition result and is included in the audio recognition result, determine whether the special vehicle has an overtaking intention according to the audio recognition result and the historical audio recognition result.

[0187] The special vehicle recognition device provided in this embodiment is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effects are similar and will not be elaborated here.

[0188] Figure 7 It is a schematic structural diagram of an electronic device provided by this application. As Figure 7 shown, the electronic device 70 includes:

[0189] A processor 71, a memory 72, and a communication interface 73;

[0190] The memory 72 is used to store the executable instructions of the processor 71;

[0191] Among them, the processor 71 is configured to execute the technical solutions in any of the foregoing method embodiments by executing the executable instructions.

[0192] Optionally, the memory 72 can be either independent or integrated with the processor 71.

[0193] Optionally, when the memory 72 is a device independent of the processor 71, the electronic device 70 may further include:

[0194] A bus 74. The memory 72 and the communication interface 73 are connected to the processor 71 through the bus 74 to complete mutual communication. The communication interface 73 is used to communicate with other devices.

[0195] Optionally, the communication interface 73 can be specifically implemented by a transceiver. The communication interface is used to implement communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include a random access memory (RAM) and may also include a non-volatile memory, such as at least one disk memory.

[0196] The bus 74 can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0197] The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0198] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments, and its implementation principle and technical effects are similar, and will not be elaborated here.

[0199] The embodiment of the present application further provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the technical solutions provided in any of the foregoing method embodiments are implemented.

[0200] The embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0201] Those of ordinary skill in the art can understand that all or part of the steps of implementing the foregoing method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

[0202] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A special vehicle identification method, characterized in that, Including: Obtain the weather data, vehicle body environment audio data, vehicle body environment image, and vehicle body environment radar data at the current moment; According to the current moment, perform special vehicle recognition processing on the vehicle body environment image to obtain an image recognition result; Perform special vehicle recognition processing on the vehicle body environment audio data and the vehicle body environment radar data respectively to obtain an audio recognition result and a radar recognition result; According to the current moment, the weather data, and the result indication identifiers and recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result, determine a target recognition result, where the result indication identifier and the target recognition result are both used to indicate whether there is a special vehicle in the vehicle body environment.

2. The method according to claim 1, wherein The performing special vehicle recognition processing on the vehicle body environment image according to the current moment to obtain an image recognition result includes: If the current moment belongs to a preset night time range, perform siren light color recognition processing on the vehicle body environment image to determine whether there is a siren light color and the recognition confidence level in the vehicle body environment image; If there is a siren light color in the vehicle body environment image, generate a result indication identifier indicating that there is a special vehicle in the vehicle body environment; If there is no siren light color in the vehicle body environment image, generate a result indication identifier indicating that there is no special vehicle in the vehicle body environment.

3. The method according to claim 2, wherein The method further includes: If the current moment does not belong to the preset night time range, perform vehicle type recognition processing on the vehicle body environment image to obtain the image recognition result.

4. The method according to claim 1, characterized in that The determining the target recognition result according to the current moment, the weather data, and the result indication identifiers and recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result includes: If there are different result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result, determine the initial scores of the image recognition result, the audio recognition result, and the radar recognition result according to the current moment, the weather data, and the corresponding relationship between the preset time range, weather range, and score; Calculate the first weights of the image recognition result, the audio recognition result, and the radar recognition result according to the recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result; Calculate the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result; Use the result indication identifier in the recognition result with the largest target score among the image recognition result, the audio recognition result, and the radar recognition result as the target recognition result.

5. The method according to claim 4, characterized in that, The method further includes: If there are no different result indication identifiers in the image recognition result, the audio recognition result, and the radar recognition result, use the result indication identifier in the image recognition result as the target recognition result.

6. The method according to claim 4, wherein Before calculating the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result, the method further includes: Determining the second weights of the image recognition result, the audio recognition result, and the radar recognition result according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous preset number of moments of the current moment; Calculating the target weights of the image recognition result, the audio recognition result, and the radar recognition result according to the first weights and the second weights of the image recognition result, the audio recognition result, and the radar recognition result; The calculating the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the first weights of the image recognition result, the audio recognition result, and the radar recognition result includes: Calculating the target scores of the image recognition result, the audio recognition result, and the radar recognition result according to the initial scores and the target weights of the image recognition result, the audio recognition result, and the radar recognition result.

7. The method according to any one of claims 1 to 6, characterized in that The method further includes: If the target recognition result indicates that there is a special vehicle in the vehicle body environment, determining whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment; If the special vehicle has an overtaking intention, controlling the vehicle to avoid.

8. The method according to claim 7, wherein The determining whether the special vehicle has an overtaking intention according to the image recognition result, the audio recognition result, and the radar recognition result, and the historical image recognition result, the historical audio recognition result, and the historical radar recognition result at the previous moment of the current moment includes: If the special vehicle position is included in the image recognition result, determining whether the special vehicle has an overtaking intention according to the image recognition result and the historical image recognition result; If the special vehicle position is not included in the image recognition result and the special vehicle position is included in the radar recognition result, determining whether the special vehicle has an overtaking intention according to the radar recognition result and the historical radar recognition result; If the special vehicle position is not included in both the image recognition result and the radar recognition result and the special vehicle position is included in the audio recognition result, determining whether the special vehicle has an overtaking intention according to the audio recognition result and the historical audio recognition result.

9. A special vehicle identification device, characterized in that, Including: An acquisition module, configured to acquire weather data, vehicle body environment audio data, vehicle body environment images, and vehicle body environment radar data at the current moment; An identification module, configured to: Perform special vehicle identification processing on the vehicle body environment image according to the current moment to obtain an image recognition result; Perform special vehicle recognition processing on the vehicle body environmental audio data and the vehicle body environmental radar data respectively to obtain an audio recognition result and a radar recognition result; A fusion module, configured to determine a target recognition result according to the current moment, the weather data, and the result indication identifiers and recognition confidence levels in the image recognition result, the audio recognition result, and the radar recognition result, where the result indication identifier and the target recognition result are both used to indicate whether there is a special vehicle in the vehicle body environment.

10. An electronic device, characterized in that, Comprising: A processor, a memory, and a communication interface; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the special vehicle recognition method according to any one of claims 1 to 8 by executing the executable instructions.

11. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the special vehicle recognition method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that, Comprising a computer program, which is used to implement the special vehicle recognition method according to any one of claims 1 to 8 when executed by the processor.

Images