Method and device for identifying collapse risk of road in front of vehicle and computer equipment

Abstract

The invention relates to a collapse risk identification method and device for a road in front of a vehicle, computer equipment, the vehicle and a storage medium. The method comprises the following steps: acquiring projection light spot feature data and radar return feature data of a vehicle; wherein the projection light spot feature data is feature data obtained after recognition processing is conducted on projection light spots, on the road in front of the vehicle, of the vehicle matrix headlamps; the radar returned feature data is feature data obtained after the radar original data is subjected to identification processing; in response to the risk identification mode being a risk comprehensive identification mode, performing feature fusion processing on the projection light spot feature data and the radar return feature data to obtain fused feature data; inputting the fused feature data into a pre-trained road collapse risk identification neural network model to obtain a collapse risk identification result of the road in front of the vehicle; wherein the front road collapse risk identification result comprises the existence of collapse or the absence of collapse. By adopting the method, the real-time performance and the accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of safe driving technology, and in particular to a method, device, computer equipment, vehicle and storage medium for identifying the risk of road collapse ahead of a vehicle. Background Art

[0002] In recent years, under the influence of natural and human factors, vehicle road collapse accidents have occurred frequently. Ground collapse is characterized by randomness, suddenness, and serious consequences. At the same time, with the continuous increase in the number of vehicles in cities, once ground collapse occurs, it is easy to cause major property losses and casualties.

[0003] However, traditional methods for identifying the risk of road collapse ahead of a vehicle have problems such as poor real-time performance or low accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, vehicle and storage medium for identifying the risk of road collapse ahead of a vehicle, which can improve real-time performance and accuracy in response to the above technical problems.

[0005] In a first aspect, a method for identifying a road collapse risk ahead of a vehicle is provided, the method comprising:

[0006] Acquire the projection light spot feature data and radar return feature data of the vehicle; wherein the projection light spot feature data is feature data obtained after identifying and processing the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; the radar return feature data is feature data obtained after identifying and processing the radar raw data;

[0007] In response to the risk identification mode being the comprehensive risk identification mode, feature fusion processing is performed on the projected light spot feature data and the radar returned feature data to obtain fused feature data;

[0008] The fused feature data is input into a pre-trained road collapse risk identification neural network model to obtain a road collapse risk identification result ahead of the vehicle; wherein the road collapse risk identification result ahead includes whether a collapse exists or not.

[0009] In one of the embodiments, the method also includes: acquiring gyroscope data of the vehicle, and determining the vehicle posture of the vehicle based on the gyroscope data; the vehicle posture includes a special vehicle posture or a non-special vehicle posture; the special vehicle posture includes a climbing posture and a turning posture; in response to the vehicle posture being a special vehicle posture and the result of the identification of the risk of road collapse ahead is that collapse exists, the result of the identification of the risk of road collapse ahead is corrected to that no collapse exists.

[0010] In one of the embodiments, the method also includes: in response to the risk identification mode being the risk identification mode, analyzing the projected light spot feature data to obtain a light spot state identification result; the light spot state identification result includes the existence or disappearance of the light spot or the non-existence or disappearance of the light spot; analyzing the feature data returned by the radar to obtain a front vehicle state identification result; the front vehicle state identification result includes the disappearance of the front vehicle or the non-disappearance of the front vehicle; and determining the front road collapse risk identification result based on the light spot state identification result and the front vehicle state identification result.

[0011] In one of the embodiments, analysis is performed based on the projection light spot feature data to obtain a light spot state recognition result, including: acquiring initial projection light spot feature data of the vehicle matrix headlights; calculating based on the initial projection light spot feature data and the projection light spot feature data to obtain the projection light spot disappearance ratio; if the projection light spot disappearance ratio is greater than a ratio threshold, determining the light spot state recognition result as the light spot existence and disappearance; if the projection light spot disappearance ratio is less than or equal to the ratio threshold, determining the light spot state recognition result as the light spot non-existence and disappearance.

[0012] In one of the embodiments, analysis is performed based on the characteristic data returned by the radar to obtain the status recognition result of the vehicle ahead, including: obtaining the historical radar raw data of the previous frame fed back by the vehicle's millimeter-wave radar; and performing comparative analysis based on the historical radar raw data and the characteristic data returned by the radar to obtain the status recognition result of the vehicle ahead.

[0013] In one of the embodiments, a road ahead collapse risk identification result is determined based on a light spot state identification result and a front vehicle state identification result, including: in response to a light spot state identification result being that the light spot exists or disappears and a front vehicle state identification result being that the front vehicle disappears, determining the road ahead collapse risk identification result as that a collapse exists; in response to a light spot state identification result being that the light spot does not exist or disappears or a front vehicle state identification result being that the front vehicle does not disappear, determining the road ahead collapse risk identification result as that a collapse does not exist.

[0014] In one of the embodiments, the method also includes: obtaining a driving mode of the vehicle; the driving mode includes an automatic driving mode or a manual driving mode; in response to the driving mode being an automatic driving mode and the result of identifying the risk of road collapse ahead is a collapse, controlling the braking system to perform emergency braking; in response to the driving mode being a manual driving mode and the result of identifying the risk of road collapse ahead is a collapse, outputting warning information about the risk of road collapse ahead.

[0015] In one embodiment, the method further comprises:

[0016] Acquire a preset number of historical projection light spot feature data and corresponding historical radar return feature data; perform feature fusion processing and random division processing on each historical projection light spot feature data and the corresponding historical radar return feature data to generate a training sample set and a test sample set; train a preset road collapse risk identification neural network initial model according to the training sample set, and test the road collapse risk identification neural network initial model according to the test sample set, adjust the model parameters of the road collapse risk identification neural network initial model based on the indicators obtained from the training and testing, until the indicators meet the preset requirements, generate a road collapse risk identification neural network model, and output the road collapse risk identification result ahead based on the road collapse risk identification neural network model.

[0017] In a second aspect, a device for identifying the risk of road collapse ahead of a vehicle is provided, the device comprising a data acquisition module, a feature fusion module and a risk identification module.

[0018] Among them, the data acquisition module is used to obtain the vehicle's projection light spot feature data and radar return feature data; wherein, the projection light spot feature data is the feature data obtained after identifying and processing the projection light spots of the vehicle's matrix headlights on the road in front of the vehicle; the radar return feature data is the feature data after identifying and processing the radar original data; the feature fusion module is used to respond to the risk identification mode being the risk comprehensive identification mode, perform feature fusion processing on the projection light spot feature data and the radar return feature data, and obtain the fused feature data; the risk identification module is used to input the fused feature data into a pre-trained road collapse risk identification neural network model to obtain the vehicle's front road collapse risk identification result; wherein the front road collapse risk identification result includes the presence or absence of collapse.

[0019] In a third aspect, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of any method in the above method embodiments are implemented.

[0020] In a fourth aspect, a vehicle is provided, comprising: a vehicle matrix headlight for projecting an initial projection light spot; wherein the initial projection light spot is determined according to the vehicle speed and a preset safety distance; a vehicle sensing device for collecting the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; a millimeter-wave radar for feeding back radar raw data; a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the steps of any one of the methods in the first aspect embodiment are implemented; wherein the processor is connected to the vehicle matrix headlight, the vehicle sensing device, and the millimeter-wave radar.

[0021] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method in the above method embodiments are implemented.

[0022] The above-mentioned method, device, computer equipment, vehicle and storage medium for identifying the risk of collapse of the road ahead of the vehicle obtain the projection light spot feature data and radar return feature data of the vehicle; wherein, the projection light spot feature data is the feature data obtained after identifying and processing the projection light spots of the vehicle matrix headlights on the road ahead of the vehicle; the radar return feature data is the feature data after identifying and processing the radar original data; then, in response to the risk identification mode being the risk comprehensive identification mode, feature fusion processing is performed on the projection light spot feature data and the radar return feature data to obtain fused feature data; then, the fused feature data is input into a pre-trained road collapse risk identification neural network model to obtain the vehicle's road collapse risk identification result; wherein, the road collapse risk identification result includes the presence or absence of collapse, thereby realizing the real-time identification of the risk of collapse of the road ahead of the vehicle, improving the real-time and accuracy of risk identification and driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A diagram showing an application environment of a method for identifying a risk of road collapse ahead of a vehicle in one embodiment;

[0024] Figure 2 Schematic diagram of a first process of a method for identifying a risk of road collapse ahead of a vehicle in one embodiment;

[0025] Figure 3 A schematic diagram of a second process of a method for identifying a risk of road collapse ahead of a vehicle in one embodiment;

[0026] Figure 4 A schematic diagram of a process for analyzing the projection light spot feature data to obtain the light spot state recognition result in one embodiment;

[0027] Figure 5 A schematic diagram of a process for analyzing characteristic data sent back by a radar to obtain a state recognition result of a vehicle ahead in one embodiment;

[0028] Figure 6 A schematic diagram of a process for determining a forward road collapse risk identification result according to a light spot state identification result and a forward vehicle state identification result in an embodiment;

[0029] Figure 7 A third flow chart of a method for identifying a risk of road collapse ahead of a vehicle in another embodiment;

[0030] Figure 8A fourth flow chart of a method for identifying a risk of road collapse ahead of a vehicle in another embodiment;

[0031] Fig. 9 A fifth flow chart of a method for identifying a risk of road collapse ahead of a vehicle in another embodiment;

[0032] Fig.10 is a structural block diagram of a device for identifying the risk of road collapse ahead of a vehicle in one embodiment;

[0033] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0038] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first resistor may be referred to as a second resistor, and similarly, a second resistor may be referred to as a first resistor. Both the first resistor and the second resistor are resistors, but they are not the same resistor.

[0039] It can be understood that the “connection” in the following embodiments should be understood as “electrical connection”, “communication connection”, etc. if the connected circuits, modules, units, etc. have electrical signals or data transmission between each other.

[0040] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof.

[0041] The method for identifying the risk of road collapse ahead of a vehicle provided in this application can be applied to Figure 1 In the application environment shown in the figure, Figure 1 The vehicle in the embodiment includes a vehicle matrix headlight 100, a vehicle sensing device 200, a millimeter wave radar 300, and a computer device 400. The vehicle matrix headlight 100 is used to project an initial projection light spot; wherein the initial projection light spot is determined according to the vehicle speed and the preset safety distance; the vehicle sensing device 200 is used to collect the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; the millimeter wave radar 300 is used to feed back radar raw data; the computer device 400 includes a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the steps of any one of the methods in the first aspect embodiment are implemented; wherein the processor is connected to the vehicle matrix headlight 100, the vehicle sensing device 200, and the millimeter wave radar 300. The above are only specific examples, which are flexibly set according to user needs in actual applications and are not limited here.

[0042] In one embodiment, Figure 2 As shown, a method for identifying the risk of road collapse ahead of a vehicle is provided, and the method is applied to Figure 1 Taking the processor of the computer device 400 in FIG. 1 as an example, the method includes the following steps: step 201 to step 203.

[0043] Step 201, obtaining the projection light spot feature data and radar return feature data of the vehicle.

[0044] The projection light spot feature data is feature data obtained after identifying and processing the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; the radar return feature data is feature data obtained after identifying and processing the radar raw data. Specifically, the processor of the computer device 400 can obtain the projection light spot feature data and the radar return feature data of the vehicle.

[0045] In a specific example, obtaining the projection light spot feature data and the radar return feature data of the vehicle includes: obtaining the projection light spot collected by the vehicle sensing device of the vehicle; and performing recognition processing on the projection light spot to obtain the projection light spot feature data. The above is only a specific example, which can be flexibly set according to user needs in actual applications and is not limited here.

[0046] In a specific example, obtaining the projection light spot feature data and radar return feature data of the vehicle includes: obtaining the radar raw data fed back by the vehicle's millimeter wave radar; and performing data processing on the radar raw data to obtain the radar return feature data. The above is only a specific example, which can be flexibly set according to user needs in actual applications and is not limited here.

[0047] Step 202, in response to the risk identification mode being the comprehensive risk identification mode, feature fusion processing is performed on the projection light spot feature data and the radar return feature data to obtain fused feature data.

[0048] The risk identification mode includes a comprehensive risk identification mode and a separate risk identification mode. Specifically, when the processor of the computer device 400 identifies that the risk identification mode is the comprehensive risk identification mode, it performs feature fusion processing on the projected light spot feature data and the radar return feature data to obtain fused feature data.

[0049] In one embodiment, if Figure 3 As shown, the method further includes steps 301 to 303.

[0050] Step 301, in response to the risk identification mode being the risk identification mode, analysis is performed based on the projection light spot feature data to obtain a light spot state identification result.

[0051] Among them, the light spot state recognition result includes the existence and disappearance of light spots or the non-existence and disappearance of light spots. The existence and disappearance of light spots indicates that some or all of the projected light spots of the vehicle matrix headlights on the road ahead of the vehicle disappear relative to the initial projection light spots of the vehicle matrix headlights. The non-existence and disappearance of light spots indicates that some or all of the projected light spots of the vehicle matrix headlights on the road ahead of the vehicle do not disappear relative to the initial projection light spots of the vehicle matrix headlights. It can be understood that when the light spot state recognition result is the existence and disappearance of light spots, there is a possibility of the risk of collapse of the road ahead of the vehicle. Specifically, when the processor of the computer device 400 identifies that the risk identification mode is the risk identification mode, it analyzes the projected light spot feature data to obtain the light spot state recognition result.

[0052] In one embodiment, if Figure 4 As shown, the light spot state recognition result is obtained by analyzing the projection light spot feature data, including steps 401 to 404.

[0053] Step 401, obtaining initial projection light spot feature data of the vehicle matrix headlight;

[0054] Step 402, calculating according to the initial projection light spot characteristic data and the projection light spot characteristic data to obtain the projection light spot disappearance ratio;

[0055] Step 403, if the proportion of the projected light spot disappearing is greater than the proportion threshold, the light spot state recognition result is determined as the light spot existing and disappearing;

[0056] Step 404: If the proportion of the projected light spot disappearing is less than or equal to the proportion threshold, the light spot state recognition result is determined as the light spot disappearing without existing.

[0057] Specifically, the processor of the computer device 400 obtains the initial projected light spot feature data of the vehicle matrix headlights; then, calculations are performed based on the initial projected light spot feature data and the projected light spot feature data to obtain the projection light spot disappearance ratio; then, if the projection light spot disappearance ratio is greater than the ratio threshold, the light spot state recognition result is determined as the light spot existence and disappearance; at the same time, if the projection light spot disappearance ratio is less than or equal to the ratio threshold, the light spot state recognition result is determined as the light spot non-existence and disappearance, thereby improving the efficiency and convenience of determining the light spot state recognition result.

[0058] In this embodiment, the initial projected light spot feature data of the vehicle matrix headlights are obtained; then, the projection light spot disappearance ratio is calculated based on the initial projected light spot feature data and the projection light spot feature data; then, if the projection light spot disappearance ratio is greater than the ratio threshold, the light spot state recognition result is determined as the light spot existence and disappearance; at the same time, if the projection light spot disappearance ratio is less than or equal to the ratio threshold, the light spot state recognition result is determined as the light spot non-existence and disappearance, which improves the efficiency and convenience of determining the light spot state recognition result.

[0059] Step 302, analyzing the characteristic data returned by the radar to obtain the state recognition result of the vehicle ahead.

[0060] The result of the state recognition of the vehicle ahead includes the disappearance of the vehicle ahead or the non-disappearance of the vehicle ahead. The disappearance of the vehicle ahead indicates that the vehicle ahead of the vehicle traveling on the road ahead of the vehicle has disappeared. The non-disappearance of the vehicle ahead indicates that the vehicle ahead of the vehicle traveling on the road ahead of the vehicle has not disappeared. It is understandable that when the result of the state recognition of the vehicle ahead is the disappearance of the vehicle ahead, there is a possibility of a risk of collapse of the road ahead of the vehicle. Specifically, the processor of the computer device 400 analyzes the characteristic data returned by the radar to obtain the result of the state recognition of the vehicle ahead.

[0061] In one embodiment, if Figure 5As shown, the characteristic data sent back by the radar is analyzed to obtain the state recognition result of the vehicle in front, including steps 501 to 502.

[0062] Step 501, obtaining the previous frame of historical radar raw data fed back by the vehicle's millimeter-wave radar.

[0063] Step 502 , performing comparative analysis based on historical radar raw data and radar return feature data to obtain a forward vehicle state recognition result.

[0064] Specifically, the processor of the computer device 400 obtains the historical radar raw data of the previous frame fed back by the vehicle's millimeter-wave radar; then, based on the comparative analysis of the historical radar raw data and the characteristic data returned by the radar, the state recognition result of the vehicle ahead is obtained, thereby improving the efficiency and convenience of obtaining the state recognition result of the vehicle ahead.

[0065] In this embodiment, the historical radar raw data of the previous frame fed back by the vehicle's millimeter-wave radar is obtained; then, a comparative analysis is performed based on the historical radar raw data and the characteristic data returned by the radar to obtain the status recognition result of the vehicle ahead, thereby improving the efficiency and convenience of obtaining the status recognition result of the vehicle ahead.

[0066] Step 303, determining a forward road collapse risk identification result based on the light spot state identification result and the forward vehicle state identification result.

[0067] Specifically, the processor of the computer device 400 may determine the forward road collapse risk identification result based on the light spot state identification result and the forward vehicle state identification result.

[0068] In one embodiment, if Figure 6 As shown, the front road collapse risk identification result is determined according to the light spot state identification result and the front vehicle state identification result, including steps 601 to 602.

[0069] Step 601, in response to the light spot state recognition result being the light spot existence or disappearance and the front vehicle state recognition result being the front vehicle disappearance, determining the front road collapse risk recognition result as the existence of collapse.

[0070] Step 602: In response to the light spot state recognition result being that the light spot has not disappeared or the front vehicle state recognition result being that the front vehicle has not disappeared, the front road collapse risk recognition result is determined as that there is no collapse.

[0071] Specifically, when the processor of the computer device 400 identifies that the light spot state recognition result is that the light spot exists or disappears and the front vehicle state recognition result is that the front vehicle disappears, it determines the road ahead collapse risk recognition result as the existence of collapse; and, when it identifies that the light spot state recognition result is that the light spot does not disappear or the front vehicle state recognition result is that the front vehicle does not disappear, it determines the road ahead collapse risk recognition result as the absence of collapse, thereby improving the efficiency and convenience of determining the road ahead collapse risk recognition result.

[0072] In this embodiment, in response to the light spot status recognition result being that the light spot exists or disappears and the front vehicle status recognition result being that the front vehicle disappears, the road ahead collapse risk recognition result is determined as the existence of collapse; and, in response to the light spot status recognition result being that the light spot does not disappear or the front vehicle status recognition result being that the front vehicle does not disappear, the road ahead collapse risk recognition result is determined as the absence of collapse, thereby improving the efficiency and convenience of determining the road ahead collapse risk recognition result.

[0073] In this embodiment, in response to the risk identification mode being the risk identification mode, analysis is performed based on the projected light spot feature data to obtain a light spot state identification result; the light spot state identification result includes the existence or disappearance of the light spot or the non-existence or disappearance of the light spot; then, analysis is performed based on the feature data returned by the radar to obtain a front vehicle state identification result; the front vehicle state identification result includes the disappearance of the front vehicle or the non-disappearance of the front vehicle; then, the front road collapse risk identification result is determined based on the light spot state identification result and the front vehicle state identification result, thereby improving the real-time and accuracy of risk identification.

[0074] Step 203 , input the fused feature data into a pre-trained road collapse risk identification neural network model to obtain a road collapse risk identification result ahead of the vehicle.

[0075] The identification result of the road collapse risk ahead includes the presence of collapse or the absence of collapse. Specifically, the processor of the computer device 400 inputs the fused feature data into a pre-trained road collapse risk identification neural network model to obtain the identification result of the road collapse risk ahead of the vehicle, thereby achieving the ability to identify the road collapse risk ahead of the vehicle in real time, improving the real-time and accuracy of risk identification and driving safety.

[0076] In one embodiment, if Figure 7 As shown, the method also includes steps 701 to 703.

[0077] Step 701, obtaining a preset number of historical projection light point feature data and corresponding historical radar return feature data.

[0078] Step 702 , performing feature fusion processing and random division processing on each historical projection light point feature data and the corresponding historical radar return feature data to generate a training sample set and a test sample set.

[0079] Step 703, train the preset road collapse risk identification neural network initial model according to the training sample set, and test the road collapse risk identification neural network initial model according to the test sample set, adjust the model parameters of the road collapse risk identification neural network initial model based on the indicators obtained from the training and testing, until the indicators meet the preset requirements, generate a road collapse risk identification neural network model, and output the road collapse risk identification result ahead based on the road collapse risk identification neural network model.

[0080] Specifically, the processor of the computer device 400 obtains a preset number of historical projection light spot feature data and corresponding historical radar return feature data; then, it performs feature fusion processing and random division processing on each historical projection light spot feature data and the corresponding historical radar return feature data to generate a training sample set and a test sample set; then, the preset road collapse risk identification neural network initial model is trained according to the training sample set, and the road collapse risk identification neural network initial model is tested according to the test sample set, and the model parameters of the road collapse risk identification neural network initial model are adjusted based on the indicators obtained from the training and testing until the indicators meet the preset requirements, and a road collapse risk identification neural network model is generated, so as to output the road collapse risk identification result ahead based on the road collapse risk identification neural network model, thereby improving the efficiency and convenience of generating a road collapse risk identification neural network model, and being beneficial to improving the real-time and accuracy of risk identification as well as driving safety.

[0081] In this embodiment, a preset number of historical projection light spot feature data and corresponding historical radar return feature data are obtained; then, feature fusion processing and random division processing are performed on each historical projection light spot feature data and the corresponding historical radar return feature data to generate a training sample set and a test sample set; then, the preset road collapse risk identification neural network initial model is trained according to the training sample set, and the road collapse risk identification neural network initial model is tested according to the test sample set, and the model parameters of the road collapse risk identification neural network initial model are adjusted based on the indicators obtained from the training and testing until the indicators meet the preset requirements, and a road collapse risk identification neural network model is generated, so as to output the road collapse risk identification result ahead based on the road collapse risk identification neural network model, thereby improving the efficiency and convenience of generating the road collapse risk identification neural network model, and being beneficial to improving the real-time and accuracy of risk identification as well as driving safety.

[0082] Based on this, the above-mentioned method for identifying the risk of road collapse ahead of the vehicle obtains the projection light spot feature data and radar return feature data of the vehicle; wherein, the projection light spot feature data is the feature data obtained after identifying and processing the projection light spots of the vehicle matrix headlights on the road ahead of the vehicle; the radar return feature data is the feature data after identifying and processing the radar original data; then, in response to the risk identification mode being the risk comprehensive identification mode, feature fusion processing is performed on the projection light spot feature data and the radar return feature data to obtain fused feature data; then, the fused feature data is input into a pre-trained road collapse risk identification neural network model to obtain the vehicle's road collapse risk identification result; wherein, the road collapse risk identification result includes the presence or absence of collapse, thereby realizing the real-time identification of the risk of road collapse ahead of the vehicle, improving the real-time and accuracy of risk identification and driving safety.

[0083] In one embodiment, if Figure 8 As shown, the method further includes step 801 to step 802.

[0084] Step 801, obtaining gyroscope data of the vehicle, and determining the vehicle posture of the vehicle according to the gyroscope data.

[0085] Step 802, in response to the vehicle posture being a special vehicle posture and the front road collapse risk identification result being a collapse, the front road collapse risk identification result is corrected to no collapse.

[0086] The vehicle posture includes a special vehicle posture or a non-special vehicle posture; the special vehicle posture includes a climbing posture and a turning posture. Specifically, the processor of the computer device 400 obtains the gyroscope data of the vehicle and determines the vehicle posture of the vehicle according to the gyroscope data; then, in response to the vehicle posture being a special vehicle posture and the front road collapse risk identification result being a collapse, the front road collapse risk identification result is corrected to no collapse, avoiding risk identification errors caused by the special vehicle posture, thereby further improving the accuracy of risk identification.

[0087] In a specific example, in response to the vehicle posture being a non-special vehicle posture and the road collapse risk identification result being a collapse, the road collapse risk identification result is maintained. The above is only a specific example, which can be flexibly set according to user needs in actual applications and is not limited here.

[0088] In one embodiment, if Fig. 9 As shown, the method also includes steps 901 to 903.

[0089] Step 901, obtaining the driving mode of the vehicle.

[0090] Step 902, in response to the driving mode being the automatic driving mode and the front road collapse risk identification result being a collapse, controlling the brake system to perform emergency braking.

[0091] Step 903: In response to the driving mode being the manual driving mode and the road collapse risk identification result being that collapse exists, outputting road collapse risk warning information.

[0092] The driving mode includes an automatic driving mode or a manual driving mode. The processor of the computer device 400 obtains the driving mode of the vehicle; then, in response to the driving mode being the automatic driving mode and the front road collapse risk identification result being a collapse, the brake system is controlled to perform emergency braking; and, in response to the driving mode being the manual driving mode and the front road collapse risk identification result being a collapse, the front road collapse risk warning information is output, thereby improving the convenience of risk identification and driving safety.

[0093] In this embodiment, the driving mode of the vehicle is acquired; then, in response to the driving mode being the automatic driving mode and the risk identification result of road collapse ahead being a collapse, the braking system is controlled to perform emergency braking; and, in response to the driving mode being the manual driving mode and the risk identification result of road collapse ahead being a collapse, road collapse risk warning information is output, thereby improving the convenience of risk identification and driving safety.

[0094] It should be understood that although Figure 2-9 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-9 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0095] Second, as Fig.10 As shown, a device for identifying the risk of road collapse ahead of a vehicle is provided, and the device includes a data acquisition module 1001, a feature fusion module 1002 and a risk identification module 1003.

[0096] Among them, the data acquisition module 1001 is used to obtain the vehicle's projection light spot feature data and radar return feature data; wherein, the projection light spot feature data is the feature data obtained after identifying and processing the projection light spots of the vehicle matrix headlights on the road in front of the vehicle; the radar return feature data is the feature data after identifying and processing the radar original data; the feature fusion module 1002 is used to respond to the risk identification mode being the risk comprehensive identification mode, to perform feature fusion processing on the projection light spot feature data and the radar return feature data to obtain the fused feature data; the risk identification module 1003 is used to input the fused feature data into a pre-trained road collapse risk identification neural network model to obtain the vehicle's front road collapse risk identification result; wherein the front road collapse risk identification result includes the presence or absence of collapse.

[0097] In one of the embodiments, the device for identifying the risk of road collapse ahead of the vehicle further includes a result correction module.

[0098] Among them, the result correction module is used to obtain the gyroscope data of the vehicle and determine the vehicle posture of the vehicle based on the gyroscope data; the vehicle posture includes a special vehicle posture or a non-special vehicle posture; the special vehicle posture includes a climbing posture and a turning posture; the result correction module is used to respond to the vehicle posture being a special vehicle posture and the result of the road collapse risk identification ahead is that there is a collapse, and correct the road collapse risk identification result ahead to no collapse.

[0099] In one of the embodiments, the risk identification module is used to respond to the risk identification mode being the risk identification mode, analyze the projected light spot feature data, and obtain a light spot state identification result; the light spot state identification result includes the existence or disappearance of the light spot or the non-existence or disappearance of the light spot; the risk identification module is used to analyze the feature data returned by the radar, and obtain a front vehicle state identification result; the front vehicle state identification result includes the disappearance of the front vehicle or the existence of the front vehicle; the risk identification module is used to determine the front road collapse risk identification result based on the light spot state identification result and the front vehicle state identification result.

[0100] In one of the embodiments, the risk identification module includes a light spot state identification result determination unit.

[0101] Among them, the light spot state recognition result determination unit is used to obtain the initial projected light spot feature data of the vehicle matrix headlights; the light spot state recognition result determination unit is used to calculate according to the initial projected light spot feature data and the projected light spot feature data to obtain the projection light spot disappearance ratio; the light spot state recognition result determination unit is used to determine the light spot state recognition result as the existence and disappearance of the light spot if the projection light spot disappearance ratio is greater than the ratio threshold; the light spot state recognition result determination unit is used to determine the light spot state recognition result as the non-existence and disappearance of the light spot if the projection light spot disappearance ratio is less than or equal to the ratio threshold.

[0102] In one of the embodiments, the risk identification module includes a leading vehicle state identification result determination unit.

[0103] Among them, the front vehicle state recognition result determination unit is used to obtain the historical radar raw data of the previous frame fed back by the vehicle's millimeter-wave radar; the front vehicle state recognition result determination unit is used to compare and analyze the historical radar raw data and the radar returned feature data to obtain the front vehicle state recognition result.

[0104] In one of the embodiments, the risk identification module includes a forward road collapse risk identification result determination unit.

[0105] Among them, the unit for determining the result of identification of risk of collapse of the road ahead is used to determine the result of identification of risk of collapse of the road ahead as the existence of collapse in response to the result of identification of the state of the light spot being the existence or disappearance of the light spot and the result of identification of the state of the vehicle ahead being the disappearance of the vehicle ahead; the unit for determining the result of identification of risk of collapse of the road ahead is used to determine the result of identification of risk of collapse of the road ahead as the absence of collapse in response to the result of identification of the state of the light spot being the existence or disappearance of the light spot and the result of identification of the state of the vehicle ahead being the existence of the vehicle ahead.

[0106] In one embodiment, the device further includes a driving control module.

[0107] Among them, the driving control module is used to obtain the driving mode of the vehicle; the driving mode includes an automatic driving mode or a manual driving mode; the driving control module is used to control the braking system to perform emergency braking in response to the driving mode being the automatic driving mode and the risk identification result of road collapse ahead is that there is collapse; the driving control module outputs warning information of the risk of road collapse ahead in response to the driving mode being the manual driving mode and the risk identification result of road collapse ahead is that there is collapse.

[0108] In one of the embodiments, the device also includes a model training module.

[0109] Among them, the model training module is used to obtain a preset number of historical projection light spot feature data and corresponding historical radar return feature data; the model training module is used to perform feature fusion processing and random division processing on each historical projection light spot feature data and the corresponding historical radar return feature data to generate a training sample set and a test sample set; the model training module is used to train the preset road collapse risk identification neural network initial model according to the training sample set, and test the road collapse risk identification neural network initial model according to the test sample set, adjust the model parameters of the road collapse risk identification neural network initial model based on the indicators obtained from training and testing, until the indicators meet the preset requirements, generate a road collapse risk identification neural network model, and output the road collapse risk identification result ahead based on the road collapse risk identification neural network model.

[0110] For the specific definition of the device for identifying the risk of collapse of the road ahead of the vehicle, please refer to the definition of the method for identifying the risk of collapse of the road ahead of the vehicle mentioned above, which will not be repeated here. Each module in the above-mentioned device for identifying the risk of collapse of the road ahead of the vehicle can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0111] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying the risk of road collapse ahead of a vehicle is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0112] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] In a third aspect, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of any method in the above method embodiments are implemented.

[0114] In a fourth aspect, a vehicle is provided, such as Figure 1As shown, the vehicle includes a vehicle matrix headlight 100, a vehicle perception device 200, a millimeter wave radar 300 and a computer device 400. The vehicle matrix headlight 100 is used to project an initial projection light spot; wherein the initial projection light spot is determined according to the vehicle speed and the preset safety distance; the vehicle perception device 200 is used to collect the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; the millimeter wave radar 300 is used to feed back radar raw data; the computer device 400 includes a memory, a processor and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the steps of any one of the methods in the first aspect embodiment are implemented; wherein the processor is connected to the vehicle matrix headlight 100, the vehicle perception device 200 and the millimeter wave radar 300.

[0115] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method in the above method embodiments are implemented.

[0116] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying the risk of road collapse ahead of a vehicle, the method comprising: Acquire the projection light spot feature data and radar return feature data of the vehicle; wherein the projection light spot feature data is feature data obtained after identifying and processing the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; the radar return feature data is feature data obtained after identifying and processing the radar raw data; In response to the risk identification mode being the comprehensive risk identification mode, performing feature fusion processing on the projection light spot feature data and the radar returned feature data to obtain fused feature data; The fused feature data is input into a pre-trained road collapse risk identification neural network model to obtain a road collapse risk identification result ahead of the vehicle; wherein the road collapse risk identification result ahead includes whether a collapse exists or does not exist.

2. The method according to claim 1, characterized in that The method further comprises: Acquire gyroscope data of the vehicle, and determine the vehicle posture of the vehicle according to the gyroscope data; the vehicle posture includes a special vehicle posture or a non-special vehicle posture; the special vehicle posture includes a climbing posture and a turning posture; In response to the vehicle posture being the special vehicle posture and the forward road collapse risk identification result being the presence of collapse, the forward road collapse risk identification result is corrected to the absence of collapse.

3. The method according to claim 1, characterized in that The method further comprises: In response to the risk identification mode being the risk identification mode, analyzing the projection light spot feature data to obtain a light spot state identification result; the light spot state identification result includes the existence or disappearance of the light spot or the non-existence or disappearance of the light spot; Analyzing the characteristic data returned by the radar to obtain a state recognition result of the vehicle ahead; the state recognition result of the vehicle ahead includes that the vehicle ahead has disappeared or that the vehicle ahead has not disappeared; The forward road collapse risk identification result is determined according to the light spot state identification result and the forward vehicle state identification result.

4. The method according to claim 3, characterized in that The step of analyzing the projection light spot feature data to obtain a light spot state recognition result includes: Acquiring initial projection light spot feature data of the vehicle matrix headlight; Calculating according to the initial projection light spot characteristic data and the projection light spot characteristic data to obtain a projection light spot disappearance ratio; If the disappearance ratio of the projection light spot is greater than the ratio threshold, the light spot state recognition result is determined as the existence and disappearance of the light spot; If the disappearance ratio of the projection light spot is less than or equal to the ratio threshold, the light spot state recognition result is determined as the light spot not existing and disappearing.

5. The method according to claim 3, characterized in that: The analyzing the characteristic data returned by the radar to obtain the state recognition result of the vehicle ahead includes: Obtaining the previous frame of historical radar raw data fed back by the millimeter-wave radar of the vehicle; The front vehicle state recognition result is obtained by comparing and analyzing the historical radar raw data and the radar returned feature data.

6. The method according to claim 3, characterized in that The determining the front road collapse risk identification result according to the light spot state identification result and the front vehicle state identification result includes: In response to the light spot state recognition result being that the light spot exists or disappears and the front vehicle state recognition result being that the front vehicle disappears, determining the front road collapse risk recognition result as the existence of collapse; In response to the light spot state recognition result being that the light spot has not disappeared or the front vehicle state recognition result being that the front vehicle has not disappeared, the front road collapse risk recognition result is determined as that the road collapse has not existed.

7. The method according to claim 1, characterized in that The method further comprises: Acquiring a driving mode of the vehicle; the driving mode includes an automatic driving mode or a manual driving mode; In response to the driving mode being the automatic driving mode and the front road collapse risk identification result being the existence of collapse, controlling the brake system to perform emergency braking; In response to the driving mode being the manual driving mode and the forward road collapse risk identification result being the existence of collapse, forward road collapse risk warning information is output.

8. The method according to claim 1, characterized in that The method further comprises: Obtain a preset number of historical projection light spot feature data and corresponding historical radar return feature data; Performing feature fusion processing and random division processing on each of the historical projection light point feature data and the corresponding historical radar return feature data to generate a training sample set and a test sample set; The preset road collapse risk identification neural network initial model is trained according to the training sample set, and the road collapse risk identification neural network initial model is tested according to the test sample set, and the model parameters of the road collapse risk identification neural network initial model are adjusted based on the indicators obtained from the training and testing until the indicators meet the preset requirements, and the road collapse risk identification neural network model is generated to output the forward road collapse risk identification result based on the road collapse risk identification neural network model.

9. A device for identifying the risk of road collapse ahead of a vehicle, characterized in that: The device comprises: A data acquisition module, used to acquire the projection light spot feature data and radar return feature data of the vehicle; wherein the projection light spot feature data is feature data obtained after identification and processing of the projection light spot of the vehicle matrix headlight on the road in front of the vehicle; the radar return feature data is feature data obtained after identification and processing of the radar raw data; a feature fusion module, configured to perform feature fusion processing on the projection light spot feature data and the radar return feature data in response to the risk identification mode being the comprehensive risk identification mode, so as to obtain fused feature data; The risk identification module is used to input the fused feature data into a pre-trained road collapse risk identification neural network model to obtain a road collapse risk identification result ahead of the vehicle; wherein the road collapse risk identification result ahead includes whether a collapse exists or does not exist.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A vehicle, characterized in that: include: A vehicle matrix headlight, used to project an initial projection light spot; wherein the initial projection light spot is determined according to the vehicle speed and a preset safety distance; A vehicle sensing device, used to collect the projection light spots of the vehicle matrix headlights on the road in front of the vehicle; Millimeter wave radar, used to feed back raw radar data; A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; the processor implements the steps of the method described in any one of claims 1 to 8 when executing the computer program; wherein the processor is connected to the vehicle matrix headlights, the vehicle sensing device, and the millimeter wave radar.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images