Automobile collision detection method and device, medium and equipment

Abstract

The invention provides an automobile collision detection method and device, a medium and equipment. The method comprises the following steps: acquiring audio data and image data of a target vehicle; based on the audio data, determining a preliminary collision detection result of the target vehicle; if the preliminary collision detection result is that collision occurs or a collision risk exists, comprehensively determining a final collision detection result in combination with the audio data and the image data; a sensor carried on a target vehicle is used for collecting audio data and image data of the target vehicle, whether the target vehicle is collided or has a collision risk is preliminarily judged according to the audio data, and if yes, whether the target vehicle is collided or has the collision risk is finally determined by further combining the audio data and the image data. In other words, after the collision is preliminarily determined by the audio data, final determination is performed in combination with the multi-modal data, so that only the audio data is calculated when no collision exists, the calculation amount is reduced as much as possible, and the collision is comprehensively detected by using the multi-modal data to ensure the accuracy of collision detection.

Description

Technical Field

[0001] The present application relates to the technical field of automobile collision detection, and in particular to an automobile collision detection method, device, medium and equipment. Background Art

[0002] As the number of cars continues to increase, the number of traffic accidents also increases. Especially in serious traffic accidents, passengers may be unable to call the police or seek help on their own due to injury or fright, resulting in delayed rescue, which is obviously not conducive to rapid rescue of traffic accidents.

[0003] With the continuous development of intelligent automobile technology, more and more cars are equipped with multiple sensors to realize intelligent driving functions. Based on these sensors, many traffic accidents can also be detected. For example, vehicle collisions are common traffic accidents. When they occur, they are accompanied by various data information such as violent vehicle vibrations, loud noises, and visual contact. Therefore, more and more intelligent driving cars are also beginning to be equipped with collision detection or recognition functions to determine whether the car has collided, thereby improving the safety of drivers and passengers and the timeliness of rescue.

[0004] However, most current automotive collision detection systems rely on single-modality sensor data, such as visual or audio signal detection. This can lead to misjudgments due to errors or anomalies in the single sensor data. Furthermore, inaccurate sensor data in specific scenarios (such as visual signal detection in low light conditions or audio signal detection in noisy environments) can also lead to inaccurate detection results. Therefore, a method is needed to improve the accuracy of automotive collision detection. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a method, apparatus, medium and device for detecting a vehicle collision.

[0006] According to one aspect of the present application, a car collision detection method is provided, comprising: collecting audio data and image data of a target vehicle; determining a preliminary collision detection result of the target vehicle based on the audio data; and if the preliminary collision detection result indicates that the target vehicle has collided or is at risk of a collision, comprehensively determining a final collision detection result of the target vehicle by combining the audio data and the image data.

[0007] In one embodiment, determining the preliminary collision detection result of the target vehicle based on the audio data includes: performing a wavelet transform on the audio data to obtain a time-frequency image; extracting audio features from the time-frequency image; and determining the preliminary collision detection result of the target vehicle based on the audio features.

[0008] In an embodiment, the determining the preliminary collision detection result of the target vehicle based on the audio feature comprises: calculating a probability value of the audio feature appearing in a collision category; and determining that the preliminary collision detection result is that the target vehicle has a collision or a collision risk if the probability value is greater than a preset probability threshold.

[0009] In an embodiment, the determining the final collision detection result of the target vehicle based on the audio data and the image data comprises: performing transformation processing on the image data to obtain a feature image of the target vehicle; and determining the final collision detection result of the target vehicle based on the time-frequency image and the feature image.

[0010] In an embodiment, the image data comprises video data; the transformation processing on the image data to obtain the feature image of the target vehicle comprises: performing transformation processing on the video data to obtain an RGB image, a difference image and an optical flow map of the target vehicle; the RGB image represents color values of red, blue and green color channels in the video data, the difference image represents difference values between adjacent frames in the video data, and the optical flow map represents object motion information between different frames in the video data.

[0011] In an embodiment, the image data comprises point cloud data; the transformation processing on the image data to obtain the feature image of the target vehicle comprises: performing transformation processing on the point cloud data to obtain a depth map and a projection map of the target vehicle; the depth map represents distances between point clouds and a point cloud sensor, and the projection map represents projection values of the point cloud data on a two-dimensional plane.

[0012] In an embodiment, the determining the final collision detection result of the target vehicle based on the time-frequency image and the feature image comprises: extracting image features in the feature image; and determining the final collision detection result of the target vehicle based on the audio feature and the image feature.

[0013] According to another aspect of the present application, there is provided a vehicle collision detection device, comprising: a data acquisition module configured to acquire audio data and image data of a target vehicle; a preliminary detection module configured to determine a preliminary collision detection result of the target vehicle based on the audio data; and a final detection module configured to determine a final collision detection result of the target vehicle based on the audio data and the image data if the preliminary collision detection result is that the target vehicle has a collision or a collision risk.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute any of the above methods.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and the processor for executing any of the above methods.

[0016] The present application provides a method, device, medium and equipment for automobile collision detection, which collect audio data and image data of a target vehicle; determine a preliminary collision detection result of the target vehicle based on the audio data; if the preliminary collision detection result is that the target vehicle has collided or there is a risk of collision, then comprehensively determine the final collision detection result of the target vehicle by combining the audio data and the image data; use a sensor mounted on the target vehicle to collect audio data and image data of the target vehicle, and first preliminarily determine whether the target vehicle has collided or there is a risk of collision based on the audio data; if so, further combine the audio data and the image data to finally determine whether the target vehicle has collided or there is a risk of collision, that is, after the audio data preliminarily determines the collision, the multimodal data is combined to make the final determination, so that only the audio data is calculated when there is no collision, minimizing the amount of calculation, and using the multimodal data to comprehensively detect collisions to ensure the accuracy of collision detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] 图1 It is a flowchart of a vehicle collision detection method provided by an exemplary embodiment of the present application.

[0019] 图2 It is a logical flow diagram of a vehicle collision detection method provided by an exemplary embodiment of the present application.

[0020] 图3 It is a structural diagram of a network model in a vehicle collision detection method provided by an exemplary embodiment of the present application.

[0021] 图4 It is a structural diagram of an automobile collision detection device provided by an exemplary embodiment of the present application.

[0022] 图5 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0024] 图1 FIG. 1 is a flow chart of a vehicle collision detection method provided by an exemplary embodiment of the present application. 图1 As shown, the vehicle collision detection method includes the following steps: Step 110: Collect audio data and image data of the target vehicle.

[0025] This application can use multiple sensors mounted on the target vehicle to collect real-time data from the target vehicle during its operation. Video data, audio data, and millimeter-wave radar data are data closely related to collisions. Among them, video data can intuitively reflect the dynamic changes of the target vehicle and its surrounding environment, such as the relative speed and movement trajectory between vehicles, providing visual clues for collision detection. Audio data can also detect some visually imperceptible collision signs, such as tire friction or sudden braking sounds, by capturing unique acoustic patterns before and after the collision. Millimeter-wave radar data has the characteristics of strong penetration and good anti-interference ability, and can measure the distance and relative speed between the target vehicle and surrounding objects with high precision under complex weather and lighting conditions. Therefore, this application can use the video data acquisition sensor (camera), audio data acquisition sensor (on-board microphone), and millimeter-wave radar mounted on the target vehicle to respectively collect video data, audio data, and millimeter-wave radar data of the target vehicle as the data basis for subsequent vehicle collisions.

[0026] Step 120: Determine a preliminary collision detection result of the target vehicle based on the audio data.

[0027] This application performs preliminary collision detection on the target vehicle based only on audio data to reduce the computational burden and thus improve the efficiency of collision detection of the target vehicle.

[0028] Step 130: If the preliminary collision detection result indicates that the target vehicle has collided or is at risk of collision, the final collision detection result of the target vehicle is determined comprehensively based on the audio data and the image data.

[0029] If the preliminary collision detection result indicates that the target vehicle has collided or there is a risk of collision, the enhanced judgment phase will be entered by combining the audio data and image data (e.g. 图2 The multi-modal data includes fusion video data, audio data, and millimeter wave radar data, etc. The multi-modal data is comprehensively analyzed and judged to determine the final collision detection result of the target vehicle, thereby improving the accuracy of collision detection. It should be understood that if the preliminary collision detection result is that the target vehicle does not collide and there is no collision risk, it is determined that the target vehicle has no collision risk, and there is no need to enter the enhanced judgment stage.

[0030] The application provides an automobile collision detection method. Audio data and image data of a target vehicle are collected. A preliminary collision detection result of the target vehicle is determined based on the audio data. If the preliminary collision detection result is that the target vehicle collides or has a collision risk, a final collision detection result of the target vehicle is comprehensively determined based on the audio data and the image data. The audio data and the image data of the target vehicle are collected by a sensor carried on the target vehicle. It is preliminarily determined whether the target vehicle collides or has a collision risk based on the audio data. If the target vehicle collides or has a collision risk, the final collision detection result of the target vehicle is determined based on the audio data and the image data. That is, after the audio data preliminarily determines the collision, the final determination is made based on the multi-modal data, so that the audio data is calculated only when there is no collision, the calculation amount is reduced as much as possible, and the multi-modal data is used to comprehensively detect the collision, thereby ensuring the accuracy of collision detection.

[0031] In an embodiment, the specific implementation of the step 120 can be: performing wavelet transform on the audio data to obtain a time-frequency image; extracting an audio feature in the time-frequency image; and determining the preliminary collision detection result of the target vehicle based on the audio feature.

[0032] After the audio data of the target vehicle is collected, the audio data is wavelet transformed to transform one-dimensional audio data (time domain data) into two-dimensional time-frequency images (the horizontal axis is time and the vertical axis is frequency). The following transformation formula can be used: ; Wherein, W(a,b) is the transformed time-frequency image signal, x(t) is the audio signal before transformation, t is time, is a wavelet function, a is a scale parameter for controlling frequency resolution, b is a translation parameter for controlling time resolution.

[0033] After performing wavelet transform on the audio data, the present application extracts the audio features in the time-frequency image and determines the preliminary collision detection result of the target vehicle based on the audio features. The audio features in the time-frequency image can be extracted by inputting the time-frequency image into a preset deep convolutional neural network model. Optionally, the present application can input the time-frequency image into the Inceptionv3 network model (whose structure is as follows 图3 (as shown) to extract audio features from time-frequency images.

[0034] In one embodiment, the specific implementation method of the above step 120 can be: calculating the probability value of the audio feature appearing in the collision category; if the probability value is greater than a preset probability threshold, determining that the preliminary collision detection result is that the target vehicle has collided or there is a collision risk.

[0035] This application extracts the audio features of the time-frequency image F W Then, the probability value of the audio feature appearing in the collision category is calculated. If the probability value is large, it means that the target vehicle is likely to have collided or there is a collision risk. The specific calculation method of the probability value of the audio feature appearing in the collision category can be: ; in, express F W Appears in category C The probability of and They are F W The weight vector and bias of .

[0036] In one embodiment, the specific implementation of step 130 may be: transforming the image data to obtain a characteristic image of the target vehicle; and comprehensively determining a final collision detection result of the target vehicle by combining the time-frequency image and the characteristic image.

[0037] After acquiring the image data of the target vehicle, the present application pre-processes the image data to obtain a characteristic image of the target vehicle. When preliminarily determining that the target vehicle has collided or is at risk of collision based on the audio data, the final collision detection result of the target vehicle is comprehensively determined by combining the time-frequency image and the characteristic image to improve the accuracy of reasonable collision detection of the target vehicle.

[0038] In an embodiment, the image data comprises video data; wherein the step 130 can be implemented by transforming the video data to obtain the RGB image, the difference image and the optical flow map of the target vehicle; wherein the RGB image represents color values of three color channels of red, blue and green in the video data, the difference image represents difference values between adjacent frames in the video data, and the optical flow map represents object motion information between different frames in the video data.

[0039] After the video data of the target vehicle is collected, the video data is transformed to obtain the difference image and the optical flow map of the target vehicle, and then the RGB image contained in the video data is added, that is, the RGB image, the difference image and the optical flow map of the target vehicle are obtained from the image data, so that the image data is analyzed from multiple dimensions of color and brightness, movement or mutation, and object motion of the scene or object, so as to further improve the accuracy of the target vehicle collision detection. The RGB image is represented as: I(x,y,t)= [ R(x,y,t),G(x,y,t),B(x,y,t) ]; wherein, I(x,y,t) represents a pixel value in the RGB image of the i-th frame, t represents a pixel coordinate in the video data of the i-th frame, (x,y,t) and t respectively represent color values of red, green and blue channels of a pixel in the video data of the i-th frame. R(x,y,t)、G(x,y,t)、B(x,y,t) t The difference image is represented as:

[0040] D t (x,y)=I t (x,y)-I t-1 (x,y) ; wherein, I t (x,y) and I t-1 (x,y) respectively represent pixel values of the i-th frame and the j-th frame (corresponding to t and t-1 respectively), I(x,y,t) I(x,y,t-1) D t (x,y) represents a difference value between the i-th frame and the j-th frame. t t-1 The optical flow map is represented as:

[0041] ; wherein, ​​​​​​Indicates the t Pixels in a frame ( x,y ) brightness, Indicates the t The amount of horizontal and vertical motion of pixels in the frame, Indicates the time of change.

[0042] For pixels ( x,y ) has a small displacement in a short period of time , Taylor expansion of its brightness gives the following formula: ; in, M x and M y Respectively represent the brightness value along x and y The spatial gradient of the direction, M t Indicates the rate of change of brightness value over time.

[0043] Based on the above two formulas, the calculation formula for the optical flow image can be obtained: .

[0044] In one embodiment, the image data includes point cloud data; wherein, the specific implementation method of the above-mentioned step 130 may be: transforming the point cloud data to obtain a depth map and a projection map of the target vehicle; wherein, the depth map represents the distance between the point cloud and the point cloud sensor, and the projection map represents the projection value of the point cloud data on a two-dimensional plane.

[0045] This application transforms the point cloud data (3D data) collected by millimeter-wave radar into 2D image data: depth map and projection map, and uses the complementary information of depth map and projection map on the 2D plane to reduce the information loss during the data dimensionality reduction process. The depth map is represented as: ; in, S(x,y) Indicates the depth value, f Indicates the focal length of the camera, b represents the baseline distance, s(x,y) Represents the parallax value, that is, the displacement of the object on the imaging plane.

[0046] The projection diagram is represented as: ; Among them, P 3D Represents 3D point cloud data, P 2D Represents the pixel value of the projection image, K represents the intrinsic parameters of the camera (such as focal length, principal point, etc.), R and t represent the rotation matrix and translation matrix of the camera respectively, Represents the matrix obtained by concatenating the rotation matrix and the translation matrix.

[0047] In one embodiment, the specific implementation of the above step 130 may be: extracting image features from the feature image; and comprehensively determining the final collision detection result of the target vehicle by combining the audio features and the image features.

[0048] After obtaining the RGB image, differential image, optical flow map, depth map and projection image, the present application uses multiple branches to extract features from each feature image to obtain the corresponding image features. For example, the following can be used: 图3 The Inceptionv3 network model shown extracts image features of each feature image, and after extracting the image features, fuses the image features and audio features to obtain fused features. For example, a Kalman filter is used to fuse multiple features to obtain fused features, and the final collision detection result of the target vehicle is determined based on the fused features. Optionally, the present application can bring the fused features into the calculation formula of the probability value of the audio feature appearing in the collision category to calculate the probability value of the fused feature appearing in the collision category, and determine the final collision detection result of the target vehicle based on the set probability threshold.

[0049] Preferably, if the final collision detection result indicates that the target vehicle has collided or is at risk of collision, the target vehicle's onboard alarm device is controlled to emit a collision warning signal, and the final collision detection result is simultaneously displayed to the driver. If the target vehicle loses power (i.e., multiple or most of the target vehicle's onboard signals return to zero or disappear) within a first preset time after the alarm device emits the collision warning signal, the alarm device will emit a higher-level alarm signal (e.g., a higher-frequency, higher-volume alarm). If this higher-level alarm signal is not manually canceled or turned off within a second preset time (e.g., thirty seconds or one minute), the occupants of the target vehicle are deemed to have lost the ability to manually cancel or turn it off, and an emergency rescue request will be immediately sent to a remote platform (e.g., the platform of the company to which the target vehicle belongs or relevant rescue department).

[0050] 图4 FIG. 1 is a schematic diagram of the structure of a vehicle collision detection device provided by an exemplary embodiment of the present application. 图4 As shown, the automobile collision detection device 40 includes: a data acquisition module 41, which is used to collect audio data and image data of the target vehicle; a preliminary detection module 42, which is used to determine the preliminary collision detection result of the target vehicle based on the audio data; and a final detection module 43, which is used to comprehensively determine the final collision detection result of the target vehicle based on the audio data and image data if the preliminary collision detection result is that the target vehicle has collided or there is a risk of collision.

[0051] The present application provides an automobile collision detection device, which collects audio data and image data of a target vehicle through a data acquisition module 41; a preliminary detection module 42 determines a preliminary collision detection result of the target vehicle based on the audio data; if the preliminary collision detection result is that the target vehicle has collided or there is a risk of collision, the final detection module 43 comprehensively determines the final collision detection result of the target vehicle in combination with the audio data and the image data; the audio data and image data of the target vehicle are collected by a sensor mounted on the target vehicle, and a preliminary judgment is first made based on the audio data whether the target vehicle has collided or there is a risk of collision, and if so, the audio data and the image data are further combined to finally determine whether the target vehicle has collided or there is a risk of collision, that is, after the audio data preliminarily determines the collision, the multimodal data is combined to make the final determination, so that only the audio data is calculated when there is no collision, minimizing the amount of calculation, and the multimodal data is used to comprehensively detect the collision to ensure the accuracy of the collision detection.

[0052] In one embodiment, the preliminary detection module 42 may be further configured to: perform wavelet transform on the audio data to obtain a time-frequency image; extract audio features from the time-frequency image; and determine a preliminary collision detection result of the target vehicle based on the audio features.

[0053] In one embodiment, the preliminary detection module 42 may be further configured to calculate the probability value of the audio feature appearing in the collision category; if the probability value is greater than a preset probability threshold, the preliminary collision detection result is determined to be that the target vehicle has collided or there is a risk of collision.

[0054] In one embodiment, the final detection module 43 may be further configured to: transform and process the image data to obtain a characteristic image of the target vehicle; and comprehensively determine a final collision detection result of the target vehicle by combining the time-frequency image and the characteristic image.

[0055] In one embodiment, the image data includes video data; wherein the above-mentioned final detection module 43 can be further configured to: transform and process the video data to obtain an RGB image, a differential image and an optical flow map of the target vehicle; wherein the RGB image represents the color values ​​of the three color channels of red, blue and green in the video data, the differential image represents the difference values ​​between adjacent frames in the video data, and the optical flow map represents the object motion information between different frames in the video data.

[0056] In one embodiment, the image data includes point cloud data; wherein, the above-mentioned final detection module 43 can be further configured to: transform the point cloud data to obtain a depth map and a projection map of the target vehicle; wherein, the depth map represents the distance between the point cloud and the point cloud sensor, and the projection map represents the projection value of the point cloud data on a two-dimensional plane.

[0057] In one embodiment, the final detection module 43 may be further configured to: extract image features from the feature image; and comprehensively determine the final collision detection result of the target vehicle by combining the audio features and the image features.

[0058] Below, reference 图5 The electronic device according to the embodiment of the present application is described. The electronic device may be either or both of the first device and the second device, or a standalone device independent of them, and the standalone device may communicate with the first device and the second device to receive collected input signals from them.

[0059] 图5 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0060] like 图5 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0061] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0062] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0063] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0064] When the electronic device is a stand-alone device, the input device 13 may be a communication network connector, configured to receive collected input signals from the first device and the second device.

[0065] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0066] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0067] Of course, to simplify, 图5 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0068] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0069] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0070] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0071] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0072] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0073] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0074] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0075] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0076] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for detecting a car collision, characterized in that: include: Collect audio data and image data of the target vehicle; determining a preliminary collision detection result of the target vehicle based on the audio data; If the preliminary collision detection result indicates that the target vehicle has collided or is at risk of collision, a final collision detection result of the target vehicle is determined comprehensively based on the audio data and the image data.

2. The vehicle collision detection method according to claim 1, characterized in that: Determining a preliminary collision detection result of the target vehicle based on the audio data includes: Performing wavelet transform on the audio data to obtain a time-frequency image; extracting audio features from the time-frequency image; Based on the audio features, a preliminary collision detection result of the target vehicle is determined.

3. The vehicle collision detection method according to claim 2, characterized in that: Determining a preliminary collision detection result of the target vehicle based on the audio feature includes: Calculating a probability value of the audio feature appearing in a collision category; If the probability value is greater than a preset probability threshold, it is determined that the preliminary collision detection result is that the target vehicle has collided or there is a risk of collision.

4. The vehicle collision detection method according to claim 2, characterized in that: The step of comprehensively determining a final collision detection result of the target vehicle by combining the audio data and the image data includes: Transforming the image data to obtain a characteristic image of the target vehicle; The final collision detection result of the target vehicle is comprehensively determined by combining the time-frequency image and the feature image.

5. The vehicle collision detection method according to claim 4, characterized in that: The image data includes video data; wherein the transforming and processing of the image data to obtain the characteristic image of the target vehicle includes: The video data is transformed and processed to obtain an RGB image, a differential image, and an optical flow map of the target vehicle; wherein the RGB image represents the color values ​​of the red, blue, and green color channels in the video data, the differential image represents the difference values ​​between adjacent frames in the video data, and the optical flow map represents object motion information between different frames in the video data.

6. The vehicle collision detection method according to claim 4, characterized in that: The image data includes point cloud data; wherein the transforming and processing the image data to obtain the characteristic image of the target vehicle includes: The point cloud data is transformed to obtain a depth map and a projection map of the target vehicle; wherein the depth map represents the distance between the point cloud and the point cloud sensor, and the projection map represents the projection value of the point cloud data on a two-dimensional plane.

7. The vehicle collision detection method according to claim 4, characterized in that: The step of comprehensively determining the final collision detection result of the target vehicle by combining the time-frequency image and the feature image includes: extracting image features from the feature image; The final collision detection result of the target vehicle is comprehensively determined by combining the audio features and the image features.

8. An automobile collision detection device, characterized in that: include: A data acquisition module, used to collect audio data and image data of the target vehicle; a preliminary detection module, configured to determine a preliminary collision detection result of the target vehicle based on the audio data; A final detection module is configured to comprehensively determine a final collision detection result of the target vehicle by combining the audio data and the image data if the preliminary collision detection result indicates that the target vehicle has collided or is at risk of collision.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the method according to any one of claims 1 to 7.

Images