Driving early warning method and device based on voice recognition, equipment and medium

Abstract

The invention relates to a driving early warning method, device and equipment based on sound recognition and a medium, and the method comprises the steps: responding to the obtained sound data collected by a vehicle, classifying the sound data, and obtaining the sound type of the sound data; under the condition that the sound type is a specified type, acquiring a sound source position of a sound source of the sound data relative to the vehicle; determining a sound source position change trend of the sound source according to the sound source positions of the sound source at the multiple moments; and under the condition that the sound source position change trend meets a preset risk condition, safety control is conducted on the vehicle. According to the technical scheme, the internal and external sounds of the vehicle can be accurately recognized in the vehicle driving process, the method is further applied to vehicle safety control, and the driving safety is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicles, and in particular, to a driving warning method, device, equipment and medium based on sound recognition. Background Art

[0002] In the vehicle driving scenario, the sounds in the environment are of great significance for the driver to understand the driving environment.

[0003] Currently, during the vehicle driving process, due to the complex and changeable scenarios, there is a situation where the sounds inside and outside the vehicle overlap, and users cannot effectively distinguish the environment through sounds. How to accurately identify the sounds in the driving environment to achieve driving warning is a technical problem to be solved urgently. Summary of the Invention

[0004] To solve the above technical problems, the present disclosure provides a driving warning method, device, equipment and medium based on sound recognition.

[0005] In a first aspect, an embodiment of the present disclosure provides a driving warning method based on sound recognition, including:

[0006] In response to obtaining the sound data collected by the vehicle, classifying the sound data to obtain the sound type of the sound data;

[0007] When the sound type is a specified type, obtaining the sound source position of the sound data relative to the vehicle;

[0008] Determining the sound source position change trend of the sound source according to the sound source positions at multiple moments;

[0009] When the sound source position change trend meets a preset risk condition, performing safety control on the vehicle.

[0010] In a second aspect, an embodiment of the present disclosure provides a driving warning device based on sound recognition, including:

[0011] A classification module, configured to classify the sound data to obtain the sound type of the sound data in response to obtaining the sound data collected by the vehicle;

[0012] An obtaining module, configured to obtain the sound source position of the sound data relative to the vehicle when the sound type is a specified type;

[0013] A determining module, configured to determine the sound source position change trend of the sound source according to the sound source positions at multiple moments;

[0014] An early warning module, configured to perform safety control on the vehicle when the changing trend of the sound source position meets a preset risk condition.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor; a memory for storing executable instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the above method.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where the storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0017] The technical solution provided by the embodiment of the present disclosure has the following advantages compared with the prior art: in response to obtaining sound data of a specified type collected by a vehicle, the sound source position of the sound data relative to the vehicle is obtained, and then according to the sound source positions of the sound source at multiple moments, the changing trend of the sound source position is determined. When the changing trend of the sound source position meets a preset risk condition, safety control is performed on the vehicle. Thus, for the sound data collected during the vehicle driving process, the sound source is continuously monitored at each moment, the position relationship between each sounding object and the vehicle can be accurately identified, the sounds inside and outside the vehicle can be accurately identified during the vehicle driving process, and further applied to vehicle safety control. Judging risks and performing safety control according to the changing trend of the sound source position of the sound source can improve driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic flowchart of a driving early warning method based on sound recognition provided by an embodiment of the present disclosure;

[0021] Figure 2 It is a schematic flowchart of another driving early warning method based on sound recognition provided by an embodiment of the present disclosure;

[0022] Figure 3 It is a schematic structural diagram of a driving early warning device based on sound recognition provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] In order to more clearly understand the above-mentioned objects, features, and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0024] Many specific details are set forth in the following description in order to provide a thorough understanding of the present disclosure, but the present disclosure may be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0025] Figure 1 The following is a schematic flowchart of a driving warning method based on voice recognition provided by an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure can be executed by a driving warning device based on voice recognition. The device can be implemented by software and / or hardware and can be integrated on any electronic device with computing capabilities.

[0026] As Figure 1 shown, the driving warning method based on voice recognition provided by the embodiment of the present disclosure may include:

[0027] Step 101, in response to obtaining voice data collected by a vehicle, classify the voice data to obtain the voice type of the voice data. In this embodiment, the vehicle is provided with a voice acquisition system. During the driving of the vehicle, the voice acquisition system collects voice data and classifies the collected voice data to determine the voice type of the voice data. Among them, the voice type is used to indicate what kind of voice the voice data is. For example, the voice type includes, but is not limited to, wind sound, rain sound, collision sound, human voice, animal sound, rhythm sound. For another example, the voice type can be more specifically divided, and no specific limitation is made here.

[0028] Optionally, the voice acquisition system includes a noise filtering hardware, a recording hardware, a voice storage medium, a storage data signal transmission, a sound source processing chip, and a power supply line. For example, the vehicle is provided with an external vehicle voice acquisition system and an internal vehicle voice acquisition system to collect voice data of the external vehicle environment and the internal vehicle environment respectively.

[0029] As an example, classifying the voice data to obtain the voice type of the voice data includes: pre-classifying the voice data, determining the target category to which the voice data belongs from a preset category, and then comparing the voice data with the reference voice features under the target category to determine the voice type of the voice data.

[0030] In this example, the preset categories include environmental sounds, biological sounds, and musical sounds. For example, sound data with sound types such as wind sounds, rain sounds, and collision sounds correspond to the environmental sound category; sound data with sound types such as human voices and animal sounds correspond to the biological sound category; and sound data with sound types such as rhythmic beats correspond to the musical sound category. Reference sound characteristics of each sound type are set under each preset category. First, use an audio classification algorithm to determine which target category among the above preset categories the sound data belongs to. Then, compare the reference sound characteristics of each sound type under the target category with the sound characteristics of the sound data, and determine the sound type with the highest similarity as the sound type of the sound data. Thus, the number of comparisons between the sound data and the reference sound characteristics can be reduced, thereby improving the query efficiency.

[0031] As another example, classifying the sound data to obtain the sound type of the sound data includes: comparing the reference sound characteristics of each pre-stored sound type with the sound data to determine the sound type of the sound data.

[0032] Step 102, when the sound type is the specified type, obtain the sound source position of the sound data relative to the vehicle.

[0033] In this embodiment, the specified type is a type associated with vehicle safety and driving warnings. For example, the specified type includes sound types such as falling rock sounds and blasting sounds that may affect the safe driving of the vehicle, and sound types such as shouting sounds and children's voices that may affect personal safety.

[0034] In this embodiment, the sound source position is used to indicate the position of the sound source relative to the vehicle. The steps for obtaining the sound source position of the sound source corresponding to the sound data include: performing object detection on the collected data of the image sensor and the radar sensor to obtain the object positions and object categories of each object inside and outside the vehicle; for each sound data, match the object category of each object with the sound type of the sound data to determine the target object corresponding to the sound data, and determine the sound source position of the sound data according to the object position of the target object. Among them, the image sensor includes a first camera and a second camera. The first camera is used to collect images outside the vehicle, and the second camera is used to collect images inside the vehicle. The radar sensor is, for example, a millimeter-wave radar. Thus, the image sensor and the radar sensor of the vehicle can be linked to achieve sound source positioning.

[0035] As an example, perform object detection on the collected data of the image sensor and the radar sensor. When it is detected that the object category of the object one outside the vehicle is a falling rock, and the sound type of the sound data collected by the external sound collection system of the vehicle is a falling rock sound, it is determined that the object category falling rock matches the sound type falling rock sound, determine that the sound data corresponds to the object one, and determine the object position of the object one as the sound source position of the sound data.

[0036] Step 103: Determine the changing trend of the sound source position based on the sound source positions at multiple moments.

[0037] In this embodiment, when sound data of a specified type is detected, the sound source positions of the sound data relative to the vehicle at multiple moments can be obtained through continuous acquisition and recognition. Then, based on the sound source positions at multiple moments, the changing trend of the sound source position is determined. Among them, the changing trend of the sound source position indicates information such as the trajectory of the sound source position and the positional relationship between the sound source and the vehicle.

[0038] Step 104: Perform safety control on the vehicle when the changing trend of the sound source position meets the preset risk conditions.

[0039] In this embodiment, risk conditions can be preset, and risk identification is performed on the changing trend of the sound source position based on the risk conditions. When a risk is identified, safety control is performed on the vehicle. Among them, safety control includes driving warning, intervention of the automatic driving system in vehicle control, etc. Optionally, prompt information is generated based on the sound type and the changing trend of the sound source position to remind the driver of driving risks. For example, when the sound type is a type with risks and the changing trend of the sound source position includes the sound source approaching the vehicle, prompt information is generated to remind the driver of the specific risk information. Optionally, the automatic driving system intervenes in vehicle control. For example, when the sound type is a type with risks and it is determined that a risk is about to occur based on the changing trend of the sound source position and the current driving state of the vehicle, the automatic driving system intervenes in vehicle control.

[0040] The following is an illustration with specific examples.

[0041] As an example, the sound data collected by the vehicle includes the sound of a shout warning outside the vehicle, and the sound type is a shout warning sound. The changing trend of the sound source position of this sound source is obtained through continuous acquisition and recognition. When it is detected that the changing trend of the sound source position of this sound source changes from moving away from the vehicle to approaching the vehicle by a certain distance, it is determined that the preset risk conditions are met, and safety control is performed at this time. In this example, safety control includes generating warning information to prompt the user to pay attention to the safety of the surrounding environment, etc. Thus, it is possible to accurately identify the situation with risks through the changing trend of the sound source position of the shout warning sound outside the vehicle and perform safety control to ensure the safety of the vehicle and improve safety.

[0042] As another example, a passenger in the vehicle includes a child, the sound data collected by the vehicle includes the sound emitted by the child in the vehicle, the sound type is the sound of a child, and the change trend of the sound source position of the sound source is obtained through continuous collection and recognition. When it is detected that the change trend of the sound source position of the sound source is from inside the vehicle to the window, or the change trend of the sound source position of the sound source is from inside the vehicle to outside the vehicle, it is determined that the preset risk condition is met, and at this time, safety control is performed. In this example, the safety control includes generating a warning message to prompt the user to pay attention to the safety of the child, and the vehicle decelerates. Thus, it is possible to accurately identify the situation with risks through the change trend of the sound source position of the child, and perform safety control to ensure the personal safety of the child in the vehicle and improve safety.

[0043] According to the technical solution of the embodiment of the present disclosure, in response to obtaining the sound data of a specified type collected by the vehicle, the sound source position of the sound data relative to the vehicle is obtained, and then according to the sound source positions at multiple moments, the change trend of the sound source position is determined. When the change trend of the sound source position meets the preset risk condition, safety control is performed on the vehicle. Thus, for the sound data collected during the vehicle driving process, the sound source is continuously monitored at each moment, and the position relationship between each sounding object and the vehicle can be accurately identified, solving the problem that the user cannot effectively judge the environment due to the overlap of the sounds inside and outside the vehicle, realizing the accurate identification of the sounds inside and outside the vehicle during the vehicle driving process, and further applying it to vehicle safety control. Judging risks and performing safety control according to the change trend of the sound source position of the sound source can improve driving safety.

[0044] Based on the above embodiment, the following will be described by taking the first moment and the second moment as examples. Figure 2 The flowchart of another driving warning method based on sound recognition provided by the embodiment of the present disclosure is as Figure 2 shown, and the method includes:

[0045] Step 201, in response to obtaining the first sound data collected by the vehicle at the first moment, classify the first sound data to obtain the sound type of the first sound data.

[0046] In this embodiment, the first sound data is collected by the sound collection system, and the collected first sound data is transmitted to the storage medium of the vehicle body processor. The vehicle body processor processes the first sound data. Optionally, the vehicle body processor further performs noise reduction processing on the digital data stored by the first sound data to improve the quality of the sound data and match the vehicle audio tuning strategy.

[0047] As an example, classifying the first sound data to obtain the sound type of the first sound data includes: pre-classifying the first sound data, determining the target category to which the first sound data belongs from the preset categories, and then comparing the first sound data with the reference sound features under the target category to determine the sound type of the first sound data.

[0048] As another example, classifying the first sound data to obtain the sound type of the first sound data includes: comparing the first sound data with the reference sound features of each pre-stored sound type to determine the sound type of the first sound data.

[0049] In an embodiment of the present disclosure, after classifying the sound data to obtain the sound type of the sound data, the sound data can also be compared with the historical sound features of each sound type to determine the comparison sound type of the sound data, and then the sound type of the sound data can be corrected according to the comparison sound type to update the sound type of the sound data, further improving the recognition accuracy of the sound type. Among them, the step of correcting the sound type of the sound data can be implemented on the server side or on the vehicle side.

[0050] As an example, taking the first sound data as an example, the vehicle side sends the first sound data and the sound type of the first sound data to the server side. The server side compares the first sound data with the historical sound features of each sound type to determine the comparison sound type of the first sound data, corrects the sound type of the first sound data according to the comparison sound type to obtain the corrected sound type, and returns the corrected sound type to the vehicle side. The vehicle side receives the corrected sound type returned by the server side to update the sound type of the first sound data.

[0051] Among them, the server side has higher performance than the vehicle side. The preset reference sound features can be stored on the vehicle side. The server side collects the historical sound features of each sound type, compares the historical sound features of each sound type with the sound features of the first sound data, and determines the sound type with the highest similarity as the comparison sound type of the first sound data. Then, the sound type of the first sound data is corrected according to the comparison sound type. For example, if the sound type of the first sound data is inconsistent with the comparison sound type, the comparison sound type is used as the corrected sound type. The historical sound features include the sound features actually collected and uploaded by the vehicle. Thus, it is possible to perform comparison and verification based on cloud data, further improving the recognition accuracy of the sound type of the first sound data.

[0052] Optionally, the preset categories include environmental sounds, biological sounds, and music sounds. According to the preset category to which the sound type of the first sound data uploaded by the vehicle belongs, the historical sound features of each sound type under the target category are compared with the sound features of the first sound data to determine the comparison sound type of the first sound data.

[0053] Step 202: In response to obtaining the second sound data collected by the vehicle at the second moment, classify the second sound data to obtain the sound type of the second sound data.

[0054] In this embodiment, the second sound data is collected by the sound collection system, and the collected second sound data is transmitted to the storage medium of the vehicle body processor. The vehicle body processor processes the second sound data, and the vehicle body processor further performs noise reduction processing on the digital data stored by the second sound data to improve the quality of the sound data and match the vehicle audio tuning strategy.

[0055] In this embodiment, the second moment is after the first moment, and the collection interval between the first moment and the second moment can be preset. The second sound data is the sound data collected by the sound collection system at the second moment.

[0056] As an example, classifying the second sound data to obtain the sound type of the second sound data includes: pre-classifying the second sound data to determine the target category to which the second sound data belongs from the preset categories. Furthermore, comparing the reference sound features under the target category with the second sound data to determine the sound type of the second sound data.

[0057] In this example, the preset categories include environmental sounds, biological sounds, and music sounds, and the reference sound features of each sound type are set under each preset category. First, determine which target category among the above preset categories the second sound data belongs to through an audio classification algorithm. Furthermore, compare the reference sound features of each sound type under the target category with the sound features of the second sound data, and determine the sound type with the highest similarity as the sound type of the second sound data. Thus, the number of comparisons between the second sound data and the reference sound features can be reduced, thereby improving the query efficiency.

[0058] In an embodiment of the present disclosure, in response to obtaining the second sound data collected by the vehicle at the second moment, the second sound data can also be source located to determine the source position of the sound source corresponding to the second sound data. The source position is used to indicate the position of the sound source relative to the vehicle.

[0059] Among them, the steps of obtaining the sound source position of the sound source corresponding to the second sound data include: performing object detection through the collected data of the image sensor and the radar sensor to obtain the object positions and object categories of each object inside and outside the vehicle; for each second sound data, matching the object category of each object with the sound type of the second sound data to determine the target object corresponding to the second sound data, and determining the sound source position of the second sound data according to the object position of the target object. The explanation of determining the sound source position for the first sound data also applies to the second sound data and will not be elaborated here.

[0060] Step 203: Determine the positional relationship between the sound source and the vehicle according to the sound type, the sound source position of the first sound data relative to the vehicle, and the sound source position of the second sound data relative to the vehicle.

[0061] In this embodiment, the first sound data and the second sound data belonging to the same sound source are determined according to the sound type. Furthermore, for the first sound data and the second sound data corresponding to each sound source, the positional relationship between the sound source and the vehicle is determined according to the change trajectory between the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle. Among them, the positional relationship can be used to represent the relative approach or separation between the sound source and the vehicle, the internal and external relationship between the sound source and the vehicle, etc. By identifying the sound type, the sound source position, and the positional relationship between the sound source and the vehicle, it can be further applied to vehicle safety control.

[0062] As an example, the first sound data and the second sound data belonging to the same sound source are determined according to the sound type. For each sound source, if the sound source position of the second sound data relative to the vehicle is closer to the vehicle than the sound source position of the first sound data relative to the vehicle, it is determined that the sound source is relatively approaching the vehicle. If the sound source position of the second sound data relative to the vehicle is farther from the vehicle than the sound source position of the first sound data relative to the vehicle, it is determined that the sound source is relatively moving away from the vehicle. In this example, the movement trend of the vehicle and the sound source can be determined to be further used for vehicle safety control.

[0063] As another example, the first sound data and the second sound data belonging to the same sound source are determined according to the sound type. For each sound source, if the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle remain unchanged, it is determined that the sound source is an in-vehicle sound source. Otherwise, it is determined that the sound source is an out-of-vehicle sound source. In this example, during the vehicle driving process, taking the whistle sound played on the rear display screen of the vehicle as an example, the sound type is the whistle sound. The first sound data and the second sound data both belong to sound source A. At this time, the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle remain unchanged, and it is determined that sound source A is an in-vehicle sound source. Thus, through the change situation of the sound source position twice for verification, the internal and external relationship between the sound source and the vehicle can be accurately identified, further improving the sound recognition accuracy to avoid the interference of overlapping sounds.

[0064] As another example, for first sound data and second sound data of the same sound type, obtain the sound frequency change information between the first sound data and the second sound data, and obtain the change trajectory between the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle. Furthermore, match the sound frequency change information with the change trajectory. If the sound frequency change information matches the change trajectory, it is determined that the first sound data and the second sound data belong to the same sound source. In this example, based on the Doppler effect, it is possible to determine whether the sound source is approaching or moving away through the sound frequency change information between the first sound data and the second sound data. At the same time, the change trajectory between the two sound source positions can also determine whether the sound source is approaching or moving away. By comparing the determination result based on the sound frequency change information with the determination result based on the change trajectory, it is determined whether the sound frequency change information matches the change trajectory. For example, if the determination result based on the sound frequency change information is the same as the determination result based on the change trajectory, it is determined that the sound frequency change information matches the change trajectory. If the determination result based on the sound frequency change information is opposite to the determination result based on the change trajectory, it is determined that the first sound data and the second sound data do not belong to the same sound source. Thus, it is possible to conduct verification based on the sound frequency change information, further improving the accuracy of sound recognition.

[0065] In the embodiments of the present disclosure, it is possible to perform double verification based on the change situation of the two sound source positions and the sound attributes, further improving the accuracy of sound recognition. Based on the current situation where the in-vehicle playback sound overlaps with the out-of-vehicle sound type, for example, when the out-of-vehicle dangerous sound is covered by the in-vehicle similar sound, the sound can be accurately recognized and further used for vehicle safety control to improve driving safety.

[0066] Figure 3 FIG. is a schematic structural diagram of a driving warning device based on sound recognition provided by the embodiments of the present disclosure. As Figure 3 shown, the driving warning device based on sound recognition includes: a classification module 31, an acquisition module 32, a determination module 33, and a warning module 34.

[0067] The classification module 31 is configured to classify the sound data in response to obtaining the sound data collected by the vehicle, and obtain the sound type of the sound data.

[0068] The acquisition module 32 is configured to obtain the sound source position of the sound data relative to the vehicle when the sound type is a specified type.

[0069] The determination module 33 is configured to determine the sound source position change trend of the sound source according to the sound source positions of the sound source at multiple moments.

[0070] An early warning module 34 for performing safety control on the vehicle when the changing trend of the sound source position meets a preset risk condition.

[0071] In an embodiment of the present disclosure, the device further includes:

[0072] A comparison module for comparing the historical sound characteristics of each sound type with the sound data to determine the comparison sound type of the sound data;

[0073] Correcting the sound type of the sound data according to the comparison sound type to update the sound type of the sound data.

[0074] In an embodiment of the present disclosure, the first processing module 41 is specifically configured to: pre-classify the sound data, and determine the target category to which the sound data belongs from a preset category; the preset category includes environmental sounds, biological sounds, and music sounds; compare the sound data with the reference sound characteristics under the target category to determine the sound type of the sound data.

[0075] In an embodiment of the present disclosure, the determination module 43 is specifically configured to: for the first sound data collected by the vehicle at the first moment and the second sound data collected at the second moment, determine the positional relationship between the sound source and the vehicle according to the sound type, the sound source position of the first sound data relative to the vehicle, and the sound source position of the second sound data relative to the vehicle.

[0076] In an embodiment of the present disclosure, the determination module 43 is specifically configured to: determine the first sound data and the second sound data belonging to the same sound source according to the sound type; for each sound source, if the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle remain unchanged, determine that the sound source is an in-vehicle sound source; otherwise, determine that the sound source is an out-of-vehicle sound source.

[0077] In an embodiment of the present disclosure, the determination module 43 is specifically configured to: for the first sound data and the second sound data with the same sound type, obtain the sound frequency change information between the first sound data and the second sound data; obtain the change trajectory between the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle; if the sound frequency change information matches the change trajectory, determine that the first sound data and the second sound data belong to the same sound source.

[0078] In an embodiment of the present disclosure, the device further includes:

[0079] A positioning module for performing object detection through the collected data of the image sensor and the radar sensor to obtain the object positions and object categories of each object inside and outside the vehicle;

[0080] For each sound data, match the object category of each object with the sound type of the sound data to determine the target object corresponding to the sound data, and determine the sound source position of the sound data according to the object position of the target object.

[0081] The apparatus provided by the embodiments of the present disclosure can execute the foregoing method provided by the embodiments of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method. The content not described in detail in the embodiments of the apparatus of the present disclosure can be referred to the description in any method embodiment of the present disclosure.

[0082] The embodiments of the present disclosure also provide an electronic device, which includes one or more processors and a memory. The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to execute desired functions. The memory may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor may run the program instructions to implement the method of the embodiments of the present disclosure above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage media.

[0083] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected through a bus system and / or other forms of connection mechanisms. In addition, the input device may include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including the determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc. In addition, according to specific application scenarios, the electronic device may further include any other appropriate components such as a bus, an input / output interface, etc.

[0084] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions that cause the processor to execute any method provided by the embodiments of the present disclosure when the processor runs.

[0085] A computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on a user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0086] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are run by a processor, the processor is caused to execute any method provided by the embodiments of the present disclosure.

[0087] The computer-readable storage medium may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0088] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0089] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A driving warning method based on sound recognition, characterized in that: The method comprises: In response to acquiring the sound data collected by the vehicle, classifying the sound data to obtain a sound type of the sound data; When the sound type is a specified type, obtaining a sound source position of the sound data relative to the vehicle; Determining a change trend of a sound source position of the sound source according to the sound source positions of the sound source at multiple moments; When the trend of the sound source position change meets a preset risk condition, the vehicle is safely controlled.

2. The method according to claim 1, characterized in that The classifying the sound data to obtain the sound type of the sound data includes: Pre-classifying the sound data, and determining the target category to which the sound data belongs from preset categories; the preset categories include environmental sound, biological sound, and musical sound; The sound data is compared with reference sound features under the target category to determine the sound type of the sound data.

3. The method according to claim 1, characterized in that After classifying the sound data to obtain the sound type of the sound data, the method further includes: comparing the sound data with historical sound features of each sound type to determine a comparative sound type of the sound data; The sound type of the sound data is corrected according to the compared sound type to update the sound type of the sound data.

4. The method according to claim 1, characterized in that The multiple moments include a first moment and a second moment, and determining a change trend of a sound source position of the sound source according to the sound source positions of the sound source at the multiple moments includes: For the first sound data collected by the vehicle at the first moment and the second sound data collected at the second moment, the positional relationship between the sound source and the vehicle is determined according to the sound type, the sound source position of the first sound data relative to the vehicle, and the sound source position of the second sound data relative to the vehicle.

5. The method according to claim 4, characterized in that Determining the positional relationship between the sound source and the vehicle according to the sound type, the sound source position of the first sound data relative to the vehicle, and the sound source position of the second sound data relative to the vehicle includes: Determining first sound data and second sound data belonging to the same sound source according to the sound type; For each sound source, if the sound source position of the first sound data relative to the vehicle and the sound source position of the second sound data relative to the vehicle remain unchanged, then the sound source is determined to be a sound source inside the vehicle; Otherwise, it is determined that the sound source is a sound source outside the vehicle.

6. The method according to claim 5, characterized in that The step of determining the first sound data and the second sound data belonging to the same sound source according to the sound type comprises: For the first sound data and the second sound data of the same sound type, obtaining sound frequency change information between the first sound data and the second sound data; Acquire a change trajectory between a sound source position of the first sound data relative to the vehicle and a sound source position of the second sound data relative to the vehicle; If the sound frequency change information matches the change trajectory, it is determined that the first sound data and the second sound data belong to the same sound source.

7. The method according to claim 1, characterized in that The steps to obtain the sound source position include: Performing object detection through the collected data of the image sensor and the radar sensor to obtain the object position and object category of each object inside and outside the vehicle; For each sound data, the object category of each object is matched with the sound type of the sound data to determine the target object corresponding to the sound data, and the sound source position of the sound data is determined according to the object position of the target object.

8. A vehicle warning device based on sound recognition, characterized in that: include: A classification module, configured to classify the sound data collected by the vehicle in response to obtaining the sound data, and obtain the sound type of the sound data; an acquisition module, configured to acquire a sound source position of the sound data relative to the vehicle when the sound type is a specified type; A determination module, configured to determine a change trend of a sound source position of the sound source according to the sound source positions of the sound source at multiple moments; The early warning module is used to perform safety control on the vehicle when the trend of the change of the sound source position meets the preset risk condition.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images