Early warning prompt method and device, electronic equipment, storage medium and program product

Abstract

This application discloses a warning notification method, device, electronic device, storage medium, and program product, belonging to the field of vehicle technology. The method includes: performing audio recognition on collected ambient sounds to obtain an audio recognition result; if the audio recognition result indicates that the ambient sounds include a vehicle start-up sound, determining the positional relationship with a target vehicle based on the loudness of the vehicle start-up sound, wherein the target vehicle is the vehicle that generated the start-up sound; and issuing a warning notification if the positional relationship meets the warning notification conditions, wherein the warning notification is used to indicate the presence of a starting vehicle in the vicinity. The method provided in this application allows an electronic device to detect surrounding vehicles through simple audio recognition and loudness detection, and to issue a timely warning when a starting vehicle is present, reducing the safety hazards caused by vehicle starting to the target object.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a warning notification method, device, electronic device, storage medium, and program product. Background Technology

[0002] Currently, due to blind spots in vehicles, drivers cannot observe the situation within those blind spots when starting the vehicle.

[0003] In related technologies, vehicles are equipped with detection devices, such as automotive radar, to detect obstacles or people near the vehicle, thus preventing collisions caused by the driver not noticing nearby vehicles, people, or obstacles in their blind spots.

[0004] In other words, related technologies involve installing devices on vehicles to warn drivers. However, in this method, people near the vehicle cannot promptly perceive the status of surrounding vehicles, which still poses certain safety hazards. Summary of the Invention

[0005] This application provides a warning notification method, device, electronic device, storage medium, and program product, which can reduce the safety hazards caused by vehicle startup to target objects. The technical solution is as follows:

[0006] On the one hand, embodiments of this application provide an early warning notification method, the method comprising:

[0007] The collected ambient sounds are subjected to audio recognition to obtain the audio recognition results;

[0008] If the audio recognition result indicates that the ambient sound contains a vehicle start-up sound, the positional relationship with the target vehicle is determined based on the loudness of the vehicle start-up sound, wherein the target vehicle is the vehicle that generated the vehicle start-up sound;

[0009] If the location relationship meets the warning prompt conditions, a warning prompt will be issued to indicate that a vehicle is starting up in the vicinity.

[0010] On the other hand, embodiments of this application provide an early warning device, the device comprising:

[0011] The audio recognition module is used to perform audio recognition on the collected ambient sounds and obtain the audio recognition results.

[0012] A location determination module is used to determine the location relationship with a target vehicle based on the loudness of the vehicle start-up sound when the audio recognition result indicates that the ambient sound contains a vehicle start-up sound, wherein the target vehicle is the vehicle that generated the vehicle start-up sound;

[0013] The early warning module is used to issue an early warning when the location relationship meets the early warning conditions, and the early warning is used to indicate that there is a vehicle starting in the vicinity.

[0014] On the other hand, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the warning prompting method as described above.

[0015] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the warning notification method as described above.

[0016] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the warning prompting method provided in the various optional implementations of the above aspects.

[0017] The beneficial effects of the technical solutions provided in this application include at least the following:

[0018] In this embodiment, the electronic device can identify vehicle start-up sounds in the ambient sound environment. Upon detecting a vehicle start-up sound, it can further determine the positional relationship between itself and the target vehicle based on the loudness of the sound, and then determine whether to issue a warning based on the positional relationship. In other words, the electronic device can detect surrounding vehicles through simple audio recognition and loudness detection, and issue a timely warning when a vehicle is starting nearby. If the target is carrying an electronic device, a warning can be issued to the target, thereby reducing the safety hazards caused by vehicle starting. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1A schematic diagram of an electronic device provided in an exemplary embodiment of this application is shown;

[0021] Figure 2 A schematic diagram of an electronic device provided in another exemplary embodiment of this application is shown;

[0022] Figure 3 A flowchart of an exemplary embodiment of the warning notification method provided in this application is shown;

[0023] Figure 4 A flowchart of an early warning notification method provided by another exemplary embodiment of this application is shown;

[0024] Figure 5 This invention illustrates a schematic diagram of the structure of a parallel CRNN model provided in an exemplary embodiment of this application.

[0025] Figure 6 A schematic diagram illustrating an implementation of an exemplary embodiment of this application is shown.

[0026] Figure 7 A flowchart of an early warning notification method provided by another exemplary embodiment of this application is shown;

[0027] Figure 8 A schematic diagram illustrating an implementation of an exemplary embodiment of the warning notification method provided in this application is shown.

[0028] Figure 9 A schematic diagram illustrating an implementation of an early warning notification process, as shown in another exemplary embodiment of this application, is provided.

[0029] Figure 10 This is a structural block diagram of an exemplary embodiment of the warning and alerting device provided in this application;

[0030] Figure 11 A structural block diagram of an electronic device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0032] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0033] In related technologies, detection and alarm devices, such as cameras and radar, are installed on vehicles to detect the surrounding environment. During vehicle startup, the area around the vehicle can be detected, and an alarm is triggered if nearby personnel are detected. In this method, the onboard device sends a warning to the driver, while nearby personnel remain unaware.

[0034] This application provides a warning notification method. An electronic device can perform audio recognition on collected ambient sounds. After recognizing a vehicle start-up sound, it further determines the device's position relative to the vehicle based on the start-up sound and provides a warning notification based on the positional relationship. When a target object is carrying an electronic device, if a nearby vehicle is starting, the electronic device can promptly warn the target object.

[0035] The method provided in this application embodiment can be executed by an electronic device, which is an electronic device with a warning and prompting function. The electronic device can be a portable electronic device, including but not limited to smartphones, tablets, wearable devices, etc.; it can also be other types of devices, including but not limited to intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0036] In one possible implementation, the electronic device can be a smart wearable device with integrated warning and alert mechanisms. The smart wearable device can be any wearable device, such as a watch, bracelet, glasses, or a pet-wearing device. Figure 1 As shown, the warning device includes a microphone 101 and a speaker 102, and incorporates an audio recognition system 103 and a loudness recognition system 104. When implementing the warning function, the microphone 101 collects ambient sound, and the audio recognition system 103 performs audio recognition on the collected ambient sound. Upon detecting a vehicle start-up sound, the loudness recognition system 104 detects the loudness of the vehicle start-up sound and determines the positional relationship with the target vehicle based on the loudness. If the positional relationship meets the warning conditions, a warning is issued through the speaker 102. When the target is wearing a smart wearable device, it can alert the target to the presence of a starting vehicle nearby, and also alert the driver to the presence of a nearby object. Compared to in-vehicle devices, the warning device is small, portable, and low-cost, with a wide range of applications.

[0037] In another possible implementation, the electronic device runs a program that provides early warning alerts. (Illustrative example, such as...) Figure 2As shown, a warning application runs within terminal 201. The microphone in terminal 201 can collect ambient sound, and the warning application can perform audio recognition on the ambient sound. When a vehicle start-up sound is detected, the loudness of the sound is detected, and the positional relationship with the target vehicle is determined based on the loudness. If the positional relationship meets the warning prompt conditions, a warning prompt can be issued. The warning application can issue warning prompts through at least one method, such as displaying a pop-up message, activating the vibration function, or broadcasting through a speaker.

[0038] The above description of the specific forms of electronic devices is illustrative but does not constitute a limitation; other forms of electronic devices may also be used.

[0039] Please refer to Figure 3 The diagram illustrates a flowchart of an exemplary embodiment of the warning notification method provided in this application. This embodiment uses an electronic device as an example for illustration, and the method includes the following steps.

[0040] Step 301: Perform audio recognition on the collected ambient sounds to obtain the audio recognition results.

[0041] In one possible implementation, the electronic device collects audio from the surrounding environment to obtain ambient sound. Optionally, the electronic device is equipped with a microphone to collect ambient sound.

[0042] In this embodiment, audio recognition is used to identify vehicle start-up sounds in the ambient sound. After the ambient sound is collected, the electronic device can perform audio recognition on the collected audio to obtain an audio recognition result, which is used to indicate whether the ambient sound contains a vehicle start-up sound.

[0043] When the target object carries an electronic device, the electronic device can perform audio recognition on the ambient sound around the target object to determine whether there is a vehicle starting sound around the target object, thereby determining whether there is a vehicle starting around the target object.

[0044] Step 302: If the audio recognition result indicates that the ambient sound includes the vehicle start sound, determine the positional relationship with the target vehicle based on the loudness of the vehicle start sound. The target vehicle is the vehicle that produced the vehicle start sound.

[0045] If the audio recognition result indicates that the ambient sound includes the sound of a vehicle starting, it means that there is a target vehicle starting around the electronic device. The target vehicle may be located far away from the electronic device or it may be located close to the electronic device. Therefore, the electronic device needs to further determine whether a warning prompt is needed based on its positional relationship with the target vehicle.

[0046] There is a certain correlation between sound loudness and sound propagation distance. In this embodiment, when it is determined that the ambient sound includes a vehicle start-up sound, the positional relationship between the electronic device and the target vehicle can be further determined based on the loudness of the vehicle start-up sound. In one possible implementation, after recognizing that the ambient sound includes a vehicle start-up sound, the loudness of the vehicle start-up sound can be detected. Furthermore, when detecting the loudness of the vehicle start-up sound, the electronic device can perform audio separation on the ambient sound to separate the vehicle start-up sound from the ambient sound, and then detect the loudness of the vehicle start-up sound. Optionally, the loudness of the vehicle start-up sound can be the decibel value of the vehicle start-up sound.

[0047] In one possible implementation, the positional relationship is used to indicate whether the location of the electronic device is within the danger zone of the target vehicle, and the electronic device determines whether it is in the danger zone based on the loudness of the vehicle's start-up sound.

[0048] In another possible implementation, the positional relationship can be used to indicate the distance between the electronic device and the target vehicle. The electronic device can determine the distance corresponding to the loudness of the vehicle start-up sound based on the correspondence between loudness and distance, thereby determining the distance between the electronic device and the target vehicle.

[0049] Step 303: If the location relationship meets the warning prompt conditions, issue a warning prompt. The warning prompt is used to indicate that there is a vehicle starting up in the vicinity.

[0050] Once the location relationship is determined, it is judged whether the warning prompt conditions are met based on the location relationship. When the location relationship is used to indicate the area where the electronic device is located, the warning prompt conditions can be related to the area; when the location relationship is used to indicate the distance between the electronic device and the target vehicle, the warning prompt conditions can be related to the distance.

[0051] For illustrative purposes, when the location relationship is used to indicate the area where the electronic device is located, the warning condition can be that the electronic device is located in a dangerous area; while when the location relationship is used to indicate the distance between the electronic device and the target vehicle, the warning condition can be that the distance between the electronic device and the target vehicle is less than a preset threshold.

[0052] When the location relationship meets the warning conditions, it indicates that the electronic device is close to the target vehicle, and therefore, the electronic device will issue a warning. When the target object is carrying an electronic device, the method provided in this application embodiment can alert the target object to the presence of a vehicle starting nearby, thus achieving the purpose of warning.

[0053] Optionally, the warning notification method may be at least one of the following: broadcasting the warning information through a speaker, displaying the warning information on a screen, or detecting equipment vibration.

[0054] In summary, in this embodiment, the terminal can identify vehicle start-up sounds in the ambient sound environment. Upon detecting a vehicle start-up sound, it can further determine the positional relationship between itself and the target vehicle based on the loudness of the sound, and then determine whether to issue a warning based on the positional relationship. That is, the terminal can detect surrounding vehicles through simple audio recognition and loudness detection, and issue a timely warning when a vehicle is starting nearby. If the target is carrying the terminal, a warning can be issued to the target, thereby reducing the safety hazards caused by vehicle starting.

[0055] In one possible implementation, the electronic device can determine the positional relationship with a target vehicle based on the relationship between the loudness of the vehicle start-up sound and a loudness threshold. Furthermore, since different vehicles have different start-up sounds, different start-up sounds at the same loudness may indicate different positional relationships. Therefore, the electronic device can also identify the vehicle type corresponding to a specific start-up sound and determine the positional relationship with the target vehicle based on the loudness and the loudness threshold corresponding to the vehicle type. An exemplary embodiment will be described below.

[0056] Please refer to Figure 4 The diagram illustrates a flowchart of an exemplary embodiment of the warning notification method provided in this application. This embodiment uses an electronic device as an example for illustration, and the method includes the following steps.

[0057] Step 401: Perform audio recognition on the ambient sound using an audio recognition model to obtain an audio recognition result. The audio recognition result is either that the ambient sound contains the vehicle start-up sound, or that the ambient sound does not contain the vehicle start-up sound.

[0058] In one possible implementation, the electronic device has a pre-trained audio recognition model. When ambient sound is detected, the audio recognition model is used to identify the ambient sound.

[0059] Optionally, the audio recognition model can be a Convolutional Recurrent Neural Network (CRNN). That is, the audio recognition model includes both a Convolutional Neural Network (CNN) model and a Recurrent Neural Network (RNN) model.

[0060] In one possible implementation, the process of audio recognition using a CRNN model includes: performing a short-time Fourier transform on the ambient sound to obtain the corresponding spectrogram; slicing the ambient sound spectrogram according to a fixed time interval to obtain several audio segment spectrograms; then, inputting these audio segment spectrograms into a CNN model for feature extraction to obtain feature maps corresponding to each audio segment spectrogram; stacking these feature maps; slicing the stacked feature maps along the time axis to obtain a sequence of feature vectors; inputting this sequence of feature vectors into an RNN model for further feature extraction to learn temporal features; and finally, inputting the RNN model output into a fully connected layer and a softmax layer for audio classification to obtain the audio recognition result.

[0061] In the above method, feature extraction is first performed using a CNN model, followed by feature extraction using an RNN model. In another possible implementation, to improve recognition speed and reduce latency, audio recognition is performed using a parallel CRNN model structure. This parallel CRNN model structure for audio recognition may include the following steps:

[0062] Step 1: Slice the spectrogram of the ambient sound into frames to obtain the spectrogram sequence. The spectrogram contains both temporal and frequency domain information of the ambient sound.

[0063] In one possible implementation, the electronic device first performs a short-time Fourier transform on the ambient sound audio to obtain the frequency domain information of the ambient sound, and then processes the frequency domain information and time domain information to obtain the spectrogram of the ambient sound. The spectrogram is a two-dimensional plane graph with time as the horizontal axis, frequency as the vertical axis, and color as the audio signal energy. That is, the spectrogram can represent the time domain information and frequency domain information of the ambient sound.

[0064] After obtaining the spectrogram, the spectrogram is sliced ​​into frames at fixed intervals, for example, at 0.1s intervals, to obtain spectrograms corresponding to several audio segments, i.e., spectrogram sequences. Then, CNN and RNN models are used to extract features from the spectrogram sequences.

[0065] Indicative, such as Figure 5 As shown, the time-domain signal 501 of the ambient sound is transformed to obtain the spectrogram 502 of the ambient sound, and then it is segmented into frames to obtain the spectrogram sequence.

[0066] Step 2: Input the spectrogram sequence into the CNN model for feature extraction to obtain the frequency domain features of the ambient sound.

[0067] The CNN model consists of multiple alternating convolutional and pooling layers. The pooling layers can employ max pooling to extract local features. In one possible implementation, the CNN model is used to extract frequency domain information from a spectrogram sequence to obtain frequency domain features.

[0068] Indicative, such as Figure 5 As shown, the CNN model consists of alternating convolutional layers 503 and pooling layers 504, extracting local features. Inputting the spectrogram sequence into the CNN model yields the frequency domain features 405 of the audio.

[0069] Step 3: Input the spectrogram sequence into the RNN model for feature extraction to obtain the temporal features of the ambient sound.

[0070] When inputting the spectrogram sequence into the CNN model, the spectrogram sequence is simultaneously input into the RNN model. That is, steps two and three are executed in parallel. Optionally, the RNN model can be a Bidirectional Long Short-Term Memory (BiLSTM) network, which performs well with long sequences and can learn contextual information, i.e., temporal information. Since vehicle start-up sounds are usually short-lived, such as within 2-3 seconds, they cannot be accurately identified solely through frequency domain features. Therefore, the RNN model is used to extract features from the spectrogram sequence, learning the temporal features of the audio segment, thereby improving recognition accuracy.

[0071] Before inputting the data into the RNN model, the spectrogram image segments are first pooled using pooling layers, and then the pooled features are input into the RNN model.

[0072] Indicative, such as Figure 5 As shown, the spectrogram sequence is input into the pooling layer 506, and then into the RNN model 507. The RNN model learns the temporal features of the spectrogram and obtains the temporal features 508.

[0073] Step 4: Perform feature concatenation between the frequency domain features and the time domain features to obtain the audio features of the ambient sound.

[0074] After extracting time-domain and frequency-domain features using CNN and RNN models, electronic devices can concatenate these features to obtain the audio features of ambient sound.

[0075] like Figure 4 As shown, frequency domain feature 505 and time domain feature 508 are concatenated to obtain audio feature 509.

[0076] Step 5: Input the audio features into the fully connected layer and the softmax layer for classification to obtain the audio recognition results.

[0077] Next, the audio features are input into a classifier consisting of a fully connected layer and a softmax layer for audio classification to obtain the audio recognition result. Optionally, the audio recognition result can be used to indicate whether the ambient sound contains the vehicle start sound, that is, the audio recognition result can be: the ambient sound contains the vehicle start sound, or the ambient sound does not contain the vehicle start sound.

[0078] Optionally, the audio recognition model can be pre-trained using a large amount of sample audio data. This sample audio data can include audio data containing the vehicle start-up sound and audio data not containing the vehicle start-up sound. During training, the model is updated in reverse using the difference between the recognition results and the labels of the audio data. Once the model converges, the training process for the audio recognition model is complete. The labels are used to indicate whether the sample audio data contains the vehicle start-up sound.

[0079] Furthermore, the audio recognition results may also include the probability that the ambient sound contains start-up sounds corresponding to different vehicle types, thereby determining the vehicle type to which the identified start-up sound belongs. That is, during training, a large amount of audio data is collected. When the audio data does not contain a vehicle start-up sound, the corresponding label indicates that the audio does not contain a start-up sound; when the audio data does contain a start-up sound, the label includes the target vehicle type corresponding to the start-up sound. The vehicle type may include at least one of vehicle power and vehicle class, with the vehicle class indicating vehicle size. After training the audio recognition model based on sample audio data and labels, when using the audio recognition model for audio recognition, the electronic device can further determine the target vehicle type based on the audio recognition results.

[0080] As an illustration, when the vehicle type is determined by its powertrain, the starting sounds of electric, gasoline, and diesel vehicles can be collected separately, and their corresponding powertrains can be labeled to obtain several sample audio data sets with vehicle powertrain labels. After training the audio recognition model using the sample audio data and corresponding labels, the audio recognition model can identify the probability that the ambient sound contains the starting sounds corresponding to different powertrain types of vehicles.

[0081] To illustrate, when the vehicle type is categorized by vehicle class, the start-up sounds of vehicles of different sizes can be collected and labeled with their corresponding vehicle classes, resulting in several sample audio data sets with vehicle class labels. After training the audio recognition model using the sample audio data and corresponding labels, the model can identify the probability that the ambient sound contains the start-up sound corresponding to different vehicle classes. For example, when the vehicle classes include first, second, and third classes, the audio recognition model can identify the probability that the ambient sound contains the start-up sound corresponding to the first class, the second class, and the third class.

[0082] Optionally, a pre-defined correspondence between vehicle class and vehicle size can be established, with a positive correlation between the two. The vehicle class may include Class I, for example, three classes, where Class I corresponds to small vehicles, Class II to medium-sized vehicles, and Class III to large vehicles.

[0083] To illustrate, when vehicle type is categorized by powertrain and vehicle class, the starting sounds of vehicles with different powertrain types and sizes can be collected separately, and their corresponding powertrain and vehicle class can be labeled, resulting in several sample audio data sets with vehicle class and powertrain labels. After training the audio recognition model using the sample audio data and corresponding labels, the audio recognition model can identify the probability that the ambient sound contains the starting sounds corresponding to different vehicle classes and powertrains. Based on the above example, the audio recognition model can identify the probability that the ambient sound contains the starting sounds corresponding to electric vehicles, gasoline vehicles, and diesel vehicles at the first level; the probability that the starting sounds of electric vehicles, gasoline vehicles, and diesel vehicles are respectively present at the second level; and the probability that the starting sounds of electric vehicles, gasoline vehicles, and diesel vehicles are respectively present at the third level.

[0084] It should be noted that the audio recognition model can be trained on the server side, and the electronic device can obtain the trained audio recognition model from the server. Alternatively, the training process can also be completed on the electronic device side. This embodiment does not limit this.

[0085] Step 402: If the audio recognition result indicates that the ambient sound includes the vehicle start-up sound, obtain the loudness of the vehicle start-up sound.

[0086] Optionally, the audio recognition result can be represented by 0 or 1. When the audio recognition result is 0, it indicates that the ambient sound does not include the vehicle start sound. When the audio recognition result is 1, it indicates that the ambient sound includes the vehicle start sound.

[0087] When the obtained audio recognition result is 1, the electronic device performs decibel detection on the vehicle start sound to obtain the loudness of the vehicle start sound.

[0088] Step 403: Determine the positional relationship with the target vehicle based on the relationship between the loudness of the vehicle start-up sound and the loudness threshold of the start-up sound.

[0089] To improve the detection efficiency of start-up sounds, in one possible implementation, it is determined whether the vehicle is located within the danger zone of the target vehicle based solely on the loudness of the start-up sound.

[0090] In one possible implementation, the electronic device has a pre-set start-up sound loudness threshold. When the loudness of the vehicle start-up sound is detected, the positional relationship with the target vehicle can be determined based on the relationship between the loudness of the vehicle start-up sound and the start-up sound loudness threshold.

[0091] Optionally, if the loudness of the vehicle's start-up sound is greater than the start-up sound loudness threshold, the vehicle is determined to be located within the danger zone of the target vehicle; if the loudness of the vehicle's start-up sound is less than the start-up sound loudness threshold, the vehicle is determined to be located outside the danger zone of the target vehicle.

[0092] In one possible implementation, the audio recognition model may only output whether the ambient sound contains a vehicle start-up sound. In this case, the electronic device cannot determine the vehicle type of the target vehicle. The start-up sound loudness threshold is a preset fixed loudness value, which can be determined based on the relationship between loudness and distance. For example, if the electronic device determines that the vehicle is in a danger zone within 10 meters, the decibel value corresponding to 10 meters can be determined as the start-up sound loudness threshold. After obtaining the loudness of the vehicle start-up sound, the positional relationship with the target vehicle is determined based on the preset fixed loudness value.

[0093] In another possible implementation, the audio recognition result also includes the probability that the ambient sound contains different vehicle start sounds. That is, the audio recognition model can identify the probability that the ambient sound contains the start sounds of different vehicle types, thereby determining the vehicle type of the target vehicle based on the identified start sounds. In this case, when determining the positional relationship with the target vehicle, a start sound loudness threshold can first be determined based on the vehicle type of the target vehicle. Determining the start sound loudness threshold may include the following steps:

[0094] Step 1: If the ambient sound contains the target vehicle's start-up sound, determine the target vehicle type based on the target vehicle's start-up sound, where the probability that the ambient sound contains the target vehicle's start-up sound is greater than a probability threshold.

[0095] In one possible implementation, if the probability that the ambient sound contains the target vehicle's start-up sound is greater than a probability threshold, the electronic device determines that the ambient sound contains the vehicle's start-up sound as the target vehicle's start-up sound. The electronic device can further determine the target vehicle type based on the vehicle type corresponding to the target vehicle's start-up sound.

[0096] Optionally, the audio recognition results may indicate the probability that the ambient sound contains the vehicle start-up sound corresponding to different vehicle types. The vehicle type includes at least one of vehicle power and vehicle class, with the vehicle class determined based on vehicle size, and the vehicle class being positively correlated with vehicle size.

[0097] The vehicle type can be determined by the vehicle's powertrain. The audio recognition result can indicate the probability that the ambient sound contains the starting sound of an electric vehicle, a gasoline vehicle, or a diesel vehicle. In this case, the determined target vehicle type is the target vehicle's powertrain. For example, if the electronic device detects that the probability of the ambient sound containing the starting sound of a gasoline vehicle is greater than the probability threshold of 80%, the target vehicle type can be determined to be a gasoline vehicle.

[0098] The vehicle type can also be the vehicle class. The audio recognition result can indicate the probability that the ambient sound contains the start sound of a vehicle corresponding to class i. In this case, the determined target vehicle type is the vehicle class of the target vehicle.

[0099] Vehicle type can also be vehicle powertrain and vehicle class. The audio recognition result can indicate the probability that the ambient sound contains the start-up sounds corresponding to vehicles with different powertrains and vehicle classes. In this case, the determined target vehicle type is the target vehicle's powertrain and vehicle class.

[0100] Step 2: Determine the starting sound volume threshold based on the target vehicle type.

[0101] When the target vehicle type is vehicle power, electric vehicles, gasoline vehicles, and diesel vehicles each correspond to different start-up sound volume thresholds. The electronic device pre-stores the start-up sound volume thresholds corresponding to different vehicle power types.

[0102] In one possible implementation, the starting sound loudness threshold corresponding to different vehicle power types can be determined based on the pre-collected correspondence between the loudness of the starting sound and distance when starting different power types of vehicles. For example, for a target power type of vehicle, a danger distance can be pre-set, and at least one vehicle starting sound can be collected at a danger distance from a vehicle belonging to the target power type. The average or minimum loudness of the at least one collected vehicle starting sound is used as the starting sound loudness threshold. The collected vehicle starting sounds can be from different vehicles belonging to the target power type. For example, the danger distance can be 10m. For gasoline vehicles, the vehicle starting sound can be collected at a distance of 10m from the gasoline vehicle, and the average or minimum loudness of the multiple collected vehicle starting sounds is determined as the starting sound loudness threshold corresponding to the gasoline vehicle.

[0103] Optionally, if the target vehicle type is an electric vehicle, the first value is determined as the start-up sound volume threshold.

[0104] Optionally, if the target vehicle type is a gasoline vehicle, the second value is determined as the start-up sound volume threshold, and the second value is greater than the first value.

[0105] Optionally, if the target vehicle type is a diesel vehicle, the third value is determined as the start-up sound volume threshold, and the third value is greater than the second value.

[0106] Specifically, the electronic device pre-stores the starting sound volume thresholds for electric vehicles as the first value, the pre-stored starting sound volume thresholds for gasoline vehicles as the second value, and the pre-stored starting sound volume thresholds for diesel vehicles as the third value. Since the starting sound of a gasoline vehicle is louder than that of an electric vehicle, and at the same volume, the relative position to a gasoline vehicle is greater than that to an electric vehicle, therefore, the starting sound volume threshold for gasoline vehicles is higher than that for electric vehicles. Similarly, the starting sound of a diesel vehicle is louder than that of a gasoline vehicle, and at the same volume, the relative position to a diesel vehicle is greater than that to a gasoline vehicle, therefore, the starting sound volume threshold for diesel vehicles is higher than that for gasoline vehicles. This reduces the probability of false identification and thus improves the accuracy of the warning prompts.

[0107] For illustrative purposes, the first value can be 30dB, the second value can be 50dB, and the third value can be 80dB.

[0108] When the target vehicle type is a vehicle class, in another possible implementation, the electronic device can determine the start-up sound threshold corresponding to the target vehicle class based on the correspondence between the vehicle class and the start-up sound volume threshold, wherein the start-up sound volume threshold is negatively correlated with the vehicle class.

[0109] This means that the electronic device stores a correspondence between Class i vehicles and their start-up sound volume thresholds. Higher vehicle classes indicate larger vehicle sizes. Larger vehicles have larger blind spots, meaning that at the same loudness, the danger zone for higher-class vehicles is larger than that for lower-class vehicles. Therefore, the start-up sound volume threshold for higher-class vehicles is higher than that for lower-class vehicles, improving warning accuracy.

[0110] The starting sound loudness threshold for Class i vehicles can be determined based on pre-collected loudness data of starting sounds from different Class i vehicles. For example, for Class i vehicles, a danger distance can be pre-set, and at least one vehicle starting sound can be collected at a danger distance from the Class i vehicle. The average or minimum loudness of the collected starting sounds is then used as the starting sound loudness threshold. The collected starting sounds can be from different Class i vehicles. Furthermore, the danger distance differs for different vehicle classes; for example, for Class I vehicles, the danger distance could be 8 meters; for Class II vehicles, it could be 10 meters; and for Class III vehicles, it could be 12 meters. When determining the starting sound loudness threshold for Class I vehicles, the starting sound can be collected at a distance of 8 meters from a Class I vehicle, and the average or minimum loudness of the collected starting sounds is used as the starting sound loudness threshold.

[0111] To illustrate, taking vehicle types including Level 1, Level 2, and Level 3 as an example, when the target vehicle type is Level 1, the starting sound volume threshold is determined to be the fourth value; when the target vehicle type is Level 2, the starting sound volume threshold is determined to be the fifth value; and when the target vehicle type is Level 3, the starting sound volume threshold is determined to be the sixth value. The sixth value is less than the fifth value, and the fifth value is less than the fourth value. For example, the fourth value is 80dB, the fifth value is 60dB, and the sixth value is 40dB.

[0112] In another possible implementation, the target vehicle type includes vehicle power and vehicle class, with different start-up sound volume thresholds corresponding to different vehicle power or vehicle class. The electronic device pre-stores the start-up sound volume thresholds corresponding to different vehicle power and different vehicle class.

[0113] When vehicles have the same power type, different vehicle classes have different start-up sound volume thresholds. Specifically, for vehicles with the same power type, the start-up sound volume threshold for higher-class vehicles should be lower than that for lower-class vehicles, ensuring that warnings can be issued from a relatively greater distance than for larger vehicles.

[0114] For vehicles of the same class, those with different powertrains have different start-up sound volume thresholds. Specifically, within the same vehicle class, the start-up sound volume threshold for electric vehicles is lower than that for gasoline vehicles, and the start-up sound volume threshold for gasoline vehicles is lower than that for diesel vehicles.

[0115] To illustrate, taking vehicle classes including Level 1, Level 2, and Level 3 as an example, Table 1 shows the starting sound volume thresholds for different vehicle powertrains and vehicle classes:

[0116] Table 1

[0117]

[0118] As shown in Table 1, when the target vehicle type indicates that the target vehicle is an electric vehicle and a Class 1 vehicle, the start-up sound level threshold is 60 dB. When all vehicles are Class 1 vehicles, the start-up sound level threshold for electric vehicles is lower than that for gasoline vehicles, and the start-up sound level threshold for gasoline vehicles is lower than that for diesel vehicles.

[0119] When all vehicles are electric vehicles, the first-level start-up sound level threshold is higher than the second-level start-up sound level threshold, and the second-level start-up sound level threshold is higher than the third-level start-up sound level threshold.

[0120] In one possible implementation, the target vehicle's start-up sound may include at least two types of vehicle start-up sounds. For example, when two types of vehicles are starting in the surrounding environment, the probability of detecting ambient sound containing the start-up sounds of both vehicle types may be greater than a probability threshold. In this case, a start-up sound intensity threshold corresponding to each type of vehicle start-up sound can be determined, and based on the respective start-up sound intensity threshold, it can be determined whether the electronic device is located within the danger zone of the corresponding vehicle. For example, when the probability of detecting ambient sound containing the start-up sounds of both electric and gasoline vehicles is greater than a probability threshold, a first start-up sound intensity threshold corresponding to the electric vehicle is determined, and based on the relationship between the loudness of the start-up sound corresponding to the electric vehicle and the first start-up sound intensity threshold, it is determined whether the device is located within the danger zone of the electric vehicle. Correspondingly, a second start-up sound intensity threshold corresponding to the gasoline vehicle is determined, and based on the relationship between the loudness of the start-up sound corresponding to the gasoline vehicle and the second start-up sound intensity threshold, it is determined whether the device is located within the danger zone of the gasoline vehicle.

[0121] Step 404: If the location relationship indicates that the location is within the danger zone of the target vehicle, determine that the conditions for issuing a warning are met, and issue a warning.

[0122] The warning condition is that the device is located within the danger zone of the target vehicle. In this case, the electronic device will issue a warning.

[0123] In one possible implementation, the warning is delivered via a loudspeaker. When using a loudspeaker, the loudness of the warning audio can be preset, alerting both the target and the driver. The warning audio can be a default setting or a user-defined setting.

[0124] Indicative, such as Figure 6As shown, the electronic device collects ambient sound through microphone 601 and inputs the ambient sound into CRNN model 602 for audio recognition. CRNN model 602 can identify the probability that the ambient sound contains the starting sound of different vehicle types, such as the probability of containing the starting sound of an electric vehicle, a gasoline vehicle, and a diesel vehicle. Alternatively, it can also contain the probability of containing the starting sound of different vehicle classes, or further, the probability of containing the starting sound of different vehicle power or different vehicle classes, such as the probability of containing the starting sounds of electric vehicles, gasoline vehicles, and diesel vehicles at the first, second, and third levels, respectively (not shown in the figure). The electronic device can determine that the starting sound of a gasoline vehicle is included based on the audio recognition results output by the model, thereby determining the starting sound volume threshold as the starting sound volume threshold corresponding to gasoline vehicles. Then, it determines whether the loudness of the vehicle starting sound is greater than the starting sound volume threshold. If it is greater than the starting sound volume threshold, a warning is issued through speaker 603.

[0125] In this embodiment, audio recognition is performed using a parallel CRNN model. By extracting the time-domain and frequency-domain features of the audio in parallel, the audio recognition speed can be improved, and the timeliness of the warning prompts can be enhanced. Furthermore, by performing audio recognition based on both time-domain and frequency-domain features, the accuracy of audio recognition can be improved.

[0126] Furthermore, in this embodiment, different start-up sound loudness thresholds are set for different vehicle types. The audio recognition model can further identify the vehicle type corresponding to the vehicle start-up sound, thereby determining the start-up sound loudness threshold based on the target vehicle's vehicle type, which can improve the accuracy of determining the positional relationship with the target vehicle based on the start-up sound loudness threshold.

[0127] Since the ambient sound does not include the vehicle's start-up sound in real time, the electronic device does not need to perform audio recognition on the collected ambient sound in real time. In one possible real-time approach, the electronic device can pre-set recognition conditions, and only perform audio recognition on the collected ambient sound when the conditions are met. Furthermore, when the positional relationship with the target vehicle meets the warning prompt conditions, the electronic device can provide warning prompts in different ways for different situations, allowing the target object to obtain more information. An exemplary embodiment will be used to illustrate this below.

[0128] Please refer to Figure 7 The diagram illustrates a flowchart of an exemplary embodiment of the warning notification method provided in this application. This embodiment uses an electronic device as an example for illustration, and the method includes the following steps.

[0129] Step 701: Obtain the ambient sound level of the collected ambient sound.

[0130] In one possible real-time approach, the recognition criteria are determined based on the ambient sound intensity. Therefore, when an electronic device collects ambient sound, it can perform decibel detection on the collected ambient sound in real time to obtain the ambient sound intensity.

[0131] Step 702: When the ambient sound intensity is greater than the ambient sound intensity threshold, perform audio recognition on the ambient sound to obtain the audio recognition result. The ambient sound intensity threshold is determined based on the lowest loudness of the starting sound of different vehicles.

[0132] The electronic device determines whether to perform audio recognition of ambient sounds based on the relationship between ambient sound intensity and an ambient sound intensity threshold. Optionally, the ambient sound intensity threshold is a pre-set fixed value, which can be the lowest loudness (i.e., lowest decibel) of different vehicle start-up sounds. Different types of vehicle start-up sounds can be pre-collected, and the loudness of each vehicle start-up sound can be detected, with the lowest loudness value determined as the ambient sound intensity threshold. Alternatively, to ensure that various vehicle start-up sounds can be recognized, loudness values ​​lower than the lowest loudness can be determined as the ambient sound intensity threshold. For example, if the lowest loudness of the collected vehicle start-up sounds is 15 dB, then 10 dB can be determined as the ambient sound intensity threshold.

[0133] Electronic devices may have collected ambient sounds over a considerable period, while the vehicle start-up sound typically lasts only 2-3 seconds. To avoid the electronic device performing audio recognition on all collected ambient sounds, when the ambient sound intensity is determined to be greater than an ambient sound intensity threshold, the ambient sounds within a target duration prior to that moment can be acquired. For example, the ambient sounds within the first 5 seconds of that moment can be acquired. Alternatively, if the loudness is detected to be greater than the ambient sound intensity threshold at the instant the vehicle starts, to ensure recognition accuracy, the ambient sounds within the target duration before and after that moment can be acquired separately. For example, the ambient sounds within the first 5 seconds and the last 5 seconds of that moment can be acquired, thereby enabling audio recognition of the acquired ambient sounds.

[0134] Step 703: If the audio recognition result indicates that the ambient sound includes the vehicle start-up sound, determine the positional relationship with the target vehicle based on the loudness of the vehicle start-up sound.

[0135] The implementation method of this step can refer to steps 402 to 403 above, and will not be repeated in this embodiment.

[0136] Step 704: When the positional relationship meets the warning prompt conditions, determine the warning prompt method based on the loudness of the vehicle start sound, wherein the warning intensity corresponding to the warning prompt method is positively correlated with the loudness of the vehicle start sound.

[0137] In one possible implementation, when the positional relationship meets the warning prompt conditions, it indicates that the electronic device is located in the danger zone of the target vehicle. At this time, the higher the volume of the vehicle's starting sound, the higher its danger factor. For different vehicle starting sounds, the electronic device can use different warning prompt methods to provide prompts. The higher the volume of the vehicle starting sound, the higher the warning intensity, so that the target object can pay attention to the surrounding starting vehicles in time.

[0138] In one possible implementation, when the electronic device determines the warning prompt mode based on the loudness of the vehicle start-up sound, it can determine the warning prompt mode based on the difference between the loudness of the vehicle start-up sound and the start-up sound loudness threshold. Optionally, different difference levels can be preset according to the difference, and corresponding warning intensities can be set for different difference levels, with the difference level and the warning intensity being positively correlated. Illustratively, when the difference belongs to the first difference level (0-5dB), the warning intensity can be determined as the first warning intensity; when the difference belongs to the second difference level (5-10dB), the warning intensity can be determined as the second warning intensity; when the difference belongs to the third difference level (10-15dB), the warning intensity can be determined as the third warning intensity, wherein the third warning intensity > the second warning intensity > the first warning intensity.

[0139] In another possible implementation, different loudness-distance models correspond to different vehicle types. Once the loudness of the vehicle's start-up sound is obtained, the specific distance to the target vehicle can be determined based on the corresponding loudness-distance model, and the warning notification method can be determined based on the distance. Illustratively, for the loudness-distance model corresponding to the target vehicle type, the corresponding vehicle start-up sounds can be collected in advance at different distances from vehicles belonging to the target vehicle type, and the correspondence between the loudness of the vehicle start-up sound and the distance can be fitted to obtain the loudness-distance model. Determining the warning notification method based on the loudness-distance model corresponding to the vehicle type may include the following steps:

[0140] Step 704a: Based on the target vehicle type indicated by the vehicle start sound, determine the loudness distance model corresponding to the target vehicle. The loudness distance model is used to indicate the correspondence between distance and loudness.

[0141] In one possible implementation, the electronic device has pre-set loudness distance models corresponding to different vehicle types. Optionally, the electronic device may have pre-set loudness distance models corresponding to different vehicle powertrains, for example, including loudness distance models corresponding to electric vehicles, gasoline vehicles, and diesel vehicles. Alternatively, it may have pre-set loudness distance models corresponding to different vehicle classes, or it may have pre-set loudness distance models corresponding to both different vehicle powertrains and different vehicle classes.

[0142] Once the target vehicle type is determined, the corresponding loudness distance model can be obtained.

[0143] Step 704b: Based on the loudness distance model, determine the target distance corresponding to the loudness of the vehicle start-up sound. The target distance is the distance between the vehicle and the target vehicle.

[0144] Based on the loudness distance model, electronic devices can determine the target distance corresponding to the vehicle start-up sound, thereby obtaining the distance between the electronic device and the target vehicle.

[0145] Step 704c: Based on the target distance, determine the warning notification method. The warning intensity of the warning notification method is positively correlated with the target distance.

[0146] Once the target distance is determined, the corresponding warning notification method can be determined based on the target distance. Optionally, the distance can be pre-divided into different distance levels, and a warning intensity corresponding to each distance level can be set. Distance and distance level are negatively correlated; the farther the distance, the lower the distance level. Conversely, distance level and warning intensity are positively correlated. For example, when the distance is at the first distance level (5-10m), the warning intensity can be set to the first warning intensity; when the distance is at the second distance level (2-5m), the warning intensity can be set to the second warning intensity; and when the distance is at the third distance level (0-2m), the warning intensity can be set to the third warning intensity. The third warning intensity > the second warning intensity > the first warning intensity.

[0147] Optionally, the warning notification method can be at least one of the following: notification duration, notification loudness, and notification content. When the warning notification method is notification duration, different warning intensities can be represented by different warning durations, and the warning intensity and warning duration are positively correlated. For example, the warning duration corresponding to the first warning intensity can be 2 seconds, the warning duration corresponding to the second warning intensity can be 5 seconds, and the warning duration corresponding to the third warning intensity can be 10 seconds.

[0148] When the warning notification method is based on loudness, different warning intensities can be represented by different alarm loudness levels, and there is a positive correlation between warning intensity and alarm loudness. For example, the warning loudness corresponding to the first warning intensity can be 30dB, the warning loudness corresponding to the second warning intensity can be 40dB, and the warning duration corresponding to the third warning intensity can be 50dB.

[0149] When the warning notification method is a notification content, different warning loudness can be represented by different warning content. For example, the warning content for a higher warning intensity is more than the warning content for a lower warning intensity.

[0150] In one possible implementation, the electronic device may be in a relatively quiet environment where the target object is likely to hear the vehicle starting sound, thus eliminating the need for the electronic device to issue a warning and avoiding unnecessary alerts.

[0151] Optionally, if the location relationship meets the warning prompt conditions, the ambient sound level of the ambient sound can be obtained.

[0152] When the location relationship meets the warning prompt conditions, the electronic device further acquires the ambient sound level, and then determines whether it is in a quiet environment based on the difference between the ambient sound level and the loudness of the vehicle start sound. When in a non-quiet environment, the target object may not be able to hear the vehicle start sound in time, therefore, a warning prompt is required. Optionally, a warning prompt is issued when the loudness difference is greater than a loudness difference threshold, where the loudness difference is the difference between the ambient sound level and the loudness of the vehicle start sound. In one possible implementation, the electronic device has a pre-set loudness difference threshold, for example, a third loudness threshold of 5 dB. When the loudness difference is greater than the loudness difference threshold, it is determined that the target object is in a non-quiet area, and a warning prompt is required.

[0153] When the electronic device is located in a quiet environment, in one possible implementation, no warning is required. In another possibility, the electronic device may be close to the target vehicle, posing a significant safety hazard; in this case, a warning may be issued even if the device is in a quiet environment. Optionally, a warning may be issued if the loudness difference is less than a third loudness threshold and the target distance is less than a distance threshold.

[0154] Electronic devices can also preset distance thresholds, for example, a distance threshold of 2 meters. If the target distance is determined to be less than the threshold, a warning will be issued. The target distance is the distance determined based on the loudness-distance model.

[0155] Step 705: If the location relationship meets the warning prompt conditions, determine the warning prompt method based on the vehicle type of the target vehicle. Different vehicle types correspond to different warning prompt methods.

[0156] In the above method, when the location relationship meets the warning prompt conditions, the electronic device determines the warning prompt method based on the loudness of the vehicle's start-up sound. In another possible implementation, the electronic device can also determine the warning prompt method based on the target vehicle's vehicle type. Different warning prompt methods are preset for different vehicle types. The warning prompt methods corresponding to different vehicle types can be default settings or user-defined settings. Thus, different warning prompt methods can be used to indicate the type of vehicle starting up in the vicinity.

[0157] When the vehicle type is vehicle power, the electronic equipment can be pre-set with corresponding warning prompts for electric vehicles, gasoline vehicles and diesel vehicles. For example, different prompt durations, prompt loudness and prompt content can be set for each type.

[0158] When the vehicle type is a vehicle class, the electronic device can be pre-set with different warning prompts corresponding to different vehicle classes. Furthermore, there is a positive correlation between the vehicle class and the warning intensity; the higher the vehicle class, the higher the warning intensity. Therefore, when a large vehicle starts up near the target object, a stronger warning is issued to the target object, reminding it to pay attention to surrounding vehicles.

[0159] Step 706: Issue an early warning through a warning notification method.

[0160] Once the warning notification method is determined, the electronic device will issue a warning notification according to that method. The warning notification method can be either a warning notification method corresponding to the loudness of the vehicle's start-up sound or a warning notification method corresponding to the vehicle type.

[0161] Step 707: If the location relationship meets the early warning conditions, send an early warning message to the associated device. The associated device has the ability to receive messages sent by the electronic device.

[0162] In one possible implementation, if the location relationship meets the early warning conditions, in addition to issuing an early warning, an early warning message can also be sent to associated devices for further early warning.

[0163] The associated device has the ability to receive messages sent by the electronic device. Optionally, the associated device may have a communication connection with the electronic device. For example, if the electronic device and the associated device have a Bluetooth connection, the electronic device can send a warning message to the associated device when the location relationship meets the warning prompt conditions.

[0164] Alternatively, the associated device may have the same warning notification program as the electronic device, and the user accounts corresponding to the warning notification levels on both devices may be linked. For example, the electronic device may be worn by a child, and the associated device may be a device belonging to the child's parent. If the accounts within the warning notification programs on both devices are linked, and the location relationship meets the warning notification conditions, the child wearing the electronic device can send a warning notification message to the associated device.

[0165] Alternatively, the associated device is a device bound to an electronic device. When the electronic device is a smart wearable device with an integrated warning and alert mechanism, the associated device can be a terminal bound to the electronic device. When the location relationship meets the warning and alert conditions, the smart wearable device sends a warning and alert message to the associated device.

[0166] As an example, when a pet is wearing an electronic device, if a vehicle is detected starting near the pet, a warning message can be sent to the associated device, so that the pet owner can pay attention to the vehicle around the pet in time.

[0167] In this embodiment, the electronic device determines whether to perform audio recognition based on the ambient sound level, thereby avoiding continuous audio recognition and reducing the power consumption of the electronic device.

[0168] Furthermore, in this embodiment, when the positional relationship meets the warning prompt conditions, the electronic device can determine the warning prompt method based on the loudness of the vehicle's start-up sound, providing warning prompts with different intensities to further reduce safety hazards. In addition, the electronic device can also determine the warning prompt method based on the vehicle type, providing different warning prompts for different vehicle types. This allows the device to simultaneously warn of the type of vehicle starting, enabling the target to avoid the starting vehicle in a timely manner.

[0169] In the above embodiments, when a vehicle start-up sound is detected in the ambient sound, a warning is issued based on the relationship between the loudness of the vehicle start-up sound and a start-up sound loudness threshold. A warning is issued if the loudness of the vehicle start-up sound exceeds the start-up sound loudness threshold. However, judging solely by the loudness of the vehicle start-up sound may be inaccurate. Therefore, in another possible implementation, after determining the positional relationship with the target vehicle based on the loudness of the vehicle start-up sound, if the positional relationship meets the warning conditions, the distance to the target vehicle can be further determined to ascertain whether the vehicle is within the target vehicle's danger distance.

[0170] Optionally, ambient sound can be collected using at least two microphones. Upon detecting a vehicle start sound, the time difference and intensity difference between the vehicle start sounds collected by the two microphones can be used for sound source localization. Illustratively, when using two microphones, the distance difference between the two microphones can be determined based on the time difference. Let the vehicle start sound be the first coordinate position, the first microphone be the second coordinate position, and the second microphone be the third coordinate position. A first distance can be determined based on the first and second coordinate positions, and a second distance can be determined based on the first and third coordinate positions. A hyperbolic function can then be constructed based on the first distance, the second distance, and the distance difference to determine the location of the vehicle start sound. Alternatively, in another possible implementation, the vehicle start sound can be located based on the arrival time difference and the sound intensity difference. When using three microphones, the Time Difference of Arrival (TDOA) method can also be used to determine the location of the vehicle start sound. The above is only an illustrative explanation of the localization method and does not constitute a limitation.

[0171] After determining the location of the vehicle's start-up sound, its direction and distance can be obtained. Based on this distance, it can be further determined whether the vehicle is within a danger zone. If it is, a warning is issued. During the warning process, at least one of the following can be used: broadcasting the direction and distance of the vehicle's start-up sound via a loudspeaker, or displaying the direction and distance on a screen. Furthermore, during the display process, the position of the warning information on the screen can be determined based on the identified target vehicle's location, with the display position corresponding to the target vehicle's location.

[0172] like Figure 8 As shown, the electronic device is equipped with a first microphone 801 and a second microphone 802. The target vehicle 803 is located based on the time difference t between the first microphone 801 and the second microphone 802 in collecting the vehicle start-up sound. If the target vehicle is determined to be within a dangerous distance, a warning message 804 can be displayed on the electronic device's screen, such as "A vehicle is starting at 3m to the right front."

[0173] Alternatively, in another possible implementation, after detecting the vehicle start sound, the location of the vehicle start sound source can be determined directly based on the time difference of the vehicle start sound collected by multiple microphones. Based on the location of the sound source, it can be determined whether it is within the danger distance of the target vehicle. If it is determined to be within the danger distance of the target vehicle, a warning prompt is issued.

[0174] In this embodiment, the target vehicle is located by collecting the difference in vehicle start-up sound from at least two microphones, thereby obtaining the target vehicle's location and distance, further improving the accuracy of detecting whether a target object is near the target vehicle and improving the accuracy of early warning prompts.

[0175] In one possible implementation, the early warning notification method can be implemented as follows: Figure 9 As shown. It includes the following steps:

[0176] Step 901: Collect ambient sound using a microphone.

[0177] Electronic devices have built-in microphones for audio acquisition.

[0178] Step 902: Perform loudness detection to obtain the loudness of the ambient sound.

[0179] Loudness detection is the measurement of the decibel value of ambient sound. Electronic devices contain loudness detection systems that can measure the loudness of ambient sound.

[0180] Step 903: Determine whether the loudness of the ambient sound is greater than the ambient sound loudness threshold. If yes, proceed to step 904; otherwise, proceed to step 902.

[0181] Step 904: Input the ambient sound into the CRNN model for audio recognition and obtain the audio recognition result.

[0182] The electronic device incorporates an audio recognition system, which includes a CRNN model. The audio recognition system uses the CRNN model to perform audio recognition and obtain audio recognition results. These results may include the probability that the ambient sound contains different vehicle start-up sounds. Specifically, this could be the probability of containing start-up sounds from vehicles with different powertrains; or, the probability of starting sounds from vehicles with different vehicle classes; or, the probability of starting sounds from both different vehicle powertrains and vehicle classes.

[0183] Step 905: Determine the target vehicle type based on the target vehicle's start-up sound.

[0184] When the audio recognition result indicates that the probability of the target vehicle's start sound is greater than the probability threshold, it is determined that the ambient sound contains the target vehicle's start sound. Then, the electronic device can determine the target vehicle type based on the target vehicle's start sound.

[0185] Step 906: Determine the starting sound volume threshold based on the target vehicle type.

[0186] The electronic device has pre-stored start-up sound volume thresholds corresponding to different vehicle types, and can directly obtain the corresponding start-up sound volume thresholds based on the target vehicle type.

[0187] Step 907: Determine whether the target vehicle is within its danger zone. If yes, proceed to step 909; otherwise, proceed to step 902.

[0188] After determining the starting sound loudness threshold, the loudness detection system inside the electronic device can detect the loudness of the vehicle starting sound and compare the loudness of the vehicle starting sound with the starting sound loudness threshold. If the loudness of the vehicle starting sound is greater than the starting sound loudness threshold, it is determined that the vehicle is located in a danger zone.

[0189] Step 908: Issue a warning via loudspeaker.

[0190] Electronic devices are equipped with speakers to provide warnings.

[0191] Figure 10 This is a structural block diagram of a warning notification device provided in an exemplary embodiment of this application. The device includes:

[0192] The audio recognition module 1001 is used to perform audio recognition on the collected ambient sounds and obtain the audio recognition results.

[0193] The location determination module 1002 is used to determine the location relationship with a target vehicle based on the loudness of the vehicle start-up sound when the audio recognition result indicates that the ambient sound includes a vehicle start-up sound. The target vehicle is the vehicle that generated the vehicle start-up sound.

[0194] The early warning module 1003 is used to issue an early warning when the positional relationship meets the early warning conditions, and the early warning is used to indicate that there is a vehicle starting in the vicinity.

[0195] Optionally, the audio recognition module 1001 is also used for:

[0196] The ambient sound is identified by an audio recognition model to obtain an audio recognition result, wherein the ambient sound contains the vehicle start sound, or the ambient sound does not contain the vehicle start sound.

[0197] Optionally, the position determination module 1002 is further configured to:

[0198] If the audio recognition result indicates that the ambient sound includes the vehicle start-up sound, the loudness of the vehicle start-up sound is obtained;

[0199] The positional relationship with the target vehicle is determined based on the relationship between the loudness of the vehicle's start-up sound and the start-up sound loudness threshold.

[0200] Optionally, the audio recognition result may also include the probability that the ambient sound contains different vehicle start-up sounds;

[0201] Optionally, the device further includes:

[0202] A type determination model is used to determine the target vehicle type of a target vehicle based on the target vehicle start-up sound when the ambient sound contains the target vehicle start-up sound, wherein the probability that the ambient sound contains the target vehicle start-up sound is greater than a probability threshold.

[0203] A threshold determination module is used to determine the starting sound volume threshold based on the target vehicle type, wherein the vehicle type includes at least one of vehicle power and vehicle class, the vehicle class is obtained based on vehicle size, and the vehicle class is positively correlated with the vehicle size.

[0204] Optionally, the vehicle type refers to the vehicle power source, which includes electric vehicles, gasoline vehicles, and diesel vehicles.

[0205] Optionally, the threshold determination module is further configured to:

[0206] When the target vehicle type is the electric vehicle, the first value is determined as the start-up sound volume threshold.

[0207] When the target vehicle type is the gasoline vehicle, the second value is determined as the start-up sound volume threshold, and the second value is greater than the first value;

[0208] When the target vehicle type is the diesel vehicle, the third value is determined as the start-up sound loudness threshold, and the third value is greater than the second value.

[0209] Optionally, the vehicle type is a vehicle class;

[0210] Optionally, the threshold determination module is further configured to:

[0211] Based on the correspondence between the vehicle level and the start-up sound volume threshold, the start-up sound volume threshold corresponding to the target vehicle level is determined, and the start-up sound volume threshold is negatively correlated with the vehicle level.

[0212] Optionally, the position determination module 1002 is further configured to:

[0213] If the loudness of the vehicle start-up sound is greater than the start-up sound loudness threshold, it is determined that the vehicle is located within the danger zone of the target vehicle.

[0214] If the loudness of the vehicle start-up sound is less than the start-up sound loudness threshold, it is determined that the vehicle is outside the danger zone of the target vehicle.

[0215] The step of issuing a warning when the positional relationship meets the warning conditions includes:

[0216] If the location relationship indicates that the location is within the danger zone of the target vehicle, and the warning prompt condition is met, a warning prompt is issued.

[0217] Optionally, the audio recognition model includes a convolutional neural network (CNN) model and a recurrent neural network (RNN) model;

[0218] The audio recognition module 1001 is also used for:

[0219] The spectrogram of the ambient sound is segmented into frames to obtain a spectrogram sequence. The spectrogram contains both time-domain and frequency-domain information of the ambient sound.

[0220] The spectrum sequence is input into the CNN model for feature extraction to obtain the frequency domain features of the ambient sound.

[0221] The spectrogram sequence is input into the RNN model for feature extraction to obtain the temporal features of the ambient sound.

[0222] The frequency domain features and the time domain features are concatenated to obtain the audio features of the ambient sound.

[0223] The audio features are input into a fully connected layer and a softmax layer for classification to obtain the audio recognition result.

[0224] Optionally, the audio recognition module 1001 is further configured to:

[0225] Obtain the ambient sound level of the collected ambient sound;

[0226] When the ambient sound intensity is greater than the ambient sound intensity threshold, audio recognition is performed on the ambient sound to obtain the audio recognition result, wherein the ambient sound intensity threshold is determined based on the lowest loudness of the starting sound of different vehicles.

[0227] Optionally, the early warning module 1003 is further configured to:

[0228] When the positional relationship satisfies the warning prompting conditions, the warning prompting method is determined based on the loudness of the vehicle start sound, wherein the warning intensity corresponding to the warning prompting method is positively correlated with the loudness of the vehicle start sound, and the warning prompting method includes at least one of prompting duration, prompting loudness, and prompting content;

[0229] Warnings are issued using the aforementioned warning notification method.

[0230] Optionally, the early warning module 1003 is further configured to:

[0231] Based on the target vehicle type indicated by the vehicle start sound, a loudness distance model corresponding to the target vehicle is determined, wherein the loudness distance model is used to indicate the correspondence between distance and loudness;

[0232] Based on the loudness distance model, the target distance corresponding to the loudness of the vehicle start-up sound is determined, where the target distance is the distance between the vehicle and the target vehicle.

[0233] Based on the target distance, the warning notification method is determined, and the warning intensity of the warning notification method is positively correlated with the target distance.

[0234] Optionally, the early warning module 1003 is further configured to:

[0235] When the positional relationship satisfies the warning prompt conditions, the ambient sound level of the ambient sound is obtained;

[0236] If the loudness difference is greater than the loudness difference threshold, an early warning will be issued. The loudness difference is the difference between the loudness of the ambient sound and the loudness of the vehicle start sound.

[0237] or,

[0238] An early warning is issued when the loudness difference is less than the loudness difference threshold and the target distance is less than the distance threshold.

[0239] Optionally, the early warning module 1003 is further configured to:

[0240] When the positional relationship satisfies the warning prompting conditions, the warning prompting method is determined based on the vehicle type of the target vehicle. Different vehicle types correspond to different warning prompting methods. The warning prompting method includes at least one of the following: prompting duration, prompting loudness, and prompting content.

[0241] Warnings are issued using the aforementioned warning notification method.

[0242] Optionally, the device further includes:

[0243] The information sending module is used to send a warning message to an associated device when the location relationship meets the warning prompt conditions, wherein the associated device has the ability to receive messages sent by the electronic device.

[0244] In summary, in this embodiment, the electronic device can identify vehicle start-up sounds in the ambient sound environment. Upon detecting a vehicle start-up sound, it can further determine the positional relationship between itself and the target vehicle based on the loudness of the sound, and then determine whether to issue a warning based on the positional relationship. That is, the electronic device can detect the surrounding vehicle situation through simple audio recognition and loudness detection methods, and issue a timely warning when a vehicle is starting nearby. When the target is carrying an electronic device, a warning can be issued to the target, thereby reducing the safety hazards caused by vehicle starting.

[0245] It should be noted that the apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their implementation process can be found in the method embodiments, which will not be repeated here.

[0246] Please refer to Figure 11This diagram illustrates a structural block diagram of an electronic device 1100 provided in an exemplary embodiment of this application. The electronic device 1100 may be a portable mobile electronic device, such as a smartphone, tablet computer, Moving Picture Experts Group Audio Layer III (MP3) player, or Moving Picture Experts Group Audio Layer IV (MP4) player. The electronic device 1100 may also be referred to as a user device, portable electronic device, or other names.

[0247] Typically, electronic device 1100 includes a processor 1101 and a memory 1102.

[0248] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0249] The memory 1102 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one instruction, which is executed by the processor 1101 to implement the method provided in the embodiments of this application.

[0250] In some embodiments, the electronic device 1100 may further include: a peripheral device interface 1103 and at least one peripheral device, the at least one peripheral device including a microphone 1104 and a speaker 1105.

[0251] Peripheral interface 1103 can be used to connect at least one input / output (I / O) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0252] Microphone 1104 is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals which are then input to processor 1101 for processing. For stereo sound acquisition or noise reduction purposes, multiple microphones can be used, each positioned in a different part of the electronic device 1100. The microphone can also be an array microphone or an omnidirectional microphone.

[0253] The speaker 1105 is used to convert electrical signals from the processor 1101 into sound waves. The speaker can be a traditional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals into sound waves that are audible to humans, or it can convert electrical signals into sound waves that are inaudible to humans for purposes such as distance measurement.

[0254] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on the electronic device 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0255] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by the processor to implement the warning notification method described in the above embodiments.

[0256] According to one aspect of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium, a processor of an electronic device reading the computer instructions from the computer-readable storage medium, the processor executing the computer instructions, causing the electronic device to perform the warning prompting method provided in various optional implementations of the above aspect.

[0257] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0258] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for issuing early warnings, characterized in that, The method includes: The collected ambient sounds are subjected to audio recognition to obtain the audio recognition results; If the audio recognition result indicates that the ambient sound includes a vehicle start-up sound, the positional relationship with the target vehicle is determined based on the loudness of the vehicle start-up sound, wherein the target vehicle is the vehicle that generated the vehicle start-up sound. When the positional relationship meets the warning prompt conditions, based on the target vehicle type indicated by the vehicle start sound, a loudness distance model corresponding to the target vehicle is determined, the loudness distance model being used to indicate the correspondence between distance and loudness; based on the loudness distance model, a target distance corresponding to the loudness of the vehicle start sound is determined, the target distance being the distance between the target vehicle and the target vehicle; based on the target distance, a warning prompt method is determined, the warning intensity corresponding to the warning prompt method being positively correlated with the target distance, and the warning intensity corresponding to the warning prompt method being positively correlated with the loudness of the vehicle start sound, the warning prompt method including at least one of prompt duration, prompt loudness, and prompt content; The warning notification method is used to alert the presence of a vehicle starting up in the vicinity.

2. The method according to claim 1, characterized in that, The process of performing audio recognition on the collected ambient sounds to obtain audio recognition results includes: The ambient sound is identified by an audio recognition model to obtain an audio recognition result, wherein the ambient sound contains the vehicle start sound, or the ambient sound does not contain the vehicle start sound. When the audio recognition result indicates that the ambient sound includes a vehicle start-up sound, determining the positional relationship with the target vehicle based on the loudness of the vehicle start-up sound includes: If the audio recognition result indicates that the ambient sound includes the vehicle start-up sound, the loudness of the vehicle start-up sound is obtained; The positional relationship with the target vehicle is determined based on the relationship between the loudness of the vehicle's start-up sound and the start-up sound loudness threshold.

3. The method according to claim 2, characterized in that, The audio recognition result also includes the probability that the ambient sound contains different vehicle start-up sounds; The method further includes: If the ambient sound contains the target vehicle's start-up sound, the target vehicle type is determined based on the target vehicle's start-up sound, and the probability that the ambient sound contains the target vehicle's start-up sound is greater than a probability threshold. Based on the target vehicle type, the starting sound volume threshold is determined, wherein the vehicle type includes at least one of vehicle power and vehicle class, the vehicle class is determined based on vehicle size, and the vehicle class is positively correlated with the vehicle size.

4. The method according to claim 3, characterized in that, The vehicle type refers to the vehicle power source, which includes electric vehicles, gasoline vehicles, and diesel vehicles. Determining the start-up sound volume threshold based on the target vehicle type includes: When the target vehicle type is the electric vehicle, the first value is determined as the start-up sound volume threshold. When the target vehicle type is the gasoline vehicle, the second value is determined as the start-up sound volume threshold, and the second value is greater than the first value; When the target vehicle type is the diesel vehicle, the third value is determined as the start-up sound loudness threshold, and the third value is greater than the second value.

5. The method according to claim 4, characterized in that, The vehicle type is a vehicle class; Determining the start-up sound volume threshold based on the target vehicle type includes: Based on the correspondence between the vehicle level and the start-up sound volume threshold, the start-up sound volume threshold corresponding to the target vehicle level is determined, and the start-up sound volume threshold is negatively correlated with the vehicle level.

6. The method according to any one of claims 2 to 5, characterized in that, Determining the positional relationship with the target vehicle based on the relationship between the loudness of the vehicle's start-up sound and a start-up sound loudness threshold includes: If the loudness of the vehicle start-up sound is greater than the start-up sound loudness threshold, it is determined that the vehicle is located within the danger zone of the target vehicle. If the loudness of the vehicle start-up sound is less than the start-up sound loudness threshold, it is determined that the vehicle is outside the danger zone of the target vehicle. The step of issuing a warning when the positional relationship meets the warning conditions includes: If the location relationship indicates that the location is within the danger zone of the target vehicle, and the warning prompt condition is met, a warning prompt is issued.

7. The method according to any one of claims 2 to 5, characterized in that, The audio recognition model includes a convolutional neural network (CNN) model and a recurrent neural network (RNN) model. The step of performing audio recognition on the ambient sound using an audio recognition model to obtain the audio recognition result includes: The spectrogram of the ambient sound is segmented into frames to obtain a spectrogram sequence. The spectrogram contains both time-domain and frequency-domain information of the ambient sound. The spectrum sequence is input into the CNN model for feature extraction to obtain the frequency domain features of the ambient sound. The spectrogram sequence is input into the RNN model for feature extraction to obtain the temporal features of the ambient sound. The frequency domain features and the time domain features are concatenated to obtain the audio features of the ambient sound. The audio features are input into a fully connected layer and a softmax layer for classification to obtain the audio recognition result.

8. The method according to any one of claims 1 to 5, characterized in that, The process of performing audio recognition on the collected ambient sounds to obtain audio recognition results includes: Obtain the ambient sound level of the collected ambient sound; When the ambient sound intensity is greater than the ambient sound intensity threshold, audio recognition is performed on the ambient sound to obtain the audio recognition result, wherein the ambient sound intensity threshold is determined based on the lowest loudness of the starting sound of different vehicles.

9. The method according to claim 1, characterized in that, The method further includes: When the positional relationship satisfies the warning prompt conditions, the ambient sound level of the ambient sound is obtained; If the loudness difference is greater than the loudness difference threshold, an early warning will be issued. The loudness difference is the difference between the loudness of the ambient sound and the loudness of the vehicle start sound. or, An early warning is issued when the loudness difference is less than the loudness difference threshold and the target distance is less than the distance threshold.

10. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the positional relationship satisfies the warning prompting conditions, the warning prompting method is determined based on the vehicle type of the target vehicle. Different vehicle types correspond to different warning prompting methods. The warning prompting method includes at least one of the following: prompting duration, prompting loudness, and prompting content. Warnings are issued using the aforementioned warning notification method.

11. The method according to any one of claims 1 to 5, characterized in that, The method is performed by an electronic device, and the method further includes: When the location relationship meets the warning prompt conditions, a warning prompt message is sent to the associated device, which has the ability to receive messages sent by the electronic device.

12. A warning and alerting device, characterized in that, The device includes: The audio recognition module is used to perform audio recognition on the collected ambient sounds and obtain the audio recognition results. The location determination module is used to determine the location relationship with a target vehicle based on the loudness of the vehicle start-up sound when the audio recognition result indicates that the ambient sound includes a vehicle start-up sound. The target vehicle is the vehicle that generated the vehicle start-up sound. The warning module is configured to, when the location relationship meets the warning conditions, determine the loudness distance model corresponding to the target vehicle based on the target vehicle type indicated by the vehicle start sound, wherein the loudness distance model is used to indicate the correspondence between distance and loudness; determine the target distance corresponding to the loudness of the vehicle start sound based on the loudness distance model, wherein the target distance is the distance between the target vehicle and the target vehicle; and determine the warning method based on the target distance, wherein the warning intensity corresponding to the warning method is positively correlated with the target distance and the warning intensity corresponding to the warning method is positively correlated with the loudness of the vehicle start sound, wherein the warning method includes at least one of the following: warning duration, warning loudness, and warning content. The warning module is also used to issue a warning through the warning method, which is used to indicate that a vehicle is starting up in the vicinity.

13. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the warning notification method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the warning notification method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the warning notification method as described in any one of claims 1 to 11.

Images