Alarm information processing method and device, storage medium and electronic device

Abstract

The present disclosure relates to the field of vehicle electronic technology, and specifically to a warning prompt information processing method, a warning prompt information processing device, a storage medium and an electronic device. The method comprises: collecting environmental audio data, analyzing the environmental audio data to extract the target type of warning audio; determining the sound source orientation information of the warning audio; generating warning prompt information according to the sound source orientation information and displaying it in the graphical user interface of the head-up display device. This solution can identify and analyze the prompt sound in the driving environment and display the corresponding prompt information in the HUD device.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle electronic technology, and in particular to a warning prompt information processing method, a warning prompt information processing device, a storage medium and an electronic device. Background Art

[0002] When the vehicle is driving, the driver needs to pay attention to the changes in the surrounding environment in real time. During driving, surrounding vehicles may sound horns to warn, or some special vehicles may emit specific warning sounds during operation, such as ambulances, fire trucks, sprinkler trucks, etc. However, it is not certain whether the warning sounds in the environment and the vehicles that emit these warning sounds will affect the driving of the vehicle.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The present invention provides a warning prompt information processing method, a warning prompt information processing device, a storage medium and an electronic device, which can identify and analyze the prompt sound in the driving environment and display the corresponding prompt information in the HUD device.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, a method for processing alarm prompt information is provided, the method comprising:

[0007] Collecting environmental audio data, and analyzing the environmental audio data to extract target type of warning audio;

[0008] Determining the sound source location information of the warning audio;

[0009] Generate warning information according to the sound source orientation information and display it in a graphical user interface of a head-up display device.

[0010] In some exemplary embodiments, the collecting of environmental audio data and analyzing the environmental audio data to extract the target type of warning audio includes:

[0011] Collect environmental audio data using microphone arrays at preset locations on the vehicle;

[0012] Performing denoising processing on the environmental audio data, and performing audio separation processing on the denoised environmental audio data to obtain at least one single audio data;

[0013] Parsing the single audio data to extract corresponding audio feature data;

[0014] The audio feature data is compared with an audio database to determine whether the single audio data is a warning audio of a target type.

[0015] In some exemplary embodiments, the warning prompt information includes warning monitoring area information and a warning icon displayed in a graphical user interface of a head-up display device.

[0016] In some exemplary embodiments, the method further comprises:

[0017] According to the audio type of the alarm audio, a corresponding alarm icon type is determined.

[0018] In some exemplary embodiments, generating warning prompt information according to the sound source orientation information includes:

[0019] Determine the alarm monitoring area according to the sound source location information according to preset rules;

[0020] Obtaining location information of other vehicles in the alarm monitoring area and configuring them as alarm monitoring vehicles;

[0021] The display content of the alarm prompt information is determined in combination with the audio type of the alarm audio and the alarm monitoring area.

[0022] In some exemplary embodiments, the determining the display content of the alarm prompt information in combination with the audio type of the alarm audio and the alarm monitoring area includes:

[0023] When the warning audio is of the first warning type, the display content of the warning prompt information is configured to include the warning monitoring area and at least one warning monitoring vehicle identification in the warning monitoring area; or

[0024] When the alarm audio is of the second alarm type, the display content of the alarm prompt information is configured to include the alarm monitoring area and the special vehicle identification in the alarm monitoring area.

[0025] In some exemplary embodiments, the method further comprises:

[0026] In response to the vehicle position update, updating the position information of the alarm monitoring area and the alarm monitoring vehicle;

[0027] Based on the updated vehicle position information and the position information of the alarm monitoring vehicle, the display content of the alarm prompt information is updated.

[0028] According to a second aspect of the present disclosure, there is provided a warning prompt information processing device, the device comprising:

[0029] An audio data acquisition module is used to collect environmental audio data and analyze the environmental audio data to extract the target type of warning audio;

[0030] A sound source orientation recognition module, used to determine the sound source orientation information of the alarm audio;

[0031] The display content calculation module is used to generate warning prompt information according to the sound source orientation information and display it in the graphical user interface of the head-up display device.

[0032] According to a third aspect of the present disclosure, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned alarm prompt information processing method is implemented.

[0033] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0034] Processor; and

[0035] A memory, configured to store executable instructions of the processor;

[0036] Wherein, the processor is configured to implement the above-mentioned alarm prompt information processing method by executing the executable instructions.

[0037] An embodiment of the present disclosure provides a method for processing warning information, which collects ambient audio in real time, analyzes the ambient audio to determine whether the current environment contains warning audio of the target type, and can first determine the corresponding sound source direction when the warning audio is contained, and then generate warning information and display it in the graphical user interface of the head-up display device. In this way, the audio warning information in the driving environment is converted into the corresponding image display content, so as to provide the driver with a warning prompt in the graphical user interface of the HUD.

[0038] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0040] Figure 1 A schematic diagram schematically illustrates a method for processing warning prompt information in an exemplary embodiment of the present disclosure;

[0041] Figure 2 A schematic diagram schematically illustrates a method for processing single audio data in an exemplary embodiment of the present disclosure;

[0042] Figure 3 A schematic diagram schematically illustrates a display area division in an exemplary embodiment of the present disclosure;

[0043] Figure 4 A schematic diagram schematically illustrates an alarm monitoring area in an exemplary embodiment of the present disclosure;

[0044] Figure 5 A schematic diagram schematically illustrates a display effect of a whistle-type warning display content in an exemplary embodiment of the present disclosure;

[0045] Figure 6 A schematic diagram schematically illustrates a display effect of a special type of alarm display content in an exemplary embodiment of the present disclosure;

[0046] Figure 7 A schematic diagram schematically illustrates a method for updating warning prompt information based on location update in an exemplary embodiment of the present disclosure;

[0047] Figure 8 A schematic diagram schematically illustrates a display effect of updating an alarm monitoring area in an exemplary embodiment of the present disclosure;

[0048] Fig. 9 A schematic diagram schematically showing a change in a vehicle position coordinate in an exemplary embodiment of the present disclosure;

[0049] Fig.10 A schematic diagram schematically illustrates a calculation of the distance difference S3 in an exemplary embodiment of the present disclosure;

[0050] Fig.11 A schematic diagram schematically illustrates another method of calculating the distance difference S3 in an exemplary embodiment of the present disclosure;

[0051] Fig.12 A schematic diagram schematically illustrating a display change of multiple display contents in an exemplary embodiment of the present disclosure;

[0052] Fig.13 A schematic diagram schematically shows the composition of an alarm prompt information processing device in an exemplary embodiment of the present disclosure;

[0053] Fig.14 The diagram schematically shows the components of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0054] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0055] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0056] In view of the shortcomings and deficiencies of the prior art, this example implementation provides a method for processing warning information, which can be applied to HUD and AR-HUD devices on vehicles, and can display vehicle warning audio information in the external environment in the driver's field of vision in the form of image animation, thereby realizing an auxiliary prompt function. Figure 1 As shown in , the above-mentioned warning prompt information processing method may include:

[0057] Step S11, collecting environmental audio data, and analyzing the environmental audio data to extract the target type of warning audio;

[0058] Step S12, determining the sound source location information of the alarm audio;

[0059] Step S13: generating warning prompt information according to the sound source orientation information and displaying it in a graphical user interface of a head-up display device.

[0060] The warning prompt information processing method provided in this example implementation collects ambient audio in real time, analyzes the ambient audio, and determines whether the current environment contains the target type of warning audio, so that when the warning audio is contained, the corresponding sound source orientation can be first determined, and then the warning prompt information can be generated and displayed in the graphical user interface of the head-up display device. In this way, the audio warning information in the driving environment is converted into the corresponding image display content, so as to provide the driver with a warning prompt in the graphical user interface of the HUD.

[0061] Below, each step of the method for processing alarm prompt information in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0062] In step S11, environmental audio data is collected and analyzed to extract warning audio of a target type.

[0063] In this example implementation, reference Figure 2 As shown, the above step S11 may include:

[0064] Step S21, collecting ambient audio data using a microphone array at a preset position of the vehicle;

[0065] Step S22, performing denoising processing on the ambient audio data, and performing audio separation processing on the denoised ambient audio data to obtain at least one single audio data;

[0066] Step S23, parsing the single audio data to extract corresponding audio feature data;

[0067] Step S24: compare the audio feature data with an audio database to determine whether the single audio data is an alarm audio of the target type.

[0068] Specifically, the above method can be applied to vehicle-mounted terminal equipment, intelligent vehicle system, HUD equipment, or intelligent mobile terminal equipment that communicates data with the vehicle system. Specifically, during driving, after starting the vehicle's vehicle system, or after starting the HUD device, the real-time collection of ambient audio data can be triggered. One or more groups of microphone arrays can be pre-installed on the vehicle to collect ambient audio. A microphone array can be constructed by installing microphones at multiple angles and multiple directions in the vehicle. Alternatively, the ambient audio data can be collected in real time through other audio collection devices installed in the vehicle.

[0069] The collected ambient audio can first be stored in the storage device of the vehicle system and calculated by the intelligent vehicle system, or it can be sent to an intelligent terminal device connected to the vehicle system, which will recognize the alarm audio and calculate the sound source direction information.

[0070] Taking the car system as an example, the real-time collected ambient audio data can be first denoised. For example, a real-time noise reduction algorithm based on a statistical model can be used to perform real-time noise reduction on the currently collected ambient audio data. Of course, the ambient audio data can also be processed in real time using a microphone array noise reduction algorithm or other methods.

[0071] For the ambient audio data after denoising, a frequency divider can be used to perform multi-audio separation processing. There are usually multiple audio information in the external environment at the same time, and most of the collected ambient audio data is in the form of multi-audio mixture. In order to determine whether there is sound information that can cause an alarm in the current environment, it is necessary to first process the noisy ambient sound, separate it into multiple single audio information, and then perform category judgment on each individual audio information. For example, the mixed signal separation algorithm can be used to separate the ambient audio data. For example, the audio separation library can also be used to separate the ambient audio data.

[0072] In addition, an audio database may be pre-established to store various types of special vehicle warning sounds, audio data of common vehicle whistle sounds, and audio feature data corresponding to the audio data. Special vehicles may include warning audio data corresponding to police cars, fire trucks, ambulances, sprinkler trucks, and other types of vehicles. The audio feature data may include frequency features, loudness features, timbre features, and other feature data of various types of warning audio.

[0073] Specifically, for at least one single audio data separated from the currently collected environmental audio data, similarity judgment is performed in turn with the audio information in the audio database; wherein, the judgment dimension may include: one or more of frequency, loudness, and timbre, and the judgment result is used to determine whether the single audio belongs to the warning audio and whether it belongs to the car horn category in the warning audio or the prompt audio category of a special vehicle. For example, after traversing all the single audios collected this time, if it is found that there is no warning audio this time, the warning prompt may not be displayed; or, if there is warning audio, the direction relationship between the audio sound position and the vehicle can be obtained based on the sound source positioning method. For example, an audio processing model based on dynamic time warping (DTW) can be used to judge and analyze audio similarity.

[0074] In step S12, the sound source orientation information of the warning audio is determined.

[0075] In this example implementation, when it is determined that a single audio data includes an alarm audio, the direction of the sound source corresponding to the alarm audio can be calculated. Specifically, the directional angle value of the alarm audio source can be obtained by sound source positioning, and the audio direction can be obtained by a method based on relative delay estimation. Due to the geometric structure of the microphone array, the signals received by each array have different degrees of delay, and the method based on relative delay estimation estimates the delay difference between each array signal through cross-correlation, generalized cross-correlation or phase difference, and then combines the geometric structure of the array to estimate the azimuth information of the sound source.

[0076] In step S13, warning prompt information is generated according to the sound source orientation information and displayed in a graphical user interface of a head-up display device.

[0077] In this example implementation, the generating of the warning prompt information according to the sound source orientation information includes:

[0078] Step S31, determining the alarm monitoring area according to the sound source orientation information according to a preset rule;

[0079] Step S32, obtaining the position information of other vehicles in the alarm monitoring area and configuring them as alarm monitoring vehicles;

[0080] Step S33, determining the display content of the alarm prompt information in combination with the audio type of the alarm audio and the alarm monitoring area.

[0081] In this example implementation, the method further includes: determining a corresponding alarm icon type according to an audio type of the alarm audio.

[0082] In this example implementation, the method also includes: determining the display content of the alarm prompt information in combination with the audio type of the alarm audio and the alarm monitoring area, including: when the alarm audio is of the first alarm type, configuring the display content of the alarm prompt information to include the alarm monitoring area, and at least one alarm monitoring vehicle identification within the alarm monitoring area; or, when the alarm audio is of the second alarm type, configuring the display content of the alarm prompt information to include the alarm monitoring area, and the special vehicle identification within the alarm monitoring area.

[0083] Specifically, the warning prompt information includes warning monitoring area information and a warning icon displayed in a graphical user interface of a head-up display device.

[0084] Specifically, the graphical user interface of the HUD device may be divided into display areas in advance. Of course, in some exemplary embodiments, the graphical user interface may also be a graphical user interface of an AR-HUD device, an electronic rearview mirror, a vehicle built-in display screen, or AR glasses. Figure 3 As shown, it can be divided into two sub-areas, namely, the vehicle identification area and the alarm display area; wherein, the vehicle position will be marked in the form of a fixed position icon in the vehicle identification area, so that the driver can clearly understand the relative position relationship between the alarm position and the vehicle; the alarm display area can be used to display the source and type of the alarm. For example, if the alarm audio is a vehicle horn category, the alarm source area and the vehicle information in the area are presented in this area; if the alarm audio is a special vehicle prompt category, the alarm source area and the special vehicle information in the area are presented in this area.

[0085] Specifically, if the current alarm audio is a vehicle horn type, that is, the first alarm type mentioned above, then all vehicle motion information corresponding to the direction range can be captured according to the alarm source direction. For example, ADAS, infrared ranging, multi-camera ranging, etc. can be used to capture external vehicle information within a preset degree range of, for example, 15 degrees to the left and right of the alarm audio direction and a preset distance range of, for example, 15m, 30m, 50m, or 150m from the vehicle, and the area obtained by angle and distance restrictions is set as the alarm source area as the alarm monitoring area, for example, reference Figure 4 The schematic diagram of the alarm monitoring area is shown. The detected vehicles in the area are recorded as a display vehicle set, and the straight-line distance value between each vehicle in the set and the vehicle and the corresponding body position coordinates are monitored and recorded in real time.

[0086] refer to Figure 5 As shown, if the audio alarm category is a vehicle horn category, the obtained alarm monitoring area and the displayed vehicle set will display the alarm area logo and the position logo of each vehicle in the displayed vehicle set relative to the vehicle at the corresponding position in the display area.

[0087] Alternatively, if the current warning audio is a special vehicle prompt category, that is, the second warning type mentioned above, then the area with a preset degree of, for example, 15 degrees to the left and right of the warning audio direction and a preset distance of, for example, 150m from the vehicle can be used as the warning source area according to the direction of the warning source, and regional vehicle monitoring is not performed on this area, and the warning area is used as the prompt display output content. Figure 6 As shown, the warning source area can be marked with a corresponding area range in the display area and a special vehicle prompt mark can be added to the area.

[0088] In this example implementation, reference Figure 7 As shown, the method also includes:

[0089] Step S41, in response to the vehicle position update, updating the position information of the alarm monitoring area and the alarm monitoring vehicle;

[0090] Step S42, based on the updated vehicle position information and the position information of the alarm monitoring vehicle, updating the display content of the alarm prompt information.

[0091] Specifically, the position of the vehicle will change in real time during driving, and the corresponding warning monitoring area and vehicle set can be updated. The update cycle can be pre-configured, such as setting a data update cycle based on time or driving distance; for example, the update cycle can be configured to a time interval of 3 seconds, 5 seconds or 1 second; or the update cycle can be configured to a driving distance of 15m, 50m, etc.

[0092] Due to the movement of the vehicle's position caused by its own driving behavior, the captured alarm source direction data will change with the angle. On this basis, the alarm monitoring area needs to be refreshed and changed synchronously. If a vehicle that was in the monitoring range before the refresh is no longer in the monitoring range after the refresh, the vehicle information will be removed from the displayed vehicle set. For example, refer to Figure 8 As shown, when a vehicle is not within the warning monitoring area after the update, the icon of the vehicle can be cleared from the display content.

[0093] Alternatively, if the vehicles in the monitoring range before the refresh are still in the monitoring range after the refresh, then the position change trajectory of each vehicle in the displayed vehicle set is used to determine whether each vehicle will have a dangerous impact on the driving of the vehicle. Fig. 9 As shown in FIG. 1 , a functional relationship is established between the current body position coordinates of each vehicle and the vehicle obtained through real-time monitoring and the body position coordinates of the previous frames, thereby obtaining the motion trajectory relationship formula of each vehicle. If it is determined that the vehicle and the vehicle do not have a trajectory intersection based on the motion trajectory of each vehicle, the vehicle is removed from the displayed vehicle set.

[0094] Alternatively, the distance difference between the monitoring vehicle and the host vehicle when the monitoring vehicle and the host vehicle move to a perpendicular relationship in the x direction may be calculated based on the movement trajectories of the vehicles and the host vehicle.

[0095] The current speed of the monitored vehicle is calculated synchronously based on the real-time position change of the monitored vehicle. The travel time T required for the monitored vehicle to move to 0 in the y direction and the corresponding x-direction distance value S1 on the coordinate axis at this time are calculated through the vehicle speed and motion trajectory function. The corresponding moving distance value S2 of the vehicle in the x direction under the travel time T is calculated based on the obtained travel time and the vehicle speed. The distance difference S3 between S1 and S2 is calculated.

[0096] Specifically, the movement trajectory of the current alarm monitoring vehicle can be monitored; the movement distance of the vehicle in the x direction can be monitored; the movement distance of the own vehicle can be calculated at the same time; and the distance difference between the monitoring vehicle and the own vehicle when the two vehicles move to a vertical relationship in the x direction.

[0097] For example, refer to Fig.10 , Fig.11 As shown in the distance calculation diagram, the x direction is the forward direction of the vehicle, the y direction is the horizontal direction perpendicular to the forward direction of the vehicle, S0 is the movement trajectory of the monitored vehicle, and the distance S3 value is judged. If S3 is greater than the safety distance value (the safety distance value is not limited and can be adjusted according to actual conditions), the vehicle is removed from the display set and no prompt is displayed for the vehicle.

[0098] In other exemplary embodiments of the present disclosure, when an alarm message is received, the corresponding content is displayed in the display area according to the above-mentioned display logic, and the display duration is set to T seconds and a countdown is performed. If similar alarm audio can still be received from the same direction during the T-second display process, the countdown for the remaining display duration is reset to T seconds and the countdown is performed again. If similar alarm audio is not received from the same direction range during the countdown, the alarm display content disappears when the countdown value is 0.

[0099] refer to Fig.12 As shown, if new external alarm audio is detected during the display process, the corresponding new display content is obtained according to the above logic, and the new display content is added to the corresponding position of the display area and a new countdown T value is created for it. The display continuation and disappearance of the two display contents are controlled by their respective T values. When the display content is added, if the categories of the two display contents are consistent, it is directly added. If the categories of the two display contents are inconsistent, it is necessary to determine whether there is a position overlap in the area where the two received display contents are located. If there is no overlap, the display content is directly added. If there is an overlap, the special alarm category content is displayed first.

[0100] The warning prompt information processing method provided by the present invention identifies and captures the vehicle horn sounds and sound prompts of other special vehicles in the external environment, thereby obtaining the warning direction range and the position information of other vehicles within the range that may affect the driving of the vehicle, and converts the audio warning information into corresponding image display content, thereby providing a warning prompt to the driver.

[0101] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0102] For further reference, Fig.13 As shown, in the implementation of this example, a warning prompt information processing device 130 is also provided, including: an audio data acquisition module 1301, a sound source orientation recognition module 1302, and a sound source orientation recognition module 1303. Among them,

[0103] The audio data collection module 1301 can be used to collect environmental audio data and analyze the environmental audio data to extract the target type of warning audio.

[0104] The sound source location identification module 1302 may be used to determine the sound source location information of the warning audio.

[0105] The display content calculation module 1303 may be configured to generate warning prompt information according to the sound source orientation information and display the warning prompt information in a graphical user interface of a head-up display device.

[0106] In this example implementation, the audio data acquisition module 1301 may include: collecting ambient audio data using a microphone array at a preset position of the vehicle; denoising the ambient audio data, and performing audio separation processing on the denoised ambient audio data to obtain at least one single audio data; parsing the single audio data to extract corresponding audio feature data; comparing the audio feature data with an audio database to determine whether the single audio data is a warning audio of the target type.

[0107] In this example implementation, the warning prompt information includes warning monitoring area information and a warning icon displayed in a graphical user interface of a head-up display device.

[0108] In this example implementation, the device further includes: an alarm icon type configuration module.

[0109] The alarm icon type configuration module may be configured to determine a corresponding alarm icon type according to an audio type of the alarm audio.

[0110] In this example implementation, the display content calculation module 1303 may include: determining the alarm monitoring area according to preset rules based on the sound source orientation information; obtaining the position information of other vehicles in the alarm monitoring area and configuring them as alarm monitoring vehicles; and determining the display content of the alarm prompt information in combination with the audio type of the alarm audio and the alarm monitoring area.

[0111] In this example implementation, the display content calculation module 1303 may include: when the alarm audio is of the first alarm type, configuring the display content of the alarm prompt information to include the alarm monitoring area, and at least one alarm monitoring vehicle identification within the alarm monitoring area; or, when the alarm audio is of the second alarm type, configuring the display content of the alarm prompt information to include the alarm monitoring area, and the special vehicle identification within the alarm monitoring area.

[0112] In this example implementation, the device further includes: an update processing module.

[0113] The update processing module may be used to update the position information of the alarm monitoring area and the alarm monitoring vehicle in response to the vehicle position update;

[0114] Based on the updated vehicle position information and the position information of the alarm monitoring vehicle, the display content of the alarm prompt information is updated.

[0115] The specific details of each module in the above-mentioned alarm prompt information processing device 130 have been described in detail in the corresponding alarm prompt information processing method, so they will not be repeated here.

[0116] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0117] Fig.14 A schematic diagram of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0118] It should be noted that Fig.14 The electronic device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0119] like Fig.14 As shown, the electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, ROM 1002 and RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0120] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0121] In particular, according to an embodiment of the present invention, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 1009, and / or installed from a removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0122] Specifically, the electronic device may be a smart mobile electronic device such as a mobile phone, a tablet computer or a laptop computer, or may be a smart electronic device such as a desktop computer.

[0123] It should be noted that the storage medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any storage medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0124] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0126] It should be noted that, as another aspect, the present application also provides a storage medium, which can be included in an electronic device; or it can exist independently without being installed in the electronic device. The above storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments. For example, the electronic device can implement the following Figure 1 The steps shown.

[0127] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0128] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0129] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing alarm information, characterized in that: The method comprises: Collecting environmental audio data, and analyzing the environmental audio data to extract target type of warning audio; Determining the sound source location information of the warning audio; Generate warning information according to the sound source orientation information and display it in a graphical user interface of a head-up display device.

2. The method for processing alarm information according to claim 1, characterized in that: The collecting of environmental audio data and analyzing the environmental audio data to extract the target type of warning audio includes: Collect environmental audio data using microphone arrays at preset locations on the vehicle; Performing denoising processing on the environmental audio data, and performing audio separation processing on the denoised environmental audio data to obtain at least one single audio data; Parsing the single audio data to extract corresponding audio feature data; The audio feature data is compared with an audio database to determine whether the single audio data is a warning audio of a target type.

3. The method for processing alarm information according to claim 1, characterized in that: The warning prompt information includes warning monitoring area information and a warning icon displayed in a graphical user interface of a head-up display device.

4. The method for processing warning information according to claim 3, characterized in that: The method further comprises: According to the audio type of the alarm audio, a corresponding alarm icon type is determined.

5. The method for processing warning information according to claim 1 or 4, characterized in that: The generating of warning prompt information according to the sound source orientation information includes: Determine the alarm monitoring area according to the sound source location information according to preset rules; Obtaining location information of other vehicles in the alarm monitoring area and configuring them as alarm monitoring vehicles; The display content of the alarm prompt information is determined in combination with the audio type of the alarm audio and the alarm monitoring area.

6. The method for processing warning information according to claim 5, characterized in that: The determining the display content of the alarm prompt information in combination with the audio type of the alarm audio and the alarm monitoring area includes: When the warning audio is of the first warning type, the display content of the warning prompt information is configured to include the warning monitoring area and at least one warning monitoring vehicle identification in the warning monitoring area; or When the alarm audio is of the second alarm type, the display content of the alarm prompt information is configured to include the alarm monitoring area and the special vehicle identification in the alarm monitoring area.

7. The method for processing warning information according to claim 5, characterized in that: The method further comprises: In response to the vehicle position update, updating the position information of the alarm monitoring area and the alarm monitoring vehicle; Based on the updated vehicle position information and the position information of the alarm monitoring vehicle, the display content of the alarm prompt information is updated.

8. An alarm information processing device, characterized in that: The device comprises: An audio data acquisition module is used to collect environmental audio data and analyze the environmental audio data to extract the target type of warning audio; A sound source orientation recognition module, used to determine the sound source orientation information of the alarm audio; The display content calculation module is used to generate warning prompt information according to the sound source orientation information and display it in the graphical user interface of the head-up display device.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the alarm prompt information processing method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the warning prompt information processing method according to any one of claims 1 to 7 by executing the executable instructions.

Images