Delivery vehicle risk prompting method, device, equipment, medium and delivery vehicle
By obtaining and analyzing audio data outside the vehicle, determining whether there is a risk in the target obstacle, and providing risk warnings, the safety problems caused by the driver's field of vision in bad weather are solved and driving safety is improved.
Patent Information
- Application Number
- CN202311034819.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-16
- Publication Date
- 2025-02-25
AI Technical Summary
In severe weather conditions, the driver is unable to respond in a timely manner due to limited vision, resulting in lower driving safety.
By obtaining external audio data outside the vehicle, determining the target obstacles related to abnormal audio data, and determining whether there is a risk based on the trajectory of the vehicle and the target obstacle, thereby conducting risk warnings.
Improves drivers' driving safety in bad weather conditions, helping drivers respond in a timely manner by prompting potential risks in advance.
Smart Images
Figure CN119516814A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the fields of intelligent transportation, computer vision, and voice technology, and in particular to a vehicle risk warning method, device, equipment, medium, program product, and vehicle. Background Art
[0002] When driving, users need to observe surrounding vehicles, pedestrians, road conditions and other environmental information, and then make appropriate driving behaviors. However, sometimes due to poor vision caused by weather and other reasons, users cannot respond in time based on the surrounding environment, and driving safety is low. Summary of the invention
[0003] In view of the above problems, the present disclosure provides a vehicle risk warning method, device, equipment, medium, program product and vehicle.
[0004] According to one aspect of the present disclosure, a vehicle risk warning method is provided, comprising: acquiring external audio data from outside the vehicle; determining a target obstacle related to the abnormal audio data based on the external audio data when it is determined that the external audio data includes abnormal audio data; determining whether the target obstacle poses a risk to the vehicle based on a first trajectory of the vehicle and a second trajectory of the target obstacle; and providing a risk warning when it is determined that the target obstacle poses a risk to the vehicle.
[0005] According to an embodiment of the present disclosure, determining a target obstacle associated with abnormal audio data based on external audio data includes: determining relative position information between a sound source object generating the abnormal audio data and a vehicle based on the external audio data; and determining the target obstacle based on the relative position information and external visual data; wherein the external visual data includes at least one of image data and point cloud data.
[0006] According to an embodiment of the present disclosure, determining whether the target obstacle poses a risk to the vehicle based on the first trajectory of the vehicle and the second trajectory of the target obstacle includes: determining that the target obstacle poses a risk to the vehicle in response to detecting that the first trajectory and the second trajectory intersect.
[0007] According to an embodiment of the present disclosure, it also includes: determining a first trajectory based on at least one of a navigation driving path of the vehicle, attribute information of the road on which the vehicle is located, and a driving state of an obstacle in front of the vehicle.
[0008] According to an embodiment of the present disclosure, when it is determined that a target obstacle poses a risk to a vehicle, issuing a risk warning includes: determining an initial risk level of the target obstacle based on driving information of the target obstacle; and issuing a risk warning according to a prompt method corresponding to the initial risk level.
[0009] According to an embodiment of the present disclosure, it also includes: after making a risk prompt according to a prompt method corresponding to the initial risk level, acquiring image data of objects inside the vehicle; determining, according to the image data of the object, whether the object generates a predetermined feedback behavior for the risk prompt within a predetermined time period after the risk prompt; and in the case of determining that the object has not generated the predetermined feedback behavior, making a risk prompt based on a prompt method corresponding to the target risk level; wherein the target risk level is higher than the initial risk level.
[0010] According to an embodiment of the present disclosure, determining whether the object generates a predetermined feedback behavior for the risk warning within a predetermined time period after a risk warning is issued based on image data for the object includes: determining a target mirror in a vehicle based on relative position information; wherein the target mirror includes at least one of a rearview mirror, a left mirror, and a right mirror; and determining that the object generates a predetermined feedback behavior in response to detecting that the object looks at the target mirror within a predetermined time period after a risk warning is issued based on image data for the object.
[0011] According to an embodiment of the present disclosure, providing risk warning includes at least one of the following: displaying warning information; controlling the vibration of the steering wheel of the vehicle; controlling the tightening of the seat belt in the cab of the vehicle; providing sound prompts; and controlling the vehicle to start an assisted driving mode.
[0012] According to an embodiment of the present disclosure, making a risk warning includes: in response to determining that the prompt method includes a sound prompt, determining the volume of internal audio data inside the vehicle; when it is determined that the volume of the internal audio data is greater than or equal to a volume threshold, performing a predetermined operation; the predetermined operation includes at least one of the following: lowering the multimedia audio volume inside the vehicle, generating a prompt message for lowering the volume of an object inside the vehicle; and making a sound prompt in response to performing the predetermined operation.
[0013] According to an embodiment of the present disclosure, making a risk warning includes: in response to determining that the prompt method includes a sound prompt, determining the volume of internal audio data inside the vehicle; and making a sound prompt based on a target volume, the target volume being greater than the volume of the internal audio data.
[0014] Another aspect of the present disclosure provides a risk warning device for a vehicle, comprising: an acquisition module, a target obstacle determination module, a risk determination module, and a prompt module. The acquisition module is used to acquire external audio data from outside the vehicle; the target obstacle determination module is used to determine the target obstacle related to the abnormal audio data according to the external audio data when it is determined that the external audio data includes abnormal audio data; the risk determination module is used to determine whether the target obstacle generates a risk to the vehicle according to the first trajectory of the vehicle and the second trajectory of the target obstacle; and the prompt module is used to provide a risk warning when it is determined that the target obstacle generates a risk to the vehicle.
[0015] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.
[0016] Another aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the above method.
[0017] Another aspect of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0018] Another aspect of the present disclosure provides a vehicle including the above-mentioned electronic device. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0020] Figure 1 A diagram schematically shows an application scenario of a vehicle risk warning method, device, equipment, medium, and program product according to an embodiment of the present disclosure;
[0021] Figure 2 A flow chart of a vehicle risk warning method according to an embodiment of the present disclosure is schematically shown;
[0022] Figure 3 A flowchart of a method for providing risk warning according to an embodiment of the present disclosure is schematically shown;
[0023] Figure 4 A structural block diagram of a vehicle risk warning device according to an embodiment of the present disclosure is schematically shown; and
[0024] Figure 5A block diagram of an electronic device suitable for implementing a vehicle risk warning method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.
[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0029] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.
[0030] Figure 1 The application scenario diagram of the vehicle risk warning method, device, equipment, medium and program product according to the embodiment of the present disclosure is schematically shown.
[0031] It should be noted that Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0032] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a server 110 , a network 120 , a vehicle 130 , and a sensor 140 .
[0033] The server 110 may be arranged at a remote end capable of establishing communication with the vehicle-mounted terminal, and may be specifically implemented as a distributed server cluster consisting of multiple servers, or may be implemented as a single server.
[0034] The server 110 may be a server that provides various services. For example, map applications, data processing applications, etc. may be installed on the server 110. Take the example of the server 110 running the data processing application: external audio data, obstacle speed and other data transmitted from the sensor 140 are received through the network 120, and these data are used as data to be processed. Then, the data to be processed is processed to determine whether the obstacles around the vehicle 130 pose a risk to the vehicle 130, and a risk warning is issued if a risk occurs.
[0035] The network 120 is used to provide a medium for a communication link between the sensor 140 and the server 110. The network 120 may include various connection types, such as wired and / or wireless communication links, and the like.
[0036] The vehicle 130 may be an electric vehicle, a fuel vehicle, a tricycle or the like.
[0037] The sensor 140 may interact with the server 110 through the network 120 to receive or send messages, etc. The sensor 140 may be a functional element integrated on the vehicle 130, such as an audio acquisition device, a camera, a laser radar, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, an inertial measurement unit, etc. The sensor 140 may be used to collect state data of perceived objects (such as pedestrians, obstacles such as vehicles, etc.) around the vehicle 130 and surrounding road data.
[0038] It should be noted that the vehicle risk warning method provided in the embodiment of the present disclosure can be executed by the server 110, and accordingly, the vehicle risk warning device provided in the embodiment of the present disclosure can also be set in the server 110. In some embodiments, the vehicle risk warning method can also be executed by the vehicle 130, and accordingly, the vehicle risk warning device provided in the embodiment of the present disclosure can also be set in the vehicle 130.
[0039] Understandably, Figure 1 The number of sensors, networks and servers in the embodiment is only for illustration. Any number of sensors, networks and servers may be provided according to the implementation requirements.
[0040] Figure 2 A flow chart of a vehicle risk warning method according to an embodiment of the present disclosure is schematically shown.
[0041] like Figure 2 As shown, the vehicle risk warning method 200 of this embodiment includes operations S210 to S250.
[0042] In operation S210, external audio data from outside the vehicle is acquired.
[0043] For example, multiple audio collection devices may be installed on the vehicle body, and external audio data may be collected through the audio collection devices, which may be microphones.
[0044] In operation S220, in a case where it is determined that the external audio data includes abnormal audio data, a target obstacle associated with the abnormal audio data is determined based on the external audio data.
[0045] For example, abnormal audio data may include horn sounds, collision sounds, sirens of special vehicles, etc. Special vehicles may include police cars, ambulances, fire trucks, etc.
[0046] For example, the target obstacle may be other vehicles, pedestrians, and other objects around the vehicle.
[0047] For example, the correlation between the target obstacle and the abnormal audio data may indicate that the abnormal audio data is generated by the target obstacle.
[0048] For example, the position and actual distance of the sound source object that generates abnormal audio data may be determined, and an object located at the position and at the actual distance from the vehicle may be regarded as a target obstacle.
[0049] In operation S230, it is determined whether the target obstacle poses a risk to the vehicle based on the first trajectory of the vehicle and the second trajectory of the target obstacle. If so, operation S240 may be performed, and if not, operation S250 may be performed, for example, the processing flow is terminated.
[0050] For example, the first trajectory may represent the trajectory of the vehicle to be driven. The second trajectory may represent the predicted trajectory of the target obstacle to be driven. The second trajectory may be determined based on the driving information of the target obstacle in the past period of time. This embodiment does not limit the method of determining the second trajectory.
[0051] When the first trajectory intersects with the second trajectory or the distance between them is close, it can be considered that there is a risk of collision between the target obstacle and the vehicle, and then it can be determined that the target obstacle will pose a risk to the vehicle.
[0052] In operation S240, a risk warning is provided.
[0053] For example, a sound prompt may be provided, such as generating a predetermined prompt sound, or generating a prompt corresponding to the intention of the target obstacle, the prompt including, for example, "the vehicle on the left is overtaking".
[0054] For example, the vehicle can be controlled to start the assisted driving mode. In the assisted driving mode, the vehicle can limit the steering wheel turning angle, assist the driver to turn the steering wheel in a specific direction, limit the speed, reduce the speed, etc.
[0055] For example, the prompt information can be displayed through the vehicle display screen, the prompt light can be controlled to flash, the steering wheel of the vehicle can be controlled to vibrate, and the seat belt in the cab of the vehicle can be controlled to tighten.
[0056] It is understandable that risk warnings may be provided in a variety of ways, such as providing sound warnings and controlling steering wheel vibration at the same time.
[0057] The technical solution provided by the embodiment of the present disclosure first analyzes whether the external audio data includes abnormal audio data, and if so, determines the target obstacle that generates the abnormal audio data, and determines whether there is a risk based on the trajectory of the target obstacle and the vehicle, and then gives a risk warning if there is a risk. It can be seen that the embodiment of the present disclosure can analyze whether there is a risk based on the external audio data and give a warning, which improves driving safety compared to the method of relying on the user to observe the surrounding environment for driving.
[0058] According to another embodiment of the present disclosure, the above-mentioned process of determining the target obstacle related to the abnormal audio data based on the external audio data may include: determining the relative position information of the sound source object generating the abnormal audio data and the vehicle based on the external audio data, and then determining the target obstacle based on the relative position information and the external visual data.
[0059] For example, the relative position information may include the orientation of the sound source object relative to the vehicle, and may also include the distance between the sound source object and the vehicle. The position of the sound source object may be located around the vehicle based on the relative position information.
[0060] For example, the external visual data includes image data collected by a camera, and may also include point cloud data collected by a laser radar. According to the location of the sound source object and the external visual data, the target obstacle can be determined from other obstacles around the vehicle.
[0061] The disclosed embodiment first determines the relative position between the sound source object and the vehicle according to the external audio data, and then determines the target obstacle from the obstacles around the vehicle based on the relative position, thereby accurately determining the target obstacle.
[0062] According to another embodiment of the present disclosure, the process of determining whether a target obstacle poses a risk to a vehicle is described below.
[0063] In one example, a first trajectory of a vehicle and a second trajectory of a target obstacle may be determined first. If the first trajectory intersects with the second trajectory, it means that the vehicle and the target obstacle may collide at the intersection position. Therefore, it can be determined that the target obstacle will pose a risk to the vehicle when the trajectories intersect.
[0064] The first trajectory may be determined in the following manner.
[0065] For example, the first trajectory is determined according to the navigation driving path of the vehicle, for example, the navigation driving path is determined as the first trajectory.
[0066] For another example, the first trajectory may be determined based on the attribute information of the road where the vehicle is located. The attribute information may include the path of the road where the vehicle is located, and the path may be determined as the first trajectory. The attribute information may also include lane change information, and the lane change information may include lane narrowing due to accidents, construction, etc., and lane widening due to an increase in lane lines, etc. The road where the vehicle is located and the changed road may be determined as the first trajectory.
[0067] For another example, the first trajectory may be determined according to the driving state of the obstacle in front of the vehicle. For example, if the obstacle in front of the vehicle is driving slowly, the vehicle may change lanes, and the road after the vehicle changes lanes may be determined as the first path.
[0068] The above describes the method for determining the first trajectory, through which the first trajectory can be accurately determined according to the road conditions and the vehicle's driving conditions, and then accurately assess whether there is a risk between the vehicle and the target obstacle. It is understandable that multiple first trajectories can be determined through the above multiple methods, and when the second trajectory intersects any of the first trajectories, it is determined that the target obstacle will cause a risk to the vehicle.
[0069] In another example, it can be determined that a risk is generated when there are no other obstacles between the vehicle and the target obstacle. It is understandable that if there are other obstacles between the vehicle and the target obstacle, the target obstacle will generate a risk for the other obstacles, and the risk of the target obstacle to the vehicle will be reduced accordingly. If there are no other obstacles between the vehicle and the target obstacle, the target obstacle will directly affect the vehicle, so there is a risk.
[0070] Figure 3 The flowchart of the method for providing risk warning according to an embodiment of the present disclosure is schematically shown.
[0071] like Figure 3 As shown, the above method 340 for providing risk warning when it is determined that the target obstacle poses a risk to the vehicle may include operations S341 to S346.
[0072] In operation S341 , an initial risk level of the target obstacle is determined according to driving information of the target obstacle.
[0073] For example, driving information may include the speed of the target obstacle and the distance between the target obstacle and the vehicle. Multiple speed thresholds and multiple distance thresholds may be pre-configured, and the risk level may be determined based on the relationship between the speed and the speed threshold and the relationship between the distance and the distance threshold.
[0074] For example, the first risk level is that the vehicle speed is greater than the first speed threshold and the distance is less than the first distance threshold, the second risk level is that the vehicle speed is greater than the first speed threshold and the distance is greater than the second distance threshold and the distance is less than the first distance threshold, or the second risk level is that the vehicle speed is less than the first speed threshold and the vehicle speed is greater than the second speed threshold and the distance is greater than the first distance threshold, and the third risk level is that the vehicle speed is less than the second speed threshold and the distance is greater than the second distance threshold. It can be seen that the risk levels of the above-mentioned first risk level, second risk level, and third risk level decrease in sequence. It can be understood that the above description takes three risk levels as an example, and this embodiment does not limit the number and division method of risk levels.
[0075] In operation S342, a risk prompt is performed according to a prompt method corresponding to the initial risk level.
[0076] For example, different prompt methods correspond to different risk levels. For example, the prompt method corresponding to the first risk level may be controlling the vehicle assisted driving, the prompt method corresponding to the second risk level may be a sound prompt, the prompt method corresponding to the third risk level may be controlling the steering wheel vibration or controlling the seat belt tightening, and the prompt method corresponding to the fourth risk level may be controlling the indicator light to flash or displaying prompt information through a display screen.
[0077] The corresponding prompt method is selected according to the initial risk level for risk warning. The prompt method can be flexibly selected according to the degree of urgency, thereby effectively prompting the user.
[0078] In operation S343, image data for objects inside the vehicle is acquired.
[0079] For example, a camera may be used to collect image data of an object inside a vehicle, and the object may be a driver or other passengers inside the vehicle.
[0080] In operation S344, it is determined whether the object generates a predetermined feedback behavior for the risk prompt within a predetermined time after the risk prompt is made according to the image data of the object. If not, the process proceeds to operation S345, and if so, the process proceeds to operation S346.
[0081] In one example, it can be detected that the subject checks at least one of the rearview mirror, the left mirror, and the right mirror within a predetermined period of time after a risk warning is given, and in the case of checking, it is determined that the subject generates a predetermined feedback behavior.
[0082] In another example, a target mirror in a vehicle may be determined based on the relative position information, the target mirror including at least one of a rearview mirror, a left mirror, and a right mirror. Then, based on the image data for the object, in response to detecting that the object looks at the target mirror within a predetermined time after the risk prompt is given, it is determined that the object generates a predetermined feedback behavior.
[0083] For example, if the relative position information indicates that the target obstacle is located on the left side of the vehicle, the target mirror includes the left mirror. It can be detected whether the subject looks at the left mirror. If so, it is determined that the subject has generated a predetermined feedback behavior; if not, it is determined that the subject has not generated a predetermined feedback behavior. By detecting whether the subject looks at the target mirror corresponding to the relative position information, it can be accurately determined whether the user has made feedback for the last risk prompt.
[0084] In operation S345, if the object does not generate a predetermined feedback behavior, a risk prompt may be performed based on a prompt method corresponding to a target risk level, where the target risk level is higher than the initial risk level.
[0085] After executing operation S345, the process may return to operation S343 until the risk warning level reaches the highest risk level.
[0086] In operation S346 , the object generates a predetermined feedback behavior, and the process may end.
[0087] After the first risk warning is given, this embodiment also determines whether the subject provides feedback based on the risk warning. The user may not notice the risk warning information due to fatigue driving, etc., and thus does not generate the predetermined feedback behavior. This embodiment can use a warning method corresponding to a higher risk level to provide a second warning, thereby improving driving safety.
[0088] According to another embodiment of the present disclosure, the risk warning includes a sound warning. In this case, the sound warning may not be clearly heard by the object in the vehicle due to the high volume of multimedia playback inside the vehicle or the high volume of conversation of the object inside the vehicle. In this case, the sound volume inside the vehicle may be lowered before the sound warning is given, or the sound warning may be given at a high volume, so as to ensure that the object inside the vehicle can hear the sound warning clearly and ensure the warning effect.
[0089] In one example, the volume of the internal audio data inside the vehicle can be determined first, for example, the average volume inside the vehicle within a predetermined time period before the abnormal audio data is collected is used as the volume of the internal audio data. Then, it is determined whether the volume of the internal audio data is greater than or equal to a volume threshold. If the volume is less than the volume threshold, a prompt can be given directly. If the volume is greater than or equal to the volume threshold, the multimedia audio volume inside the vehicle can be reduced, or a prompt message can be generated, and the prompt message can be used to prompt the objects inside the vehicle to lower the volume of their conversations. After multimedia noise reduction is performed and prompt information is generated, risk prompts can be given in the form of sound prompts.
[0090] In one example, the volume of the internal audio data inside the vehicle may be determined first, and the determination method may refer to the above. Next, a sound prompt may be given based on a target volume, and the target volume is greater than the volume of the internal audio data.
[0091] Based on the above-mentioned vehicle risk warning method, the present disclosure also provides a vehicle risk warning device. Figure 4 The device is described in detail.
[0092] Figure 4 The structural block diagram of the vehicle risk warning device according to an embodiment of the present disclosure is schematically shown.
[0093] like Figure 4 As shown, the vehicle risk warning device 400 of this embodiment includes an acquisition module 410, a target obstacle determination module 420, a risk determination module 430 and a warning module 440.
[0094] The acquisition module 410 is used to acquire external audio data from outside the vehicle.
[0095] The target obstacle determination module 420 is configured to determine a target obstacle related to the abnormal audio data according to the external audio data when it is determined that the external audio data includes abnormal audio data.
[0096] The risk determination module 430 is used to determine whether the target obstacle creates a risk to the vehicle based on the first trajectory of the vehicle and the second trajectory of the target obstacle.
[0097] The prompt module 440 is used to issue a risk prompt when it is determined that the target obstacle poses a risk to the vehicle.
[0098] According to another embodiment of the present disclosure, the target obstacle determination module includes: a position determination submodule and an obstacle determination submodule. The position determination submodule is used to determine the relative position information between the sound source object generating the abnormal audio data and the vehicle according to the external audio data. The obstacle determination submodule is used to determine the target obstacle according to the relative position information and the external visual data, wherein the external visual data includes at least one of image data and point cloud data.
[0099] According to another embodiment of the present disclosure, the risk determination module includes: a risk determination submodule for determining the risk that the target obstacle poses to the vehicle in response to detecting the intersection of the first trajectory and the second trajectory.
[0100] According to another embodiment of the present disclosure, the above-mentioned device also includes: a first trajectory determination module, which is used to determine the first trajectory based on at least one of the navigation driving path of the vehicle, the attribute information of the road on which the vehicle is located, and the driving status of an obstacle in front of the vehicle.
[0101] According to another embodiment of the present disclosure, the prompt module includes: an initial level determination submodule and a first prompt submodule. The initial level determination submodule is used to determine the initial risk level of the target obstacle according to the driving information of the target obstacle. The first prompt submodule is used to perform risk prompts according to the prompt method corresponding to the initial risk level.
[0102] According to another embodiment of the present disclosure, the prompt module also includes: an image acquisition submodule, a behavior determination submodule and a second prompt submodule. The image acquisition submodule is used to acquire image data for objects inside the vehicle after performing a risk prompt according to a prompt method corresponding to the initial risk level. The behavior determination submodule is used to determine, based on the image data for the object, whether the object generates a predetermined feedback behavior for the risk prompt within a predetermined time period after the risk prompt is performed. The second prompt submodule is used to perform a risk prompt based on a prompt method corresponding to a target risk level when it is determined that the object has not generated a predetermined feedback behavior, wherein the target risk level is higher than the initial risk level.
[0103] According to another embodiment of the present disclosure, the behavior determination submodule includes: a mirror determination unit and a behavior determination unit. The mirror determination unit is used to determine a target mirror in a vehicle according to relative position information; wherein the target mirror includes at least one of a rearview mirror, a left mirror, and a right mirror. The behavior determination unit is used to determine that the object generates a predetermined feedback behavior in response to detecting that the object looks at the target mirror within a predetermined time after the risk prompt is issued, based on image data for the object.
[0104] According to another embodiment of the present disclosure, the prompt module includes at least one of a display submodule, a first control submodule, a second control submodule, a sound prompt submodule, and a third control submodule, wherein the display submodule is used to display prompt information. The first control submodule is used to control the vibration of the steering wheel of the vehicle. The second control submodule is used to control the tightening of the seat belt in the cab of the vehicle. The sound prompt submodule is used to make a sound prompt. The third control submodule is used to control the vehicle to start the assisted driving mode.
[0105] According to another embodiment of the present disclosure, the prompt module includes: a volume determination submodule, an execution submodule and a third prompt submodule. The volume determination submodule is used to determine the volume of the internal audio data inside the vehicle in response to determining that the prompt method includes a sound prompt. The execution submodule is used to perform a predetermined operation when it is determined that the volume of the internal audio data is greater than or equal to a volume threshold; the predetermined operation includes at least one of the following: reducing the multimedia audio volume inside the vehicle, and generating prompt information for reducing the volume of an object inside the vehicle. The third prompt submodule is used to perform a sound prompt in response to executing the predetermined operation.
[0106] According to another embodiment of the present disclosure, the prompt module includes: a volume determination submodule and a fourth prompt submodule. The volume determination submodule is used to determine the volume of the internal audio data inside the vehicle in response to determining that the prompt mode includes a sound prompt. The fourth prompt submodule is used to perform a sound prompt based on a target volume, and the target volume is greater than the volume of the internal audio data.
[0107] According to an embodiment of the present disclosure, any multiple modules of the acquisition module 410, the target obstacle determination module 420, the risk determination module 430, and the prompt module 440 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 410, the target obstacle determination module 420, the risk determination module 430, and the prompt module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 410, the target obstacle determination module 420, the risk determination module 430, and the prompt module 440 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.
[0108] Figure 5 A block diagram of an electronic device suitable for implementing a vehicle risk warning method according to an embodiment of the present disclosure is schematically shown.
[0109] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0110] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.
[0111] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.
[0112] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0113] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.
[0114] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the item recommendation method provided by the embodiment of the present disclosure.
[0115] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0116] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0117] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.
[0118] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0119] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0120] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0121] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A method for risk warning of a vehicle, comprising: acquiring external audio data from outside the vehicle; In a case where it is determined that the external audio data includes abnormal audio data, determining a target obstacle related to the abnormal audio data based on the external audio data; Determining whether the target obstacle poses a risk to the vehicle based on the first trajectory of the vehicle and the second trajectory of the target obstacle; as well as When it is determined that the target obstacle poses a risk to the vehicle, a risk warning is issued.
2. The method according to claim 1, wherein: Determining, according to the external audio data, a target obstacle associated with the abnormal audio data comprises: Determining relative position information between a sound source object generating the abnormal audio data and the vehicle according to the external audio data; and Determining the target obstacle according to the relative position information and external visual data outside the vehicle; Wherein, the external visual data includes at least one of image data and point cloud data.
3. The method according to claim 1, wherein: The determining, based on the first trajectory of the vehicle and the second trajectory of the target obstacle, whether the target obstacle generates a risk to the vehicle comprises: In response to detecting that the first trajectory and the second trajectory intersect, it is determined that the target obstacle presents a risk to the vehicle.
4. The method according to claim 1, further comprising: The first trajectory is determined based on at least one of a navigation driving path of the vehicle, attribute information of a road on which the vehicle is located, and a driving state of an obstacle in front of the vehicle.
5. The method according to any one of claims 1 to 4, wherein: When it is determined that the target obstacle poses a risk to the vehicle, providing a risk prompt includes: determining an initial risk level of the target obstacle based on the driving information of the target obstacle; and A risk warning is provided according to a warning method corresponding to the initial risk level.
6. The method according to claim 5, further comprising: After the risk warning is given according to the warning method corresponding to the initial risk level, Acquiring image data of an object inside the vehicle; determining, based on the image data of the object, whether the object generates a predetermined feedback behavior in response to the risk prompt within a predetermined time period after the risk prompt is given; as well as When it is determined that the subject has not generated the predetermined feedback behavior, providing a risk prompt based on a prompt method corresponding to the target risk level; Wherein, the target risk level is higher than the initial risk level.
7. The method according to claim 1, wherein: Risk warnings include at least one of the following: Display prompt information; controlling the vibration of the steering wheel of the vehicle; Controlling the tightening of the seat belt in the cab of the vehicle; Provide sound prompts; as well as Control the vehicle to start the assisted driving mode.
8. The method according to claim 1, wherein: Risk warnings include: In response to determining that the prompting mode includes an audio prompt, determining a volume of internal audio data within the vehicle; If it is determined that the volume of the internal audio data is greater than or equal to the volume threshold, a predetermined operation is performed; the predetermined operation includes at least one of the following: reducing the volume of the multimedia audio inside the vehicle, generating prompt information for causing an object inside the vehicle to reduce the volume; and In response to executing the predetermined operation, a sound prompt is given.
9. A vehicle risk warning device, comprising: An acquisition module, used for acquiring external audio data from outside the vehicle; a target obstacle determination module, configured to determine a target obstacle related to the abnormal audio data according to the external audio data when it is determined that the external audio data includes abnormal audio data; a risk determination module, configured to determine whether the target obstacle generates a risk to the vehicle based on the first trajectory of the vehicle and the second trajectory of the target obstacle; as well as The prompt module is used to provide a risk prompt when it is determined that the target obstacle poses a risk to the vehicle.
10. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 8.
11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 8.
12. A vehicle comprising the electronic device according to claim 10.