A vehicle early warning method, device, equipment and storage medium

Abstract

A vehicle early warning method, device, equipment and storage medium, the method comprises: acquiring real-time monitoring video of a vehicle-mounted camera on a current vehicle (S101); performing early warning scene identification on the current vehicle based on the real-time monitoring video (S103); in the case where the current vehicle is identified to have an early warning scene, determining an early warning target (S105); processing the real-time monitoring video to obtain perception information of the early warning target (S107); based on the perception information, marking the early warning target in the real-time monitoring video to obtain a real-time early warning video (S109); and outputting the real-time early warning video to a vehicle-end display device of the current vehicle (S111). It can intuitively and clearly warn the driver and passengers in the vehicle of danger in the form of an early warning video, help the driver and passengers quickly identify the early warning target in the environment around the vehicle, thereby avoiding dangerous scenes, and effectively improve the accuracy and speed of early warning.

Description

Technical Field

[0001] This application relates to the fields of assisted driving technology and autonomous driving technology, specifically to a vehicle warning method, device, equipment and storage medium. Background Technology

[0002] Currently, deep learning-based vehicle environment perception solutions are being widely applied in intelligent driving vehicles. Among these, the most common applications are intelligent driving solutions that use visual sensors to perceive the vehicle's environmental situation. A robust sensor layout, powerful computing chips, and a vast array of algorithms are fundamental conditions for creating a safe driving environment for drivers and passengers. Vehicle warning solutions that use cameras as visual sensors to perceive the vehicle's environmental situation can provide timely warnings to drivers and passengers when the surrounding environment changes or when danger may be imminent.

[0003] However, current intelligent driving vehicle warning solutions present numerous warning scenarios but offer relatively limited warning methods, typically consisting of sound or vibration warnings. This makes it difficult for drivers and passengers to intuitively understand the type or specific location of the warning target. On one hand, it requires drivers to distract themselves to identify the warning target, interfering with their normal driving and potentially leading to accidents and reducing the effectiveness of the warning. On the other hand, it can cause drivers to become bored and distrustful, leading them to disable the warning function. Therefore, more effective and practical technical solutions are needed. Summary of the Invention

[0004] To address the problems of existing technologies, this application provides a vehicle early warning method, device, equipment, and storage medium. The technical solution is as follows:

[0005] On the one hand, a vehicle warning method is provided, the method comprising:

[0006] Obtain real-time monitoring video from the vehicle's onboard camera;

[0007] Based on the real-time monitoring video, the current vehicle is identified for a warning scenario;

[0008] If a warning scenario is detected for the current vehicle, the warning target is determined;

[0009] The real-time monitoring video is processed to obtain the perception information of the early warning target;

[0010] Based on the perceived information, the warning target is marked in the real-time monitoring video to obtain the real-time warning video;

[0011] The real-time warning video is output to the vehicle-mounted display device of the current vehicle.

[0012] On the other hand, a vehicle warning device is provided, the device comprising:

[0013] The real-time monitoring video acquisition module is used to acquire real-time monitoring video from the vehicle's onboard camera.

[0014] The early warning scene recognition module is used to recognize the early warning scene of the current vehicle based on the real-time monitoring video;

[0015] The warning target determination module is used to determine the warning target when the current vehicle is identified as having a warning scenario;

[0016] The perception information module is used to process the real-time monitoring video to obtain the perception information of the early warning target;

[0017] A real-time early warning video generation module is used to mark the early warning target in the real-time monitoring video based on the perceived information, so as to obtain a real-time early warning video.

[0018] The real-time warning video output module is used to output the real-time warning video to the vehicle-side display device of the current vehicle.

[0019] On the other hand, a vehicle warning device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the vehicle warning method as described above.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the vehicle warning method as described above.

[0021] The vehicle early warning method, device, equipment, and storage medium provided in this application have the following technical effects:

[0022] The technical solution provided in this application is based on real-time monitoring video of the vehicle's environmental situation acquired by a vehicle-mounted camera; the warning scene is identified and the warning target is confirmed based on the real-time monitoring video; the real-time monitoring video is processed to obtain the perception information of the warning target; the warning target is marked in the real-time monitoring video based on the perception information to generate a real-time warning video, which is output to the display device in the vehicle, providing intuitive and clear danger warnings to the driver and passengers, helping them to quickly identify the warning target in the surrounding environment of the vehicle, thereby avoiding dangerous scenarios and improving the accuracy and speed of the warning. Attached Figure Description

[0023] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic flowchart of a vehicle warning method provided in an embodiment of this application;

[0025] Figure 2 This is a flowchart illustrating a method for processing real-time monitoring video provided in an embodiment of this application;

[0026] Figure 3 This is a flowchart illustrating a method for locating and marking the coordinates of a warning target according to an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of a process for marking early warning targets based on annotation style information, provided in an embodiment of this application;

[0028] Figure 5 This is a flowchart illustrating a method for determining the warning range, provided in an embodiment of this application.

[0029] Figure 6 This is a schematic diagram of a process for determining a target display device provided in an embodiment of this application;

[0030] Figure 7 This is a schematic diagram of a process for generating warning audio provided in an embodiment of this application;

[0031] Figure 8 This is a schematic diagram of a vehicle warning device provided in an embodiment of this application;

[0032] Figure 9 This is a schematic diagram of a sensing information module provided in an embodiment of this application;

[0033] Figure 10 This is a hardware structure block diagram of the vehicle-side server for a vehicle early warning method provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged in real time where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0036] The following describes a vehicle warning method provided by an embodiment of this application. Figure 1 This is a flowchart illustrating a vehicle warning method provided in an embodiment of this application. It should be noted that this specification provides the operational steps of the method as described in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. In actual system or product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 1 As shown, the above method may include:

[0037] S101, acquire the real-time monitoring video from the onboard camera of the current vehicle.

[0038] Specifically, the aforementioned vehicle-mounted cameras may include multiple vehicle-mounted cameras that monitor the current vehicle from all directions in real time. Through these multiple vehicle-mounted cameras, the environmental situation around the current vehicle can be perceived, providing a safer driving environment for the driver.

[0039] S103, based on the aforementioned real-time monitoring video, identify the warning scenario for the current vehicle.

[0040] Specifically, based on the real-time monitoring video from the aforementioned multiple vehicle cameras, the surrounding environment of the current vehicle is identified as a warning scenario. If there are warning targets in the environment that affect the safety of the target vehicle, the environment in which the target vehicle is located is identified as a warning scenario. Warning scenarios may include, but are not limited to: lane change warning, reversing warning, and door opening warning.

[0041] S105, upon recognizing the aforementioned warning scenario for the current vehicle, determines the warning target.

[0042] In one specific embodiment, determining the warning target may include:

[0043] 1) Perform target analysis on real-time monitoring video to obtain targets in the real-time monitoring video images;

[0044] In practical applications, the above targets can include dynamic targets and static targets. Dynamic targets can include people or objects in motion in the environment where the target vehicle is located. Specifically, dynamic targets can include, but are not limited to, vehicles in motion and pedestrians. Static targets can include people or objects in stationary state in the environment where the target vehicle is located. Specifically, static targets can include, but are not limited to, stationary vehicles, roads, and trees.

[0045] 2) Based on the preset early warning conditions, determine the early warning targets from the above targets.

[0046] In practical applications, the above-mentioned preset early warning conditions can be obtained by summarizing and generalizing based on the type information of the early warning scenario and a large number of sample early warning scenarios.

[0047] S107, The real-time monitoring video is processed to obtain the perception information of the warning target.

[0048] In some embodiments, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for processing real-time monitoring video provided in an embodiment of this specification. Specifically, it may include:

[0049] S201, Target detection is performed on the aforementioned real-time monitoring video to determine the type information of the aforementioned warning targets.

[0050] In one specific embodiment, determining the type information of the aforementioned early warning target may include:

[0051] 1) Perform target detection on the above real-time monitoring video to obtain the feature information of the warning target. The feature information may include, but is not limited to: height information, width information, and pixel information;

[0052] 2) Based on the above feature information, the type of the warning target is identified to determine the type information of the warning target. The type information of the warning target can characterize the classification category of the warning target. The type information of the warning target may include, but is not limited to: pedestrians, vehicles, roads, and trees.

[0053] S203, perform target tracking on the aforementioned real-time monitoring video to determine multiple frames of images including the aforementioned warning target.

[0054] Specifically, multiple frames containing the aforementioned warning targets are acquired from the images in the aforementioned real-time monitoring video. These multiple frames may include consecutive frames.

[0055] S205, based on the above-mentioned multi-frame images, perform motion trajectory analysis on the above-mentioned warning target to determine the real-time relative position information, real-time speed information and real-time heading angle information of the above-mentioned warning target relative to the above-mentioned current vehicle.

[0056] Specifically, the motion trajectory of the warning target is extracted from the above multi-frame images, and the motion trajectory is analyzed to obtain the real-time relative position information, real-time speed information and real-time heading angle information of the current vehicle. The relative position information may include, but is not limited to, distance information.

[0057] S207, the aforementioned target type information, the aforementioned real-time relative position information, the aforementioned real-time speed information, and the aforementioned real-time heading angle information are used as the aforementioned sensing information.

[0058] Using the technical solutions of the embodiments described above, perception information of the warning target, including type information, real-time relative position information, real-time speed information, and real-time heading angle information, can be obtained.

[0059] S109, Based on the above-mentioned perception information, the above-mentioned early warning targets are marked in the above-mentioned real-time monitoring video to obtain real-time early warning video.

[0060] In some embodiments, such as Figure 3 As shown, after processing the real-time monitoring video to obtain the perception information of the warning target, the method may further include:

[0061] S209, construct the target coordinate system corresponding to the above-mentioned multiple frames of images, with the current vehicle as the origin of the coordinate system.

[0062] Specifically, a corresponding target coordinate system is constructed on each of the above multi-frame images. The origin of the target coordinate system is the current vehicle. The target coordinate system may include, but is not limited to, a two-dimensional coordinate system including the X-axis and the Y-axis.

[0063] Accordingly, based on the aforementioned perceived information, marking the aforementioned warning targets in the aforementioned real-time monitoring video to obtain the real-time warning video may include:

[0064] S211, Based on the above real-time relative position information, determine the coordinate information of the warning target in the target coordinate system corresponding to the above multi-frame images.

[0065] Specifically, based on the real-time relative position information of the warning target with respect to the current vehicle, the coordinate information of the warning target in the target coordinate system with the current vehicle as the origin is determined, so as to facilitate the positioning of the warning target in multiple frames of images.

[0066] S213, Based on the type information and coordinate information of the warning target, the warning target is marked in the multi-frame image to obtain the real-time warning video.

[0067] In an optional embodiment, such as Figure 4 As shown, the process of marking warning targets in multiple frames of images based on the type information and coordinate information of the warning targets to obtain the real-time warning video may include:

[0068] S401, Based on the type information of the aforementioned warning targets, determine the labeling style information corresponding to the aforementioned warning targets.

[0069] Specifically, for each type of information, corresponding annotation style information is preset. The annotation style information distinguishes the warning targets of different types of information. The annotation style information may include, but is not limited to: color, outline format, and fill format.

[0070] S403, based on the above annotation style information and the above coordinate information, mark the warning target in the above multi-frame images to obtain the marked multi-frame images.

[0071] Specifically, based on the annotation style information and coordinate information, the warning target is marked in multiple frames of images to achieve the technical effect of distinguishing the warning target from other environments in the multiple frames of images, so as to intuitively show the type information of the warning target and its position information relative to the current vehicle to the driver and passengers.

[0072] S405, Based on the marked multi-frame images, generate the aforementioned real-time warning video.

[0073] In a specific embodiment, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating a method for determining the warning range, as provided in an embodiment of this specification. Specifically, it may include:

[0074] S501, Based on the aforementioned real-time relative position information, real-time speed information, and real-time heading angle information, generate the predicted motion trajectory of the aforementioned warning target.

[0075] Specifically, based on the preset motion trajectory generation algorithm, a predicted motion trajectory is generated using real-time relative position information, real-time velocity information, and real-time heading angle information. The aforementioned motion trajectory generation algorithm can be obtained by summarizing and generalizing multiple sample real-time position information, sample real-time velocity information, sample real-time heading angle information, and corresponding sample motion trajectories.

[0076] S503, based on the predicted motion trajectory, determine whether the warning target is within the warning range of the current vehicle.

[0077] Specifically, the system can obtain real-time driving information of the current vehicle and generate a predicted trajectory of the current vehicle based on the real-time driving information. When the predicted trajectory of the warning target intersects with the predicted trajectory of the current vehicle, it is determined that the warning target is within the warning range of the current vehicle.

[0078] S505, when the determination is yes, execute the above steps of marking the warning target in the real-time monitoring video based on the above-mentioned perception information to obtain the real-time warning video.

[0079] Specifically, when the warning target is within the warning range of the current vehicle, the warning target is marked in the real-time monitoring video based on the perception information to obtain the real-time warning video.

[0080] S111, output the aforementioned real-time warning video to the vehicle-mounted display device of the current vehicle.

[0081] In a specific embodiment, such as Figure 6 As shown, Figure 6 This is a schematic flowchart illustrating the process of determining a target display device according to an embodiment of this specification. Specifically, it may include:

[0082] S601, upon recognizing that the current vehicle is in a warning scenario, determines the type information of the warning scenario.

[0083] Specifically, the system can identify preset landmarks in real-time monitoring video, obtain the identification results, and acquire the current vehicle's signal information. Based on the identification results and signal information, the type of warning scenario can be determined. For example, when a lane dashed line is identified and a turn signal is acquired, the warning scenario type is determined to be a lane change warning; when a door opening signal is acquired, the warning scenario type is determined to be a door opening warning.

[0084] S603, based on the type information of the aforementioned warning scenario and the aforementioned perception information, determine the target display device among the aforementioned vehicle-mounted display devices.

[0085] In practical applications, based on the type and perception information of the warning scenario, the target display device corresponding to the target person among the occupants is determined. Specifically, if the target person is the driver, the corresponding target display device may include the central control display, steering wheel display, or head-up display; if the target person is a passenger, the corresponding target display device may include the front passenger display or rear seat display. For example, when the warning scenario type is a door opening warning, the door opening position affected by the warning target is determined based on the perception information of the warning target; and the target display device corresponding to the target person among the occupants is determined based on the door opening position.

[0086] S605, the above-mentioned output of the real-time warning video to the vehicle-side display device of the current vehicle includes outputting the real-time warning video to the target display device.

[0087] Using the technical solutions provided in the above embodiments, real-time early warning video can be output to the target display device corresponding to the target early warning personnel, thereby improving the pertinence and effectiveness of the early warning.

[0088] In an optional embodiment, such as Figure 7 As shown, Figure 7 This is a schematic diagram of a process for generating warning audio provided in an embodiment of this specification. Specifically, it may include:

[0089] S701, Based on the type information of the aforementioned warning scenario and the aforementioned perception information, generate the corresponding warning audio.

[0090] Specifically, multiple warning audio templates corresponding to the type information of the warning scenario are pre-set, and the target audio template is determined from the multiple warning audio templates based on the perception information of the warning target to generate the corresponding warning audio.

[0091] S703 outputs the aforementioned warning audio to the vehicle-mounted audio playback device of the current vehicle.

[0092] Using the technical solutions provided in the above embodiments, corresponding warning audio can be output simultaneously with the warning video, helping drivers and passengers avoid risks through multiple warning methods.

[0093] This application provides a video early warning device, such as... Figure 8 As shown, the above-mentioned device includes:

[0094] The real-time monitoring video acquisition module 810 is used to acquire real-time monitoring video from the vehicle-mounted camera currently on the vehicle.

[0095] The early warning scene recognition module 820 is used to recognize the early warning scene of the current vehicle based on the aforementioned real-time monitoring video.

[0096] The warning target determination module 830 is used to determine the warning target when the above-mentioned warning scenario for the current vehicle is identified;

[0097] The perception information module 840 is used to process the above-mentioned real-time monitoring video to obtain the perception information of the above-mentioned early warning target;

[0098] The real-time early warning video generation module 850 is used to mark the early warning targets in the real-time monitoring video based on the above-mentioned perception information to obtain the real-time early warning video.

[0099] The real-time warning video output module 860 is used to output the real-time warning video to the vehicle-side display device of the current vehicle.

[0100] In some embodiments, such as Figure 9 As shown, the aforementioned sensing information module 840 may include:

[0101] The target detection unit 841 is used to perform target detection on the above-mentioned real-time monitoring video and determine the type information of the above-mentioned warning target.

[0102] The target tracking unit 842 is used to track targets in the real-time monitoring video and determine multiple frames of images including the warning targets.

[0103] The motion trajectory analysis unit 843 performs motion trajectory analysis on the warning target based on the above-mentioned multi-frame images to determine the real-time relative position information, real-time speed information and real-time heading angle information of the warning target relative to the current vehicle.

[0104] The sensing information unit 844 is used to use the type information of the warning target, the real-time relative position information, the real-time speed information and the real-time heading angle information as the sensing information.

[0105] In some embodiments, the above-described apparatus may further include:

[0106] The target coordinate system construction unit is used to construct the target coordinate system corresponding to the above-mentioned multiple frames of images, with the current vehicle as the coordinate origin.

[0107] Correspondingly, the aforementioned real-time early warning video generation module 850 may include:

[0108] The coordinate information determination unit is used to determine the coordinate information of the warning target in the target coordinate system corresponding to the above-mentioned multi-frame images based on the above-mentioned real-time relative position information;

[0109] The first early warning target marking unit is used to mark the early warning target in the above-mentioned multi-frame images according to the type information and coordinate information of the early warning target, so as to obtain the above-mentioned real-time early warning video.

[0110] In an optional embodiment, the first warning target marking unit may include:

[0111] The annotation style information determination unit is used to determine the annotation style information corresponding to the warning target based on the type information of the warning target.

[0112] The second early warning target marking unit is used to mark early warning targets in the above multi-frame images according to the above marking style information and the above coordinate information, so as to obtain the marked multi-frame images.

[0113] The real-time early warning video generation unit is used to generate the real-time early warning video based on the marked multi-frame images.

[0114] In one specific embodiment, the above-described apparatus may further include:

[0115] The predicted motion trajectory generation unit is used to generate the predicted motion trajectory of the warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information.

[0116] The warning range determination unit is used to determine, based on the predicted motion trajectory, whether the warning target is within the warning range of the current vehicle.

[0117] The step execution unit is used to execute the step of marking the warning target in the real-time monitoring video based on the above-mentioned perception information to obtain the real-time warning video when the determination is yes.

[0118] In one specific embodiment, the above-described apparatus may further include:

[0119] The warning scenario type information determination unit is used to determine the type information of the warning scenario when the current vehicle is identified to be in a warning scenario.

[0120] The target display device determination unit is used to determine the target display device among the above-mentioned vehicle-mounted display devices based on the type information of the aforementioned early warning scenario and the aforementioned perception information.

[0121] Correspondingly, the aforementioned real-time early warning video output module 860 may include:

[0122] The target display device output unit is used to output the aforementioned real-time warning video to the aforementioned target display device.

[0123] In an optional embodiment, the above-described apparatus may further include:

[0124] The warning audio generation unit is used to generate corresponding warning audio based on the type information of the warning scenario and the perception information mentioned above.

[0125] The warning audio output unit is used to output the aforementioned warning audio to the vehicle-side audio playback device of the current vehicle.

[0126] This application provides a vehicle warning device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the vehicle warning method provided in the above method embodiments.

[0127] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the aforementioned devices. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0128] The methods and embodiments provided in this application can be executed in a mobile terminal, computer terminal, server, or similar computing device; that is, the aforementioned computer device may include a mobile terminal, computer terminal, server, or similar computing device. Taking running on a server as an example... Figure 10 This is a hardware structure block diagram of a warning server for a vehicle warning method provided in an embodiment of this application. For example... Figure 10 As shown, the early warning server 1000 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1010 (CPUs 1010 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 1030 for storing data, and one or more storage media 1020 (e.g., one or more mass storage devices) for storing application programs 1023 or data 1022. The memory 1030 and storage media 1020 may be temporary or persistent storage. The program stored in the storage media 1020 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 1010 may be configured to communicate with the storage media 1020 and execute the series of instruction operations in the storage media 1020 on the early warning server 1000. The early warning server 1000 may also include one or more power supplies 1060, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1040, and / or one or more operating systems 1021, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, etc.

[0129] The input / output interface 1040 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the early warning server 1000. In one example, the input / output interface 1040 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 1040 may be a radio frequency (RF) module for wireless communication with the Internet.

[0130] Those skilled in the art will understand that Figure 10 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the early warning server 1000 may also include... Figure 10 The more or fewer components shown, or having the same Figure 10 The different configurations shown.

[0131] This application embodiment also provides a storage medium, which can be set in a server to store at least one instruction or at least one program related to implementing a vehicle warning method in one of the method embodiments. The at least one instruction or the at least one program is loaded and executed by the processor to implement the vehicle warning method provided in the above method embodiment.

[0132] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0133] As can be seen from the embodiments of the vehicle warning method, device, equipment, or storage medium provided in this application, the technical solution provided in this application can acquire real-time monitoring video of the vehicle's environmental situation based on a vehicle-mounted camera; identify warning scenarios and confirm warning targets based on the real-time monitoring video; process the real-time monitoring video to obtain perceptual information of the warning targets; annotate the warning targets in multiple frames of the real-time monitoring video based on the perceptual information to generate real-time warning video, and output it to the display device in the vehicle, providing intuitive and clear danger warnings to the occupants, helping them quickly identify warning targets in the surrounding environment; it can also annotate the warning targets based on the annotation style information corresponding to the type information in the perceptual information, distinguishing the warning targets from other environments in the multiple frames of images, so as to intuitively display the type information and position information of the warning targets relative to the current vehicle to the occupants, improving the accuracy and speed of the warning; it can further determine the target display device corresponding to the target warning person among the occupants, and output the real-time warning video to the target display device, improving the pertinence and effectiveness of the warning; it can also output corresponding warning audio at the same time as outputting the warning video, helping occupants avoid risks through multiple warning methods.

[0134] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0136] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0137] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A vehicle early warning method, characterized in that, The method includes: Obtain real-time monitoring video from the vehicle's onboard camera; Based on the real-time monitoring video, the current vehicle is identified for a warning scenario; Upon identifying a warning scenario for the current vehicle, determine the warning target and the type information of the warning scenario; The real-time monitoring video is processed to obtain the perception information of the early warning target; Based on the perceived information, a predicted motion trajectory of the warning target is generated; Obtain real-time driving information of the current vehicle and generate a predicted motion trajectory of the current vehicle based on the real-time driving information; Based on the trajectory intersection result of the predicted motion trajectory of the warning target and the predicted motion trajectory of the current vehicle, it is determined whether the warning target is within the warning range of the current vehicle. If the warning target is within the warning range of the current vehicle, the warning target is marked in the real-time monitoring video based on the perception information to obtain a real-time warning video; Based on the type information of the warning scenario and the perception information, the target display device in the vehicle-mounted display device of the current vehicle is determined; The real-time warning video is output to the target display device.

2. The method according to claim 1, characterized in that, The process of processing the real-time monitoring video to obtain the perception information of the early warning target includes: Target detection is performed on the real-time monitoring video to determine the type information of the warning target; Target tracking is performed on the real-time monitoring video to determine multiple frames of images including the warning target; Based on the multi-frame images, the motion trajectory of the warning target is analyzed to determine the real-time relative position information, real-time speed information, and real-time heading angle information of the warning target relative to the current vehicle; The type information of the warning target, the real-time relative position information, the real-time speed information, and the real-time heading angle information are used as the sensing information.

3. The method according to claim 2, characterized in that, After processing the real-time monitoring video to obtain the perception information of the warning target, the method further includes: On each of the multiple image frames, a target coordinate system is constructed with the current vehicle as the origin. The step of marking the warning target in the real-time monitoring video based on the perceived information to obtain the real-time warning video includes: Based on the real-time relative position information, the coordinate information of the warning target in the target coordinate system corresponding to the multi-frame images is determined; Based on the type information and coordinate information of the warning target, the warning target is marked in the multi-frame images to obtain the real-time warning video.

4. The method according to claim 3, characterized in that, The step of marking the warning target in the multi-frame images according to the type information and coordinate information of the warning target to obtain the real-time warning video includes: Based on the type information of the warning target, determine the labeling style information corresponding to the warning target; Based on the annotation style information and the coordinate information, warning targets are marked in the multi-frame images to obtain marked multi-frame images; The real-time warning video is generated based on the marked multi-frame images.

5. The method according to claim 2, characterized in that, The process of generating the predicted motion trajectory of the early warning target based on the perceived information includes: The predicted trajectory of the warning target is generated based on the real-time relative position information, the real-time speed information, and the real-time heading angle information.

6. The method according to claim 1, characterized in that, After determining the type information of the warning scenario, the method further includes: Based on the type information of the warning scenario and the perception information, a corresponding warning audio is generated; The warning audio is output to the vehicle's audio playback device.

7. A vehicle warning device, characterized in that, The device includes: The real-time monitoring video acquisition module is used to acquire real-time monitoring video from the vehicle's onboard camera. The early warning scene recognition module is used to recognize the early warning scene of the current vehicle based on the real-time monitoring video; The warning target determination module is used to determine the warning target and the type information of the warning scenario when the current vehicle is identified to be in a warning scenario. The perception information module is used to process the real-time monitoring video to obtain the perception information of the early warning target; A predicted motion trajectory generation unit is used to generate a predicted motion trajectory of the warning target based on the perceived information; acquire real-time driving information of the current vehicle and generate a predicted motion trajectory of the current vehicle based on the real-time driving information; The warning range determination unit is used to determine whether the warning target is within the warning range of the current vehicle based on the trajectory intersection result of the predicted motion trajectory of the warning target and the predicted motion trajectory of the current vehicle. A real-time early warning video generation module is used to mark the early warning target in the real-time monitoring video based on the perception information if the early warning target is within the early warning range of the current vehicle, thereby obtaining a real-time early warning video; The target display device determination unit is used to determine the target display device in the vehicle-mounted display device of the current vehicle based on the type information of the warning scenario and the perception information. The target display device output unit is used to output the real-time warning video to the target display device.

8. The apparatus according to claim 7, characterized in that, The sensing information module includes: The target detection unit is used to perform target detection on the real-time monitoring video and determine the type information of the warning target; The target tracking unit is used to track targets in the real-time monitoring video and determine multiple frames of images including the warning target; The motion trajectory analysis unit performs motion trajectory analysis on the warning target based on the multi-frame images to determine the real-time relative position information, real-time speed information, and real-time heading angle information of the warning target relative to the current vehicle. The sensing information unit is used to use the type information of the warning target, the real-time relative position information, the real-time speed information, and the real-time heading angle information as the sensing information.

9. The apparatus according to claim 8, characterized in that, The device further includes: The target coordinate system construction unit is used to construct the target coordinate system corresponding to the multiple frames of images, with the current vehicle as the coordinate origin, on the multiple frames of images respectively; Correspondingly, the real-time early warning video generation module includes: The coordinate information determination unit is used to determine the coordinate information of the warning target in the target coordinate system corresponding to the multi-frame images based on the real-time relative position information. The first early warning target marking unit is used to mark the early warning target in the multi-frame images according to the type information and coordinate information of the early warning target, so as to obtain the real-time early warning video.

10. The apparatus according to claim 9, characterized in that, The first warning target marking unit further includes: The annotation style information determination unit is used to determine the annotation style information corresponding to the warning target based on the type information of the warning target; The second early warning target marking unit is used to mark early warning targets in the multi-frame images according to the marking style information and the coordinate information, so as to obtain the marked multi-frame images; A real-time early warning video generation unit is used to generate the real-time early warning video based on the marked multi-frame images.

11. The apparatus according to claim 8, characterized in that, The predicted motion trajectory generation unit is also used to generate the predicted motion trajectory of the warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information.

12. The apparatus according to claim 7, characterized in that, The device further includes: The warning audio generation unit is used to generate corresponding warning audio based on the type information of the warning scenario and the perception information; The warning audio output unit is used to output the warning audio to the vehicle-mounted audio playback device of the current vehicle.

13. A vehicle warning device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the vehicle warning method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the vehicle warning method as described in any one of claims 1 to 6.

Images