A vehicle warning method, system, device, apparatus and storage medium

Abstract

A vehicle early warning method, system, device, equipment and storage medium, the method comprises: receiving the real-time monitoring video of a plurality of vehicle cameras on a target vehicle sent by the vehicle terminal of the target vehicle; based on the real-time monitoring video, the target vehicle is identified, and the identification result is obtained, the identification result includes the early warning target (S205); the real-time monitoring video is processed, and the perception information of the early warning target is obtained (S207); based on the perception information, the early warning target is marked in the real-time monitoring video, and the real-time early warning video is obtained (S209); the real-time early warning video is output to the display device (S211). With the above method, the dangerous target around the vehicle can be displayed intuitively and clearly, helping the remote driver to discover the dangerous target in time to avoid danger, on the one hand, improving the timeliness and accuracy of early warning, on the other hand, protecting the property and personal safety of the driver and passenger.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote control driving, in particular to a vehicle early warning method, system, device, equipment and storage medium. BACKGROUND

[0002] Remote control driving systems have been developed and popularized by traditional car companies and Internet companies in recent years, and are mainly used in harsh environments and weather, port operations, logistics transportation, L3 / L4 level automatic driving vehicle remote takeover, etc. The remote control driving system on the market now is to monitor the environment around the vehicle through the camera on the vehicle, and transmit the monitoring video to the remote driving control end through the vehicle terminal and display it on the display of the remote control end. The remote driver observes the environment around the vehicle displayed on the display screen, and controls the vehicle end using the cockpit simulator of the remote control end. Specifically, the remote control end sends control instructions to the vehicle terminal, and the vehicle end executes the corresponding control instructions.

[0003] However, there is no corresponding danger warning system in the current remote control driving system. When there is a dangerous scene around the vehicle, such as vehicle rear-end / collision, pedestrian / electric vehicle suddenly appearing, video dead angle, etc., the remote driver can only judge the danger of the scene through the video display of the remote control end and avoid the danger by operating the cockpit simulator. Due to the limitations of the frame rate of the display and the angle of the camera, the remote driver cannot make timely and accurate judgments on many dangerous scenes, which leads to the inability to guarantee the safety of remote driving. Therefore, a more effective technical solution is needed. SUMMARY

[0004] In order to solve the problems of the prior art, the present application provides a vehicle early warning method, system, device, equipment and storage medium. The technical solution is as follows:

[0005] On the one hand, a vehicle early warning method is provided, which comprises:

[0006] The vehicle terminal of the target vehicle acquires real-time monitoring videos of a plurality of vehicle cameras on the target vehicle;

[0007] The vehicle terminal sends the real-time monitoring videos to the remote driving control end of the target vehicle;

[0008] The remote driving control end performs early warning scene recognition on the target vehicle based on the real-time monitoring videos, and obtains a recognition result, wherein the recognition result includes a warning target;

[0009] The remote driving control end processes the real-time monitoring videos to obtain perception information of the warning target;

[0010] The remote driving remote control end marks the early warning target in the real-time monitoring video based on the perception information, to obtain a real-time early warning video.

[0011] The remote driving remote control end outputs the real-time early warning video to a display device.

[0012] In another aspect, a vehicle early warning method is provided, the method comprising:

[0013] receiving real-time monitoring videos of multiple vehicle-mounted cameras on a target vehicle sent by a vehicle-mounted terminal of the target vehicle;

[0014] performing early warning scene identification on the target vehicle based on the real-time monitoring videos, to obtain an identification result, the identification result comprising an early warning target;

[0015] processing the real-time monitoring videos to obtain perception information of the early warning target;

[0016] marking the early warning target in the real-time monitoring video based on the perception information, to obtain a real-time early warning video;

[0017] outputting the real-time early warning video to a display device.

[0018] In another aspect, a vehicle early warning system is provided, the system comprising: a vehicle-mounted terminal of a target vehicle and a remote driving remote control end of the target vehicle; wherein,

[0019] the vehicle-mounted terminal is configured to acquire real-time monitoring videos of multiple vehicle-mounted cameras on the target vehicle; and send the real-time monitoring videos to the remote driving remote control end;

[0020] the remote driving remote control end is configured to receive the real-time monitoring videos sent by the vehicle-mounted terminal; perform early warning scene identification on the target vehicle based on the real-time monitoring videos, to obtain an identification result, the identification result comprising an early warning target; process the real-time monitoring videos to obtain perception information of the early warning target; mark the early warning target in the real-time monitoring video based on the perception information, to obtain a real-time early warning video; and output the real-time early warning video to a display device.

[0021] In another aspect, a vehicle early warning device is provided, the device comprising:

[0022] a real-time monitoring video receiving module configured to receive real-time monitoring videos of multiple vehicle-mounted cameras on a target vehicle sent by a vehicle-mounted terminal of the target vehicle;

[0023] The early warning scene recognition module is configured to perform early warning scene recognition on the target vehicle based on the real-time monitoring video, and obtain a recognition result, wherein the recognition result comprises a warning target;

[0024] The perception information module is configured to process the real-time monitoring video, and obtain perception information of the warning target.

[0025] The early warning target marking module is configured to mark the warning target in the real-time monitoring video based on the perception information, and obtain a real-time early warning video.

[0026] The real-time early warning video output module is configured to output the real-time early warning video to a display device.

[0027] In another aspect, a vehicle early warning device is provided, which comprises a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the vehicle early warning method as described above.

[0028] In another aspect, a computer readable storage medium is provided, wherein the storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the vehicle early warning method as described above.

[0029] The vehicle early warning method, system, device, equipment and storage medium provided by the present application have the following technical effects:

[0030] The technical solution provided by the present application places the monitoring, marking and early warning algorithm into the remote driving remote control end, the remote driving remote control end receives the real-time monitoring video sent by the vehicle terminal, marks the dangerous target in the dangerous scene about to occur in the real-time monitoring video, generates a corresponding early warning video, and outputs the early warning video to a display device, which intuitively and clearly displays the dangerous target around the vehicle, helps the remote driver to discover the dangerous target in time to avoid danger, on the one hand, realizes timely and accurate early warning, and on the other hand, protects the property and personal safety of the driver and passenger. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0032] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0033] Figure 2 is a flowchart of a vehicle early warning method provided by an embodiment of the present application;

[0034] Figure 3 is a flowchart of a method for sending real-time monitoring video provided by an embodiment of the present application;

[0035] Figure 4 is a flowchart of a method for processing real-time monitoring video provided by an embodiment of the present application;

[0036] Figure 5 is a flowchart of a method for positioning and marking coordinates of an early warning target provided by an embodiment of the present application;

[0037] Figure 6 is a flowchart of a method for determining an early warning range provided by an embodiment of the present application;

[0038] Figure 7 is a flowchart of a method for determining a target display device provided by an embodiment of the present application;

[0039] Figure 8 is a flowchart of a method for outputting real-time early warning video provided by an embodiment of the present application;

[0040] Figure 9 is a flowchart of another vehicle early warning method provided by an embodiment of the present application;

[0041] Figure 10 is a schematic diagram of a vehicle early warning device provided by an embodiment of the present application;

[0042] Figure 11 is a schematic diagram of a perception information module provided by an embodiment of the present application; DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0045] Please refer to Figure 1 , Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application, as Figure 1 indicated, the application environment includes a vehicle terminal 01 and a remote driving remote control end 02.

[0046] Specifically, the vehicle terminal 01 can include a display device, a vehicle camera, a network communication unit, a processor and a memory, etc. Specifically, the vehicle terminal 01 can obtain real-time monitoring videos of a plurality of vehicle cameras on a target vehicle; and send the real-time monitoring videos to the remote driving remote control end.

[0047] Specifically, the remote driving remote control end 02 can include a cockpit simulator, a display device, a network communication unit, a processor and a memory, etc. Specifically, the remote driving remote control end 02 can receive the real-time monitoring videos sent by the vehicle terminal; perform early warning scene identification on the target vehicle based on the real-time monitoring videos to obtain an identification result, the identification result including an early warning target; process the real-time monitoring videos to obtain perception information of the early warning target; mark the early warning target in the real-time monitoring videos based on the perception information to obtain a real-time early warning video; and output the real-time early warning video to the display device.

[0048] In actual application, the remote driving remote control end can interact with the vehicle terminal through wireless communication, and the remote driving remote control end perceives the environment around the vehicle end through the real-time monitoring videos transmitted by the vehicle terminal, and then sends corresponding control instructions to the vehicle terminal from the remote driving remote control end to ensure the normal driving of the vehicle.

[0049] A vehicle early warning method provided by an embodiment of the present application is introduced below, Figure 2A flowchart of a vehicle early warning method provided in an embodiment of the present application. It should be noted that the present specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. In actual system or product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically as Figure 2 The above method can include:

[0050] S201, the vehicle terminal of the target vehicle obtains real-time monitoring videos of a plurality of vehicle cameras on the target vehicle.

[0051] In an embodiment of the present specification, the target vehicle can be a vehicle based on remote control driving, and the target vehicle can include a vehicle terminal. Optionally, the remote driving control end can interact with the vehicle terminal through wireless communication. Specifically, the remote driving control end sends a vehicle control instruction to the vehicle terminal, and the vehicle terminal responds to the vehicle control instruction to control the vehicle end effector to execute the corresponding instruction.

[0052] In actual application, the vehicle based on remote control driving is equipped with a plurality of vehicle cameras in each direction at the vehicle end to monitor the surrounding environment of the vehicle end in real time.

[0053] S203, the vehicle terminal sends the real-time monitoring video to the remote driving control end of the target vehicle.

[0054] In actual application, the driver of the remote driving control end observes the surrounding environment of the vehicle end based on the real-time monitoring video sent by the vehicle terminal, and controls the vehicle end through the cockpit simulator of the remote driving control end.

[0055] In an optional embodiment, as Figure 3 The vehicle terminal sending the real-time monitoring video to the remote driving control end of the target vehicle can include:

[0056] S301, the vehicle terminal encodes the real-time monitoring video to obtain an encoded real-time monitoring video.

[0057] Specifically, the real-time monitoring video is encoded to compress the video pixel data, reduce the data volume of the video, improve the speed of video transmission, and ensure the real-time performance of the remote driving control end receiving the monitoring video.

[0058] S303, the vehicle terminal sends the encoded real-time monitoring video to the remote driving control end.

[0059] Optionally, the receiving the encoded real-time monitoring video sent by the vehicle terminal can comprise: receiving the encoded real-time monitoring video sent by the vehicle terminal based on 5G communication.

[0060] In actual application, video transmission based on 5G communication can further reduce the time delay in video transmission, so that the remote driving control end can receive the monitoring video of the vehicle end in real time.

[0061] S305, the remote driving control end decodes the encoded real-time monitoring video to obtain the real-time monitoring video.

[0062] Specifically, the encoded real-time monitoring video is decoded to obtain the original real-time monitoring video, so as to ensure the accurate transmission of information in the video.

[0063] S205, the remote driving control end performs early warning scene identification on the target vehicle based on the real-time monitoring video to obtain an identification result, wherein the identification result comprises an early warning target.

[0064] Specifically, the early warning scene identification is performed on the environment around the vehicle end according to the real-time monitoring video, and if there is an early warning target affecting the safety of the target vehicle in the environment where the target vehicle is located, the environment where the target vehicle is located is identified as an early warning scene. Specifically, the early warning scene can comprise a scene threatening the safety of the target vehicle. In specific embodiments, the early warning scene can comprise but is not limited to: a lane changing early warning scene, a reversing early warning scene, and a door opening early warning scene.

[0065] In the embodiments of the present application, the real-time monitoring video is identified based on a preset early warning scene identification algorithm to obtain an identification result. Specifically, the preset early warning scene identification algorithm can comprise a plurality of identification algorithms for identifying different types of early warning scenes, and each identification algorithm can be obtained based on a large number of sample video images corresponding to each early warning scene.

[0066] In actual application, the early warning target can be a dynamic early warning target or a static early warning target. The dynamic early warning target can comprise a person or object in motion in the environment where the target vehicle is located and affecting the safe driving of the target vehicle. In specific embodiments, the dynamic early warning target can comprise but is not limited to a vehicle in motion and a pedestrian. The static early warning target can comprise a person or object in a stationary state in the environment where the target vehicle is located and affecting the safe driving of the target vehicle. In specific embodiments, the static early warning target can comprise but is not limited to a stationary vehicle, a road, and a tree.

[0067] S207, the remote driving control end processes the real-time monitoring video to obtain perception information of the early warning target.

[0068] In some embodiments, as Figure 4 shown, Figure 4 is a flowchart of a processing method for real-time monitoring video provided by an embodiment of the present specification, and specifically can include:

[0069] S401, the remote driving remote control end detects the target of the real-time monitoring video to determine the type information of the early warning target.

[0070] In one specific embodiment, determining the type information of the early warning target can include:

[0071] 1) performing target detection on the real-time monitoring video to obtain feature information of the early warning target, the feature information can include but is not limited to: height information, width information, pixel information;

[0072] 2) based on the feature information, identifying the type of the early warning target to determine the type information of the early warning target, the type information of the early warning target can represent the classification category of the early warning target, specifically, the type information can include but is not limited to: pedestrian, vehicle, road, tree.

[0073] S403, the remote driving remote control end performs target tracking on the real-time monitoring video to determine multiple frames of images including the early warning target.

[0074] Specifically, multiple frames of images containing the early warning target are collected from the images of the real-time monitoring video, and the multiple frames of images can include consecutive frames of images.

[0075] S405, the remote driving remote control end performs motion trajectory analysis on the early warning target based on the multiple frames of images to determine the real-time relative position information, real-time speed information and real-time heading angle information of the early warning target relative to the target vehicle.

[0076] Specifically, the motion trajectory of the early warning target is extracted from the multiple frames of images, and the motion trajectory is analyzed to obtain the real-time relative position information, real-time speed information and real-time heading angle information of the target vehicle, and the relative position information can include but is not limited to distance information.

[0077] S407, the remote driving remote control end takes the type information of the early warning target, the real-time relative position information, the real-time speed information and the real-time heading angle information as the perception information.

[0078] By using the technical solutions of the above embodiments, the perception information of the early warning target including type information, real-time relative position information, real-time speed information and real-time heading angle information can be obtained.

[0079] S209, the remote driving remote control end marks the pre-warning target in the real-time monitoring video based on the perception information, and obtains a real-time pre-warning video.

[0080] In some embodiments, as Figure 5 shown, after the remote driving remote control end processes the real-time monitoring video and obtains the perception information of the pre-warning target, the method can further include:

[0081] S409, the remote driving remote control end constructs a target coordinate system corresponding to the multiple images respectively with the target vehicle as the coordinate origin.

[0082] Specifically, a corresponding target coordinate system is constructed on each of the multiple images, and the coordinate origin of the target coordinate system is the target vehicle. The target coordinate system can include but is not limited to a two-dimensional coordinate system including an X-axis and a Y-axis.

[0083] Correspondingly, the remote driving remote control end marks the pre-warning target in the real-time monitoring video based on the perception information, and obtains a real-time pre-warning video, which can include:

[0084] S411, the remote driving remote control end determines the coordinate information of the pre-warning target in the target coordinate system corresponding to the multiple images according to the real-time relative position information.

[0085] Specifically, according to the real-time relative position information of the pre-warning target relative to the target vehicle, the coordinate information of the pre-warning target in the target coordinate system with the target vehicle as the coordinate origin is determined, so as to position the pre-warning target in the multiple images.

[0086] S413, the remote driving remote control end marks the pre-warning target in the multiple images according to the type information of the pre-warning target and the coordinate information, and obtains the real-time pre-warning video.

[0087] In one specific embodiment, marking the pre-warning target can include:

[0088] 1) determining the annotation style information corresponding to the pre-warning target according to the type information of the pre-warning target.

[0089] In actual application, for each type of information, the corresponding annotation style information is set in advance, and the annotation style information is used to distinguish pre-warning targets of different types of information. Specifically, the annotation style information can include but is not limited to color, contour format, and filling format.

[0090] 2) marking the pre-warning target in the multiple images according to the annotation style information and the coordinate information, and obtaining the multiple images after marking.

[0091] Specifically, according to the labeling style information and the coordinate information, the pre-warning target is labeled in the multiple frames of images, and a technical effect of distinguishing the pre-warning target from other environments in the multiple frames of images is achieved, so as to intuitively show the type information of the pre-warning target and the position information relative to the target vehicle to the driver and passenger.

[0092] 3) According to the multiple frames of images after labeling, the real-time pre-warning video is generated.

[0093] In some embodiments, as shown in Figure 6 , Figure 6 is a flowchart for judging a pre-warning range provided by an embodiment of the present specification, and specifically can include:

[0094] S601, the remote driving remote control end generates a predicted motion trajectory of the pre-warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information.

[0095] Specifically, the predicted motion trajectory is generated by the real-time relative position information, the real-time speed information and the real-time heading angle information according to a preset motion trajectory generation algorithm. The motion trajectory generation algorithm can be obtained by inductive summary based on multiple sample real-time position information, sample real-time speed information, sample real-time heading angle information and corresponding sample motion trajectory.

[0096] S603, the remote driving remote control end judges whether the pre-warning target is within the pre-warning range of the target vehicle based on the predicted motion trajectory.

[0097] Specifically, the real-time driving information of the target vehicle can be obtained, and the predicted motion trajectory of the target vehicle is generated based on the real-time driving information. When the predicted motion trajectory of the pre-warning target intersects with the predicted motion trajectory of the target vehicle, it is judged that the pre-warning target is within the pre-warning range of the target vehicle.

[0098] S605, when it is judged to be yes, the remote driving remote control end executes the step of labeling the pre-warning target in the real-time monitoring video based on the perception information to obtain the real-time pre-warning video.

[0099] In actual application, when the pre-warning target is within the pre-warning range of the target vehicle, the pre-warning target is labeled in the real-time monitoring video based on the perception information to obtain the real-time pre-warning video.

[0100] S211, the remote driving remote control end outputs the real-time pre-warning video to a display device.

[0101] In the embodiment of the present specification, as shown in Figure 7As shown, when the identification result further includes the type information of the early warning scene, before the remote driving remote control end outputs the real-time early warning video to the display device, the method can further include:

[0102] S701, the remote driving remote control end determines the target display device of the early warning video based on the type information of the early warning scene.

[0103] In actual application, the application scenarios of remote driving control are widely used, and in some application scenarios, the vehicle end and the remote control end need to jointly or separately warn the dangerous scene.

[0104] In a specific embodiment, according to the type information of the early warning scene and the perception information of the early warning target, the target warning personnel in the driver and passenger corresponding to the early warning video is determined, and the corresponding target display device is determined based on the target warning personnel. Specifically, if the target warning personnel is the driver, the corresponding target display device can include the remote driving remote control end display device; if the target warning personnel is the passenger, the corresponding target display device can include the vehicle end display device. For example, when the type information of the early warning scene is door opening warning, the target warning personnel is the passenger, and the target display device is the vehicle end display device; when the type information of the early warning scene is lane changing warning, the target warning personnel is the driver, and the target display device is the remote driving remote control end display device.

[0105] S703, when the target display device includes the vehicle end display device, the remote driving remote control end outputs the real-time early warning video to the display device, including the remote driving remote control end outputting the real-time early warning video to the vehicle end display device.

[0106] In an optional embodiment, as shown, the remote driving remote control end outputs the real-time early warning video to the vehicle end display device, which can include: Figure 8

[0107] S801, the remote driving remote control end encodes the real-time early warning video to obtain the encoded real-time early warning video.

[0108] In actual application, by encoding the real-time early warning video, the video pixel data is compressed, the data amount of the video is reduced, the speed of video transmission is improved, and the real-time performance of the vehicle terminal receiving the early warning video is ensured.

[0109] S803, the remote driving remote control end sends the encoded real-time early warning video to the vehicle terminal.

[0110] Optionally, the remote driving remote control end sends the encoded real-time early warning video to the vehicle terminal, which can include that the remote driving remote control end sends the encoded real-time early warning video to the vehicle terminal based on 5G communication.​

[0111] In practical applications, video transmission based on 5G communication can further reduce the time delay during video transmission, so that the vehicle terminal can receive the monitoring video in real time.

[0112] S805, the vehicle terminal decodes the encoded real-time warning video to obtain the real-time warning video.

[0113] In practical applications, the encoded real-time warning video is decoded to obtain the original real-time warning video, so as to ensure accurate transmission of information in the video.

[0114] S807, the vehicle terminal outputs the real-time warning video to the vehicle display device.

[0115] The technical scheme of the embodiment of the specification can quickly transmit the warning video from the remote driving remote control end to the vehicle terminal, and timely warn the passengers in the vehicle.

[0116] S705, when the target display device includes a remote driving remote control end display device, the remote driving remote control end outputs the real-time warning video to the display device, including the remote driving remote control end outputting the real-time warning video to the remote driving remote control end display device.

[0117] Specifically, when the target display device includes the remote driving remote control end display device and the vehicle display device, outputting the real-time warning video to the display device can include outputting the real-time warning video to the remote driving remote control end display device and outputting the real-time warning video to the vehicle display device.

[0118] In practical applications, the warning audio corresponding to the type information of the warning scene can also be set in advance, and the corresponding warning audio is output at the same time as the real-time warning video is output to the target display device, further reminding the driver and passenger to avoid danger.

[0119] As can be seen from the above embodiments of the specification, the remote driving remote control end receives the real-time monitoring video sent by the vehicle terminal, marks the warning target in the real-time monitoring video to obtain the real-time warning video, and displays the real-time warning video to the remote driver through the remote driving remote control end display device, helping the remote driver to avoid risks. When the warning target affects the passengers in the vehicle, the real-time warning video is sent to the vehicle terminal and displayed to the passengers in the vehicle through the vehicle display device, effectively improving the accuracy and pertinence of the warning.

[0120] The following describes a specific embodiment of another vehicle warning method of the specification with the remote driving remote control end as the execution subject, Figure 9is a flowchart of another vehicle early warning method provided by the embodiment of the present application, the present specification provides the method operation steps as described in the embodiment or flowchart, but more or less operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiment is only one of the many step execution orders, and does not represent the only execution order. In actual system or server product execution, the method order shown in the embodiment or the drawing can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically as Figure 9 The above method can include:

[0121] S901, receiving the real-time monitoring video of the plurality of vehicle-mounted cameras on the target vehicle sent by the vehicle terminal of the target vehicle.

[0122] In an optional embodiment, the above receiving the real-time monitoring video of the plurality of vehicle-mounted cameras on the target vehicle sent by the vehicle terminal of the target vehicle can include:

[0123] 1) receiving the encoded real-time monitoring video sent by the vehicle terminal, the encoded real-time monitoring video being obtained by encoding the real-time monitoring video by the vehicle terminal.

[0124] Optionally, the above receiving the encoded real-time monitoring video sent by the vehicle terminal can include receiving the encoded real-time monitoring video sent by the vehicle terminal based on 5G communication.

[0125] 2) decoding the encoded real-time monitoring video to obtain the real-time monitoring video.

[0126] S903, performing early warning scene recognition on the target vehicle based on the real-time monitoring video to obtain a recognition result, the recognition result including a warning target.

[0127] S905, processing the real-time monitoring video to obtain perception information of the warning target.

[0128] In some embodiments, the above processing the real-time monitoring video to obtain the perception information of the warning target can include:

[0129] 1) performing target detection on the real-time monitoring video to determine type information of the warning target.

[0130] 2) performing target tracking on the real-time monitoring video to determine a plurality of images including the warning target.

[0131] 3) based on the above-mentioned multi-frame images, performing motion trajectory analysis on the above-mentioned early warning target to determine real-time relative position information, real-time speed information and real-time heading angle information of the above-mentioned early warning target relative to the above-mentioned target vehicle.

[0132] 4) taking the type information of the above-mentioned early warning target, the above-mentioned real-time relative position information, the above-mentioned real-time speed information and the above-mentioned real-time heading angle information as the above-mentioned perception information.

[0133] By using the technical solutions of the above-mentioned embodiments, the perception information of the early warning target including type information, real-time relative position information, real-time speed information and real-time heading angle information can be obtained.

[0134] S907, based on the above-mentioned perception information, marking the above-mentioned early warning target in the above-mentioned real-time monitoring video to obtain a real-time early warning video.

[0135] In some embodiments, after processing the above-mentioned real-time monitoring video to obtain the perception information of the above-mentioned early warning target, the above-mentioned method can further include:

[0136] respectively taking the target vehicle as a coordinate origin on the multi-frame images to construct a target coordinate system corresponding to the multi-frame images.

[0137] Correspondingly, the above-mentioned marking the above-mentioned early warning target in the above-mentioned real-time monitoring video based on the above-mentioned perception information to obtain a real-time early warning video can include:

[0138] 1) determining coordinate information of the above-mentioned early warning target in the target coordinate system corresponding to the multi-frame images according to the above-mentioned real-time relative position information.

[0139] 2) performing early warning target marking in the multi-frame images according to the type information of the above-mentioned early warning target and the above-mentioned coordinate information to obtain the above-mentioned real-time early warning video.

[0140] In a specific embodiment, the above-mentioned performing early warning target marking in the multi-frame images according to the type information of the above-mentioned early warning target and the above-mentioned coordinate information to obtain the above-mentioned real-time early warning video can include:

[0141] 1) determining annotation style information corresponding to the above-mentioned early warning target according to the type information of the above-mentioned early warning target.

[0142] 2) performing early warning target marking in the multi-frame images according to the above-mentioned annotation style information and the above-mentioned coordinate information to obtain marked multi-frame images.

[0143] 3) generating the above-mentioned real-time early warning video according to the above-mentioned marked multi-frame images.

[0144] In some embodiments, before the step of marking the pre-warning target in the real-time monitoring video based on the perception information to obtain the real-time pre-warning video, the method can further include:

[0145] 1) generating a predicted motion trajectory of the pre-warning target based on the real-time relative position information, the real-time speed information, and the real-time heading angle information.

[0146] 2) judging whether the pre-warning target is within the pre-warning range of the target vehicle based on the predicted motion trajectory.

[0147] 3) when the judgment is yes, performing the step of marking the pre-warning target in the real-time monitoring video based on the perception information to obtain the real-time pre-warning video.

[0148] S909, outputting the real-time pre-warning video to a display device.

[0149] In one specific embodiment, when the identification result further includes type information of the pre-warning scene, before the step of outputting the real-time pre-warning video to the display device, the method can further include:

[0150] 1) determining a target display device of the pre-warning video based on the type information of the pre-warning scene.

[0151] 2) when the target display device includes the vehicle-end display device, the step of outputting the real-time pre-warning video to the display device includes outputting the real-time pre-warning video to the vehicle-end display device.

[0152] In one optional embodiment, the step of outputting the real-time pre-warning video to the vehicle-end display device can include:

[0153] (1) encoding the real-time pre-warning video to obtain an encoded real-time pre-warning video.

[0154] (2) sending the encoded real-time pre-warning video to the vehicle-mounted terminal, so that the vehicle-mounted terminal decodes the encoded real-time pre-warning video to obtain the real-time pre-warning video, and outputs the real-time pre-warning video to the vehicle-end display device.

[0155] Optionally, the step of sending the encoded real-time pre-warning video to the vehicle-mounted terminal can include sending the encoded real-time pre-warning video to the vehicle-mounted terminal based on 5G communication.

[0156] 3) when the target display device includes the remote driving remote control end display device, the step of outputting the real-time pre-warning video to the display device includes outputting the real-time pre-warning video to the remote driving remote control end display device.

[0157] In the embodiments of the present application, when the target display device includes the remote driving remote control end display device and the vehicle end display device, the outputting the real-time early warning video to the display device can include outputting the real-time early warning video to the remote driving remote control end display device and outputting the real-time early warning video to the vehicle end display device.

[0158] As can be seen from the above embodiments of the present application, the real-time monitoring video sent by the vehicle-mounted terminal is received by the remote driving remote control end, the dangerous target in the dangerous scene about to occur is marked in the real-time monitoring video, the corresponding early warning video is generated, and the early warning video is output to the display device, so that the dangerous target around the vehicle is intuitively and clearly displayed, and timely and accurate early warning is achieved.

[0159] The specific refinement steps of the embodiments of the vehicle early warning method written from the remote driving remote control end can be referred to the embodiments of the interactive side vehicle early warning method, which will not be described here.

[0160] The embodiments of the present application also provide a vehicle early warning system for realizing the above method. Figure 2 The system embodiments correspond to the above method embodiments, and can realize all the contents of the above method embodiments. For easy reading, the system embodiments only briefly describe the contents of the above method embodiments, and do not describe the details of the method embodiments one by one. The above system includes a vehicle-mounted terminal of a target vehicle and a remote driving remote control end of the target vehicle; wherein,

[0161] The vehicle-mounted terminal is configured to acquire real-time monitoring videos of a plurality of vehicle-mounted cameras on the target vehicle; and send the real-time monitoring videos to the remote driving remote control end.

[0162] The remote driving remote control end is configured to receive the real-time monitoring videos sent by the vehicle-mounted terminal; perform early warning scene identification on the target vehicle based on the real-time monitoring videos to obtain an identification result, wherein the identification result includes a warning target; process the real-time monitoring videos to obtain perception information of the warning target; mark the warning target in the real-time monitoring videos based on the perception information to obtain a real-time early warning video; and output the real-time early warning video to a display device.

[0163] In an optional embodiment, the vehicle-mounted terminal is further configured to encode the real-time monitoring videos to obtain encoded real-time monitoring videos; and send the encoded real-time monitoring videos to the remote driving remote control end.

[0164] The remote driving remote control terminal is further configured to receive the encoded real-time monitoring video sent by the vehicle terminal, and decode the encoded real-time monitoring video to obtain the real-time monitoring video.

[0165] In an optional embodiment, the vehicle terminal is further configured to send the encoded real-time monitoring video to the remote driving remote control terminal based on 5G communication.

[0166] The remote driving remote control terminal is further configured to receive the encoded real-time monitoring video sent by the vehicle terminal based on 5G communication.

[0167] In some embodiments, the remote driving remote control terminal is further configured to perform target detection on the real-time monitoring video to determine type information of the early warning target, perform target tracking on the real-time monitoring video to determine a plurality of images including the early warning target, perform motion trajectory analysis on the early warning target based on the plurality of images to determine real-time relative position information, real-time speed information, and real-time heading angle information of the early warning target relative to the target vehicle, and use the type information of the early warning target, the real-time relative position information, the real-time speed information, and the real-time heading angle information as the perception information.

[0168] Specifically, the remote driving remote control terminal is further configured to construct a target coordinate system corresponding to the plurality of images with the target vehicle as the coordinate origin on the plurality of images, determine coordinate information of the early warning target in the target coordinate system corresponding to the plurality of images according to the real-time relative position information, and perform early warning target marking in the plurality of images according to the type information of the early warning target and the coordinate information to obtain the real-time early warning video.

[0169] In some embodiments, the remote driving remote control terminal is further configured to generate a predicted motion trajectory of the early warning target based on the real-time relative position information, the real-time speed information, and the real-time heading angle information, determine whether the early warning target is within an early warning range of the target vehicle based on the predicted motion trajectory, and perform the step of marking the early warning target in the real-time monitoring video based on the perception information to obtain the real-time early warning video when the determination is positive.

[0170] In a specific embodiment, when the identification result further includes type information of the early warning scene, the remote driving remote control terminal is further configured to determine a target display device of the early warning video based on the type information of the early warning scene, output the real-time early warning video to a vehicle-end display device when the target display device is the vehicle-end display device, and output the real-time early warning video to a remote driving remote control terminal display device when the target display device is the remote driving remote control terminal display device.

[0171] In an optional embodiment, the remote driving remote control end described above is further configured to encode the real-time early warning video to obtain an encoded real-time early warning video, and send the encoded real-time early warning video to the vehicle-mounted terminal.

[0172] The vehicle-mounted terminal is further configured to receive the encoded real-time early warning video sent by the remote driving remote control end.

[0173] In an optional embodiment, the remote driving remote control end described above is further configured to send the encoded real-time early warning video to the vehicle-mounted terminal based on 5G communication.

[0174] The vehicle-mounted terminal is further configured to receive the encoded real-time early warning video sent by the remote driving remote control end based on 5G communication.

[0175] From the above description, it can be seen that in the embodiments of the present application, the vehicle-mounted terminal of the target vehicle acquires real-time monitoring videos of multiple vehicle-mounted cameras on the target vehicle and sends the real-time monitoring videos to the remote driving remote control end of the target vehicle. Then, the remote driving remote control end performs early warning scene identification on the target vehicle based on the real-time monitoring videos, obtains an identification result, the identification result includes an early warning target, and processes the real-time monitoring videos to obtain perception information of the early warning target. Then, based on the perception information, the early warning target is marked in the real-time monitoring videos to obtain a real-time early warning video. Finally, the real-time early warning video is output to a display device to intuitively and clearly display dangerous targets around the vehicle, helping the remote driver to discover dangerous targets in time to avoid danger and improving the timeliness and accuracy of early warning.

[0176] The embodiments of the present application provide a video early warning device, as shown in Figure 10 The device comprises:

[0177] The real-time monitoring video receiving module 1010 is configured to receive real-time monitoring videos of multiple vehicle-mounted cameras on a target vehicle sent by a vehicle-mounted terminal of the target vehicle.

[0178] The early warning scene identification module 1020 is configured to perform early warning scene identification on the target vehicle based on the real-time monitoring videos to obtain an identification result, the identification result including an early warning target.

[0179] The perception information module 1030 is configured to process the real-time monitoring videos to obtain perception information of the early warning target.

[0180] The early warning target marking module 1040 is configured to mark the early warning target in the real-time monitoring videos based on the perception information to obtain a real-time early warning video.

[0181] The real-time early warning video output module 1050 is configured to output the real-time early warning video to a display device.

[0182] In an optional embodiment, the real-time monitoring video receiving module 1010 described above can include:

[0183] The first video receiving unit is configured to receive the encoded real-time monitoring video sent by the vehicle terminal central control, wherein the encoded real-time monitoring video is obtained by encoding the real-time monitoring video by the vehicle terminal central control.

[0184] The video decoding unit is configured to decode the encoded real-time monitoring video to obtain the real-time monitoring video.

[0185] In an optional embodiment, the first video receiving unit described above can include:

[0186] The second video receiving unit is configured to receive the encoded real-time monitoring video sent by the vehicle terminal central control based on 5G communication.

[0187] In some embodiments, as shown in FIG. 10B, the perception information module 1030 can include: Figure 11

[0188] The target detection unit 1031 is configured to perform target detection on the real-time monitoring video to determine the type information of the early warning target.

[0189] The target tracking unit 1032 is configured to perform target tracking on the real-time monitoring video to determine a plurality of images including the early warning target.

[0190] The motion trajectory analysis unit 1033 is configured to perform motion trajectory analysis on the early warning target based on the plurality of images to determine real-time relative position information, real-time speed information, and real-time heading angle information of the early warning target relative to the target vehicle.

[0191] The perception information unit 1034 is configured to take the type information of the early warning target, the real-time relative position information, the real-time speed information, and the real-time heading angle information as the perception information.

[0192] In some embodiments, after the perception information module 1030, the device can further include:

[0193] The target coordinate system construction unit is configured to construct a target coordinate system corresponding to the plurality of images respectively with the target vehicle as a coordinate origin on the plurality of images.

[0194] Correspondingly, the early warning target marking module 1040 can include:

[0195] ​The coordinate information determination unit is configured to determine coordinate information of the early warning target in a target coordinate system corresponding to the multiple frames of images according to the real-time relative position information.

[0196] The early warning target marking unit is configured to mark the early warning target in the multiple frames of images according to the type information of the early warning target and the coordinate information, to obtain the real-time early warning video.

[0197] In the embodiments of the present disclosure, the device can further include:

[0198] The predicted motion trajectory generation unit is configured to generate a predicted motion trajectory of the early warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information.

[0199] The early warning range judgment unit is configured to judge whether the early warning target is within the early warning range of the target vehicle based on the predicted motion trajectory.

[0200] The step execution unit is configured to execute the step of marking the early warning target in the real-time monitoring video based on the perception information to obtain the real-time early warning video when the judgment is positive.

[0201] In a specific embodiment, when the identification result further includes type information of the early warning scene, the device can further include:

[0202] The target display device determination unit is configured to determine a target display device of the early warning video based on the type information of the early warning scene.

[0203] When the target display device includes a vehicle-end display device, the real-time early warning video output module 850 can include:

[0204] The first video output unit is configured to output the real-time early warning video to the vehicle-end display device.

[0205] When the target display device includes a remote control-end display device, the real-time early warning video output module 850 can include:

[0206] The second video output unit is configured to output the real-time early warning video to the remote control-end display device.

[0207] In a specific embodiment, the first video output unit can further include:

[0208] The video encoding unit is configured to encode the real-time early warning video to obtain an encoded real-time early warning video.

[0209] The first video sending unit is configured to send the encoded real-time early warning video to the vehicle-end central controller, so that the vehicle-end central controller decodes the encoded real-time early warning video to obtain the real-time early warning video and outputs the real-time early warning video to the vehicle-end display device.

[0210] In an optional embodiment, the first video sending unit can further include:

[0211] The second video sending unit is configured to send the encoded real-time early warning video to the vehicle-end central controller based on 5G communication.

[0212] The device in the device embodiment and the method embodiment are based on the same inventive concept.

[0213] The vehicle early warning device provided in the embodiments of the present application includes a processor and a memory. The memory stores at least one instruction or at least one program. The at least one instruction or the at least one program is loaded and executed by the processor to implement the vehicle early warning method provided in the above method embodiments.

[0214] The memory can be used to store software programs and modules. The processor executes various functions and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, application programs required for functions, etc. The data storage area can store data created according to the use of the above device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0215] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal, or a similar computing device, i.e., the above computer device can include a mobile terminal, a computer terminal, or a similar computing device.

[0216] The embodiments of the present application also provide a storage medium. The storage medium can be arranged in a server to save at least one instruction or at least one program related to the vehicle early warning method in the method embodiments. The at least one instruction or the at least one program is loaded and executed by the processor to implement the vehicle early warning method provided in the above method embodiments.

[0217] Optionally, in the embodiment, the storage medium can be located in at least one of the plurality of network servers of the computer network. Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media capable of storing program codes.

[0218] It can be seen from the embodiments of the vehicle early warning method, system, device, equipment or storage medium provided by the present application that the technical solutions provided by the present application can intuitively and clearly display the dangerous targets around the vehicle, help the remote driver to discover the dangerous targets in time to avoid the danger, improve the timeliness and accuracy of early warning, and protect the property and personal safety of the driver and passenger. The target display device corresponding to the target early warning personnel among the driver and passenger can be further determined, the real-time early warning video is output to the target display device, and the pertinence and effectiveness of early warning are improved.

[0219] It should be noted that the above-mentioned embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above describes a specific embodiment of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0220] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0221] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing the relevant hardware to complete. The above-mentioned program can be stored in a computer readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk.

[0222] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle pre-warning method characterized by, The method comprises: The vehicle terminal of the target vehicle acquires real-time monitoring videos of multiple vehicle cameras on the target vehicle; The vehicle terminal sends the real-time monitoring videos to a remote driving remote control terminal of the target vehicle; The remote driving remote control terminal performs early warning scene identification on the target vehicle based on the real-time monitoring videos, and obtains an identification result, which comprises type information of an early warning scene and an early warning target affecting safe driving of the target vehicle; The remote driving remote control terminal processes the real-time monitoring videos, and obtains perception information of the early warning target, which comprises real-time relative position information, real-time speed information and real-time heading angle information of the early warning target relative to the target vehicle; The remote driving remote control terminal generates a predicted motion trajectory of the early warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information, acquires real-time driving information of the target vehicle and generates a predicted motion trajectory of the target vehicle based on the real-time driving information, judges whether the early warning target is within an early warning range of the target vehicle based on the predicted motion trajectory of the early warning target and the predicted motion trajectory of the target vehicle, and if the early warning target is within the early warning range of the target vehicle, marks the early warning target in the real-time monitoring videos based on the perception information, and obtains real-time early warning videos; The remote driving remote control terminal determines a target display device of the early warning videos based on the type information of the early warning scene, and outputs the real-time early warning videos to the target display device, which comprises a vehicle-end display device when the target display device comprises the vehicle-end display device, and outputs the real-time early warning videos to a remote driving remote control terminal display device when the target display device comprises the remote driving remote control terminal display device.

2. A vehicle warning method characterized by, The method comprises: Receiving real-time monitoring videos of multiple vehicle cameras on a target vehicle sent by a vehicle terminal of the target vehicle; Performing early warning scene identification on the target vehicle based on the real-time monitoring videos, and obtaining an identification result, which comprises type information of an early warning scene and an early warning target affecting safe driving of the target vehicle; Processing the real-time monitoring videos, and obtaining perception information of the early warning target, which comprises real-time relative position information, real-time speed information and real-time heading angle information of the early warning target relative to the target vehicle; Generating a predicted motion trajectory of the early warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information; acquiring real-time driving information of the target vehicle and generating a predicted motion trajectory of the target vehicle based on the real-time driving information; Judging whether the early warning target is within an early warning range of the target vehicle based on the predicted motion trajectory of the early warning target and the predicted motion trajectory of the target vehicle; If the early warning target is within the early warning range of the target vehicle, marking the early warning target in the real-time monitoring videos based on the perception information, and obtaining real-time early warning videos; determine a target display device of the early warning video based on the type information of the early warning scene; when the target display device comprises a vehicle-end display device, output the real-time early warning video to the vehicle-end display device; when the target display device comprises a remote driving remote control-end display device, output the real-time early warning video to the remote driving remote control-end display device.

3. The method of claim 2, wherein, The receiving of the real-time monitoring video of the plurality of vehicle-mounted cameras on the target vehicle sent by the vehicle terminal of the target vehicle comprises: receiving the encoded real-time monitoring video sent by the vehicle terminal, the encoded real-time monitoring video being obtained by encoding the real-time monitoring video by the vehicle terminal; decoding the encoded real-time monitoring video to obtain the real-time monitoring video.

4. The method of claim 3, wherein, The receiving of the encoded real-time monitoring video sent by the vehicle terminal comprises: receiving the encoded real-time monitoring video sent by the vehicle terminal based on 5G communication.

5. The method of claim 2, wherein, The processing of the real-time monitoring video to obtain the perception information of the early warning target comprises: target detection on the real-time monitoring video to determine type information of the early warning target; target tracking on the real-time monitoring video to determine a plurality of images including the early warning target; motion trajectory analysis on the early warning target based on the plurality of images to determine real-time relative position information, real-time speed information and real-time heading angle information of the early warning target relative to the target vehicle; the type information of the early warning target, the real-time relative position information, the real-time speed information and the real-time heading angle information of the early warning target as the perception information.

6. The method of claim 5, wherein, After the processing of the real-time monitoring video to obtain the perception information of the early warning target, the method further comprises: respectively constructing target coordinate systems corresponding to the plurality of images on the plurality of images with the target vehicle as the coordinate origin; The marking of the early warning target in the real-time monitoring video based on the perception information to obtain a real-time early warning video comprises: determining coordinate information of the early warning target in the target coordinate systems corresponding to the plurality of images according to the real-time relative position information; performing early warning target marking in the plurality of images according to the type information of the early warning target and the coordinate information to obtain the real-time early warning video.

7. The method of claim 2, wherein, The output of the real-time early warning video to the vehicle-end display device comprises: encoding the real-time early warning video to obtain an encoded real-time early warning video; sending the encoded real-time early warning video to the vehicle terminal, so that the vehicle terminal decodes the encoded real-time early warning video to obtain the real-time early warning video, and outputs the real-time early warning video to the vehicle-end display device.

8. The method of claim 7, wherein, The sending of the encoded real-time early warning video to the vehicle terminal comprises: sending the encoded real-time early warning video to the vehicle terminal based on 5G communication.

9. A vehicle warning system, characterized by The system comprises a vehicle terminal of a target vehicle and a remote driving remote control-end of the target vehicle; wherein, The vehicle terminal is configured to acquire real-time monitoring videos of a plurality of vehicle cameras on the target vehicle; and send the real-time monitoring videos to the remote driving remote control terminal. The remote driving remote control terminal is configured to receive the real-time monitoring videos sent by the vehicle terminal; perform pre-warning scene identification on the target vehicle based on the real-time monitoring videos to obtain an identification result, wherein the identification result includes type information of a pre-warning scene and a pre-warning target affecting safe driving of the target vehicle; process the real-time monitoring videos to obtain perception information of the pre-warning target, wherein the perception information includes real-time relative position information, real-time speed information and real-time heading angle information of the pre-warning target relative to the target vehicle; generate a predicted motion trajectory of the pre-warning target based on the real-time relative position information, the real-time speed information and the real-time heading angle information, acquire real-time driving information of the target vehicle and generate a predicted motion trajectory of the target vehicle based on the real-time driving information, judge whether the pre-warning target is within a pre-warning range of the target vehicle based on the predicted motion trajectory of the pre-warning target and the predicted motion trajectory of the target vehicle, if the pre-warning target is within the pre-warning range of the target vehicle, mark the pre-warning target in the real-time monitoring videos based on the perception information to obtain a real-time pre-warning video; determine a target display device of the pre-warning video based on the type information of the pre-warning scene, and output the real-time pre-warning video to the target display device, wherein the target display device includes a vehicle-end display device or a remote driving remote control terminal display device.

10. The system of claim 9, wherein, The vehicle terminal is further configured to encode the real-time monitoring videos to obtain encoded real-time monitoring videos. The vehicle terminal is further configured to send the encoded real-time monitoring videos to the remote driving remote control terminal. The remote driving remote control terminal is further configured to receive the encoded real-time monitoring videos sent by the vehicle terminal, and decode the encoded real-time monitoring videos to obtain the real-time monitoring videos.

11. The system of claim 10, wherein, The vehicle terminal is further configured to send the encoded real-time monitoring videos to the remote driving remote control terminal based on 5G communication. The remote driving remote control terminal is further configured to receive the encoded real-time monitoring videos sent by the vehicle terminal based on 5G communication.

12. The system of claim 9, wherein, The remote driving remote control terminal is further configured to perform target detection on the real-time monitoring videos to determine type information of the pre-warning target, and perform target tracking on the real-time monitoring videos to determine a plurality of images including the pre-warning target. The remote driving remote control terminal is further configured to perform motion trajectory analysis on the pre-warning target based on the plurality of images to determine real-time relative position information, real-time speed information and real-time heading angle information of the pre-warning target relative to the target vehicle. The remote driving remote control terminal is further configured to use the type information of the pre-warning target, the real-time relative position information, the real-time speed information and the real-time heading angle information as the perception information.

13. The system of claim 12, wherein, The remote driving remote control terminal is further configured to construct a target coordinate system corresponding to the multiple frames of images with the target vehicle as a coordinate origin point for each of the multiple frames of images; determine coordinate information of the early warning target in the target coordinate system corresponding to the multiple frames of images according to the real-time relative position information; According to the type information and the coordinate information of the early warning target, mark the early warning target in the multiple frames of images to obtain the real-time early warning video.

14. The system of claim 9, wherein, The remote driving remote control terminal is further configured to encode the real-time early warning video to obtain an encoded real-time early warning video, and transmit the encoded real-time early warning video to the vehicle-mounted terminal; The vehicle-mounted terminal is further configured to receive the encoded real-time early warning video transmitted by the remote driving remote control terminal.

15. The system of claim 14, wherein, The remote driving remote control terminal is further configured to transmit the encoded real-time early warning video to the vehicle-mounted terminal based on 5G communication; The vehicle-mounted terminal is further configured to receive the encoded real-time early warning video transmitted by the remote driving remote control terminal based on 5G communication.

16. A vehicle early warning device, characterized in that the device comprises: a real-time monitoring video receiving module configured to receive real-time monitoring videos of multiple vehicle-mounted cameras of a target vehicle transmitted by a vehicle-mounted terminal of the target vehicle; an early warning scene identification module configured to identify an early warning scene of the target vehicle based on the real-time monitoring videos to obtain an identification result, the identification result comprising type information of the early warning scene and an early warning target affecting safe driving of the target vehicle; a perception information module configured to process the real-time monitoring videos to obtain perception information of the early warning target, the perception information comprising real-time relative position information, real-time speed information, and real-time heading angle information of the early warning target relative to the target vehicle; a predicted motion trajectory generation unit configured to generate a predicted motion trajectory of the early warning target based on the real-time relative position information, the real-time speed information, and the real-time heading angle information, and generate a predicted motion trajectory of the target vehicle based on real-time driving information of the target vehicle; an early warning range judgment unit configured to judge whether the early warning target is within an early warning range of the target vehicle based on the predicted motion trajectory of the early warning target and the predicted motion trajectory of the target vehicle; an early warning target marking module configured to mark the early warning target in the real-time monitoring videos based on the perception information if the early warning target is within the early warning range of the target vehicle to obtain a real-time early warning video; a target display device determination unit configured to determine a target display device of the early warning video based on the type information of the early warning scene; a first video output unit configured to output the real-time early warning video to a vehicle-end display device when the target display device comprises the vehicle-end display device; a second video output unit configured to output the real-time early warning video to a remote driving remote control terminal display device when the target display device comprises the remote driving remote control terminal display device.

17. The apparatus of claim 16, wherein, The real-time monitoring video receiving module comprises: The first video receiving unit is configured to receive the encoded real-time monitoring video sent by the vehicle terminal. The video decoding unit is configured to decode the encoded real-time monitoring video to obtain the real-time monitoring video.

18. The apparatus of claim 17, wherein, The first video receiving unit comprises: The second video receiving unit is configured to receive the encoded real-time monitoring video sent by the vehicle terminal based on 5G communication.

19. The apparatus of claim 16, wherein, The perception information module comprises: The target detection unit is configured to perform target detection on the real-time monitoring video to determine type information of the early warning target. The target tracking unit is configured to perform target tracking on the real-time monitoring video to determine a plurality of images including the early warning target. The motion trajectory analysis unit is configured to perform motion trajectory analysis on the early warning target based on the plurality of images to determine real-time relative position information, real-time speed information and real-time heading angle information of the early warning target relative to the target vehicle. The perception information unit is configured to take the type information of the early warning target, the real-time relative position information, the real-time speed information and the real-time heading angle information as the perception information.

20. The apparatus of claim 19, wherein, The device further comprises: The target coordinate system construction unit is configured to construct a target coordinate system corresponding to the plurality of images respectively with the target vehicle as a coordinate origin on the plurality of images. The early warning target marking module comprises: The coordinate information determination unit is configured to determine coordinate information of the early warning target in the target coordinate system corresponding to the plurality of images according to the real-time relative position information. The early warning target marking unit is configured to perform early warning target marking in the plurality of images according to the type information of the early warning target and the coordinate information to obtain the real-time early warning video.

21. The apparatus of claim 16, wherein, The first video output unit comprises: The video encoding unit is configured to encode the real-time early warning video to obtain the encoded real-time early warning video. The first video sending unit is configured to send the encoded real-time early warning video to the vehicle terminal, so that the vehicle terminal decodes the encoded real-time early warning video to obtain the real-time early warning video and outputs the real-time early warning video to the vehicle end display device.

22. The apparatus of claim 21, wherein, The first video sending unit comprises: The second video sending unit is configured to send the encoded real-time early warning video to the vehicle terminal based on 5G communication.

23. A vehicle warning device, characterized by comprising: The device comprises a processor and a memory, and the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the vehicle early warning method according to any one of claims 2 to 8.

24. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the vehicle early warning method according to any one of claims 2 to 8.

Images