Driving safety monitoring display method and system

By using deep learning and optical flow visualization technology in a remote driving system to detect and label the motion state of objects around the vehicle and generate a motion relationship graph, the problem of limited information displayed in remote driving is solved, thus improving driving safety.

CN117274948BActive Publication Date: 2026-04-28NINGBO LOTUS ROBOTICS CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
NINGBO LOTUS ROBOTICS CO LTD
Filing Date
2023-09-05
Publication Date
2026-04-28

AI Technical Summary

Technical Problem

Existing remote driving video display methods display overly simplistic information and fail to effectively mark the motion status of surrounding objects, resulting in useless warnings and reduced driving safety in low-speed environments.

Method used

A deep learning target detection model and a moving object detection model are used to detect monitored targets around the vehicle. The motion status of the vehicle and the target is displayed on the remote monitoring terminal through optical flow visualization image rules, and a motion relationship graph is generated to mark potential collision risks.

Benefits of technology

It enriches the image content displayed on the remote monitoring terminal, accurately marks the movement status of objects in the surrounding environment, proactively alerts potential collision risks, improves driving safety, and avoids collision accidents caused by misjudgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117274948B_ABST
    Figure CN117274948B_ABST
Patent Text Reader

Abstract

The application relates to a driving safety monitoring display method and system. The method comprises the following steps: when a vehicle is in a driving state, the motion state of a monitoring target of a preset type in a preset monitoring range around the vehicle is detected through a preset deep learning target detection model and a preset moving object detection model; the motion state of the vehicle is marked in an image displayed at a remote monitoring end according to a preset optical flow visualization image rule; a moving relationship graph is generated according to the motion state of the monitoring target and the motion state of the vehicle; and whether the monitoring target has a collision risk for the vehicle is determined through the moving relationship graph. The method can enrich the image content displayed at the remote monitoring end, effectively mark the motion state of the surrounding environment objects, actively and accurately prompt the related information for the driver at the remote monitoring end, further improve the driving safety, and avoid the collision caused by the judgment error of the driver at the remote driving due to not being on the real vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to intelligent vehicle technology, and in particular to a driving safety monitoring and display method and system. Background Technology

[0002] With the development of remote driving technology, augmented reality (AR) technology is being increasingly widely applied to it. Because the remote driver is not in the actual vehicle, issues such as large blind spots, distorted perspectives, and lack of acceleration / deceleration awareness reduce the safety of remotely controlling a vehicle. Adding augmented reality technology, by overlaying driving assistance information onto the images captured by the original onboard cameras, can improve safety.

[0003] However, current remote driving video display methods, devices, and electronic devices, while incorporating augmented reality technology to acquire real-time images of the vehicle's surroundings and dynamically display distance indicators on the remote driving terminal, allowing operators to effectively assess the distance between the vehicle and its environment and improving remote driving safety, suffer from overly simplistic information display, limited to distance indicators. While Gaode Maps' AR real-scene technology, which uses a mobile phone camera to capture images of the road ahead, can identify and render drivable roads and the positions of surrounding vehicles, it fails to recognize vehicle speed and trajectory, generating excessive and useless false vehicle location alerts in low-speed urban scenarios such as parking lots and garages with multiple vehicles. Summary of the Invention

[0004] Therefore, it is necessary to provide a driving safety monitoring and display method and system to address the above-mentioned technical problems. This method and system can enrich the content displayed on the remote monitoring terminal, effectively mark the movement status of objects in the surrounding environment, and proactively and accurately provide relevant information. This will enable operators on the remote driving terminal to effectively monitor the vehicle and its surrounding environment, thereby improving driving safety and preventing collisions.

[0005] One aspect of the present invention provides a method for displaying driving safety monitoring, the method comprising:

[0006] When the vehicle is in motion, the motion state of the preset type of monitoring targets within the preset monitoring range around the vehicle is detected by the preset deep learning target detection model and the preset moving object detection model.

[0007] The motion state of the vehicle is marked in the image displayed on the remote monitoring terminal according to the preset optical flow visualization image rules;

[0008] A motion relationship diagram is generated based on the motion state of the monitored target and the motion state of the vehicle;

[0009] The movement relationship diagram is used to determine whether the monitored target poses a collision risk to the vehicle.

[0010] If there is a risk of collision, a preset risk marker will be added to the monitored target in the image displayed on the remote monitoring terminal.

[0011] Another aspect of the present invention provides a driving safety monitoring and display system, the system comprising:

[0012] The state detection module is used to detect the motion state of preset types of monitoring targets within a preset monitoring range around the vehicle when the vehicle is in motion, using a preset deep learning target detection model and a preset moving object detection model.

[0013] The status marking module is used to mark the motion status of the vehicle in the image displayed on the remote monitoring terminal according to the preset optical flow visualization image rules;

[0014] The motion relationship generation module is used to generate a motion relationship diagram based on the motion state of the monitored target and the motion state of the vehicle.

[0015] The risk prediction module is used to determine whether the monitored target poses a collision risk to the vehicle through the movement relationship diagram;

[0016] The aforementioned driving safety monitoring and display method and system, when the vehicle is in motion, detects the motion state of preset types of monitoring targets within a preset monitoring range around the vehicle using a preset deep learning target detection model and a preset moving object detection model; marks the vehicle's motion state in the image displayed on the remote monitoring terminal according to preset optical flow visualization image rules; and generates a motion relationship diagram based on the motion states of the monitoring targets and the vehicle. This enriches the image content displayed on the remote monitoring terminal and effectively marks the motion states of surrounding environmental objects. Furthermore, the motion relationship diagram determines whether the monitoring target poses a collision risk to the vehicle. If a collision risk exists, a preset risk marker is added to the monitoring target in the image displayed on the remote monitoring terminal, proactively and accurately providing relevant information to the driver at the remote monitoring terminal, further improving driving safety and preventing collision accidents caused by misjudgments by the remote driver who is not physically present in the vehicle. Attached Figure Description

[0017] Figure 1 This is a flowchart of a vehicle safety monitoring and display method according to an embodiment of the present invention;

[0018] Figure 2 application Figure 1 The optical flow visualization marking method is illustrated in the embodiment of the present invention as a schematic diagram of optical flow visualization image rules.

[0019] Figure 3 This is a schematic diagram of a motion relationship image in an embodiment of the present invention, used for... Figure 1 Step S103 generates a motion relationship diagram based on the motion state of the monitored target and the motion state of the vehicle;

[0020] Figure 4 This is a schematic diagram illustrating the prediction of the location of pedestrian 1 at the next preset time point in an embodiment of the present invention, used for... Figure 1 Step S104 assesses and predicts the collision risk of each monitored target using a motion relationship graph;

[0021] Figure 5 This is a schematic diagram of a risk marking method according to an embodiment of the present invention, used for... Figure 1 Step S105: Add a corresponding preset tag to each of the monitored targets on the remote monitoring terminal;

[0022] Figure 6 This is a schematic diagram of a vehicle safety monitoring and display system provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] See Figure 1 This invention provides a method for displaying driving safety monitoring, which can be executed by a driving safety monitoring system provided in this invention. The driving safety monitoring system can be implemented using software and / or hardware. The driving safety monitoring display method includes the following steps:

[0025] Step S101: When the vehicle is in motion, the motion state of a preset type of monitoring target within a preset monitoring range around the vehicle is detected by a preset deep learning target detection model and a preset moving object detection model.

[0026] Understandably, when a vehicle is in motion, real-time images of its surroundings can be acquired by installing cameras or other imaging devices on the vehicle. This embodiment of the invention does not limit the placement of the cameras or other imaging devices. After acquiring the real-time images of the vehicle's surroundings, the motion state of preset types of monitored targets within a preset monitoring range around the vehicle is detected using a preset deep learning target detection model and a preset moving object detection model.

[0027] The preset monitoring targets can be pre-set according to actual applications, such as pedestrians, cars, trucks, motorcycles, bicycles, etc. These objects are all objects that the driver needs to pay attention to when the vehicle is in motion. The preset deep learning object detection model is pre-set and is a YOLO image object detection model trained on collected road condition data images. YOLO (You Only Look Once) is a real-time object detection model that transforms the object detection task into a regression problem, predicting the object category and bounding box in a single network. YOLO has high real-time performance and accuracy. The preset deep learning object detection model can identify and detect preset monitoring targets within a preset monitoring range around the vehicle and mark the position and size of the monitoring targets in the form of rectangular boxes. The preset moving object detection model is obtained using the FlowNet optical flow method, combining the state information, position, and size of the detected monitoring targets with the optical flow image of the monitoring targets obtained through the optical flow method to generate the moving object detection model. The moving object detection model can statistically determine the movement direction and velocity of all pixels within the bounding box of the monitored target. The modulo of these movement directions and velocities can be used as the movement velocity information *v* of the monitored target, thus determining its motion state. The calculation formula for the movement direction and velocity of all pixels within the bounding box of the monitored target using the moving object detection model is as follows, where the unit is pixels / frame, representing the number of pixels that can move between consecutive image frames. The calculation formula is:

[0028]

[0029] The moving object detection model can detect the motion state of the monitored target and filter out errors caused by background pixels. FlowNet is a deep learning-based optical flow model used to estimate pixel displacements between video frames. The basic idea of ​​FlowNet is to use a convolutional neural network to learn motion vectors between pixels and directly output the optical flow field from the input image pairs through supervised learning. FlowNet employs a two-stream network structure, where the two streams represent two input images. Features are extracted through convolutional layers with shared weights and used to predict optical flow. The training process of FlowNet typically uses the ground truth labels of the optical flow field to supervise network training, and uses the error between the ground truth and predicted values ​​of the optical flow field to optimize network parameters.

[0030] Step S102: Mark the motion state of the vehicle in the image displayed on the remote monitoring terminal according to the preset optical flow visualization image rules.

[0031] Understandably, color-coded visualization in optical flow visualization rules encodes the displacement vector of each pixel in the optical flow field as a color. Common color mapping methods include HSV and Rainbow. Generally, the displacement direction is represented by the hue (H) of the color, and the displacement magnitude is represented by the saturation (S) or luminance (V) of the color. Color-coded visualization can help to intuitively display the direction and speed of pixel movement, and can distinguish movement in different directions during visualization. Based on this idea, optical flow visualization rules can be preset, and the movement state of the vehicle can be marked in the image displayed on the remote monitoring terminal.

[0032] In one embodiment, preset optical flow visualization rules can be set, such as using different color hues to represent the vehicle's direction of travel and using color brightness to represent the speed. After calculating the vehicle's direction of travel and speed, according to the preset optical flow visualization rules, the vehicle's travel area is marked with corresponding colors at the remote monitoring end, and the color intensity is marked according to the vehicle's speed. See also Figure 2 , Figure 2 This is a schematic diagram of an optical flow visualization image rule according to an embodiment of the present invention. Figure 2 application Figure 1 The optical flow visualization marking method described in step S102 sets four directions in the diagram: top left, top right, bottom left, and bottom right. Different directions are represented by different colors, and different brightness levels of the same color represent different speeds.

[0033] Step S103: Generate a motion relationship diagram based on the motion state of the monitored target and the motion state of the vehicle.

[0034] A motion relationship diagram is generated based on the motion states of the monitored target and the vehicle. The motion relationship diagram uses preset optical flow colors and arrows to represent the position, direction, and speed of the monitored target, as well as the vehicle's travel area and speed direction. The preset optical flow colors and arrows are pre-set and customized according to the arrow visualization concept in optical flow visualization. Arrow visualization in optical flow visualization is a common method used to display the displacement vector of each pixel in the calculated optical flow field. This visualization method represents the displacement of each pixel as an arrow, where the starting point of the arrow represents the pixel's position, and the direction and length of the arrow represent the direction and magnitude of the pixel's displacement. In optical flow visualization, arrow visualization is often used to visually represent the motion direction and speed of pixels. The direction of the arrow shows the motion direction of the pixel in the optical flow field. The length of the arrow indicates the pixel's speed; a long arrow indicates fast motion, and a short arrow indicates slow motion or stillness. See also... Figure 3 , Figure 3 This is a schematic diagram of a motion relationship image in an embodiment of the present invention, used for... Figure 1Step S103 generates a motion relationship diagram based on the motion state of the monitored target and the motion state of the vehicle. The diagram detects three pedestrians and one moving vehicle, with their corresponding movement directions and speeds indicated by light flow colors and arrows. Pedestrian 1 is marked orange with a corresponding arrow, pedestrian 2 is marked blue with a corresponding arrow, pedestrian 3 is marked pink with a corresponding arrow, vehicle 1 is marked yellow with a corresponding arrow, and the vehicle's travel area corresponds to purple with a corresponding arrow.

[0035] Step S104: Determine whether the monitored target poses a collision risk to the vehicle through the movement relationship diagram.

[0036] The collision risk of each monitored target is assessed using a movement relationship graph, and the assessment method is as follows:

[0037] Based on the vehicle's gear information: D / R gear determines the focus. In D gear, the focus is on assessing the risk of a frontal collision; conversely, in R gear, the focus is on assessing the risk of a rear collision.

[0038] According to the movement relationship diagram, if the location of the monitored target overlaps with the vehicle's driving area at the current time point, it is considered to be at high collision risk.

[0039] If the location of the monitored target overlaps with the vehicle's driving area at the next time point, it is considered a medium collision risk.

[0040] The method for predicting the position of the monitored target at the next time point is as follows: The position of the next time point is obtained by adding the movement direction and velocity to the coordinates of the rectangular frame of the Nth image of the monitored target, while the size of the rectangular frame remains unchanged. (See also...) Figure 4 , Figure 4 For Figure 1 Step S104 determines whether the monitored target poses a collision risk to the vehicle through the movement relationship diagram. This is a schematic diagram of predicting the location of pedestrian 1 at the next preset time point in this embodiment of the invention. In the diagram, the coordinates of the rectangle where pedestrian 1 is currently located are added to the direction of movement and the speed as the predicted location of pedestrian 1 in the next image. The size dimension of the rectangle remains unchanged, and the rectangle filled with dots is the predicted location.

[0041] The remaining monitored targets that do not overlap with the vehicle's driving area at the current time point and the next time point are considered to have low collision risk or no collision risk.

[0042] Step S105: Add a corresponding preset tag to each of the monitored targets on the remote monitoring terminal.

[0043] If the monitored target poses a collision risk, a preset risk marker is added to the image displayed on the remote monitoring terminal. In one embodiment, the preset risk marker can be pre-set; for example, if the monitored target is assessed as having a high collision risk, it is marked with "High Risk" in red font in the center of the corresponding rectangular frame of the monitored target in the image; if the monitored target is assessed as having a medium collision risk, it is marked with "Medium Risk" in yellow font in the center of the corresponding rectangular frame of the monitored target in the image.

[0044] If there is no collision risk, a preset non-risk marker is added to the monitored target in the image displayed on the remote monitoring terminal. In one embodiment, the preset non-risk marker can also be pre-set; for example, if the monitored target is assessed as having low collision risk or no collision risk, it is marked with "Low Risk" in green font in the center of the corresponding rectangular frame of the monitored target in the image. See also Figure 5 , Figure 5 This is a schematic diagram of a risk marking method according to an embodiment of the present invention, used for... Figure 1 Step S105: Add a corresponding preset label to each of the monitored targets on the remote monitoring terminal. Based on the evaluation of the image, pedestrian 1 is predicted to have a high collision risk, so it is marked with "High Risk" in red in the center of the pedestrian 1 rectangle. Based on the evaluation of the image, vehicle 1 is predicted to have a medium collision risk, so it is marked with "Medium Risk" in yellow in the center of the vehicle 1 rectangle. Pedestrian 2 and pedestrian 3 are predicted to have a low collision risk, so they are marked with "Low Risk" in green in the center of their rectangles.

[0045] The aforementioned driving safety monitoring and display method, when the vehicle is in motion, detects the motion state of preset types of monitoring targets within a preset monitoring range around the vehicle using a preset deep learning target detection model and a preset moving object detection model; marks the vehicle's motion state in the image displayed on the remote monitoring terminal according to preset optical flow visualization image rules; and generates a motion relationship diagram based on the motion states of the monitoring targets and the vehicle. This enriches the image content displayed on the remote monitoring terminal and effectively marks the motion states of surrounding environmental objects. Furthermore, the motion relationship diagram determines whether the monitoring target poses a collision risk to the vehicle. If a collision risk exists, a preset risk marker is added to the monitoring target in the image displayed on the remote monitoring terminal, proactively and accurately providing relevant information to the driver at the remote monitoring terminal, further improving driving safety and preventing collision accidents caused by misjudgments by the remote driver who is not physically present in the vehicle.

[0046] It should be noted that the embodiments of the present invention can be used for remotely controlled unmanned vehicles, displaying video and prompting images on the remote driving terminal. In some other embodiments, it can also be applied to vehicles operated by a driver, displaying video and prompting images on the vehicle terminal. Furthermore, the embodiments of the present invention do not limit the order of steps S101 and S105.

[0047] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0048] In one embodiment, such as Figure 6 As shown, a schematic diagram of a driving safety monitoring and display system is provided, including: a status detection module 610, a status marking module 620, a movement relationship generation module 630, and a risk prediction module 640, wherein:

[0049] The state detection module 610 is used to detect the motion state of a preset type of monitoring target within a preset monitoring range around the vehicle when the vehicle is in motion, by using a preset deep learning target detection model and a preset moving object detection model.

[0050] The status marking module 620 is used to mark the motion status of the vehicle in the image displayed on the remote monitoring terminal according to the preset optical flow visualization image rules;

[0051] The motion relationship generation module 630 is used to generate a motion relationship diagram based on the motion state of the monitored target and the motion state of the vehicle.

[0052] The risk prediction module 640 is used to determine whether the monitored target poses a collision risk to the vehicle through the movement relationship diagram.

[0053] In one embodiment, the state detection module 610 further includes:

[0054] The target detection submodule is used to detect a preset type of monitoring target within a preset monitoring range around the vehicle when the vehicle is in motion.

[0055] A motion state detection submodule is used to detect the motion state of a preset type of monitoring target within a preset monitoring range around the vehicle when the vehicle is in motion.

[0056] In one embodiment, the status marking module 620 further includes:

[0057] The color setting module is used to set different marking colors for different types of motion states according to preset optical flow visualization image marking rules;

[0058] The color marking module is used to mark the movement area of ​​the vehicle in the image displayed by remote monitoring according to the vehicle's movement status.

[0059] In one embodiment, the risk prediction module 640 further includes:

[0060] The risk marking submodule is used to add a preset risk mark to the monitored target in the image within the monitoring range of the remote monitoring terminal if there is a collision risk.

[0061] Specific limitations regarding the driving safety monitoring and display system can be found in the limitations of the driving safety monitoring and display method described above, and will not be repeated here. Each module in the aforementioned driving safety monitoring and display system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0062] The driving safety monitoring system in this embodiment detects the movement of preset types of monitored targets within a preset monitoring range around the vehicle when the vehicle is in motion using the state detection module; marks the vehicle's movement state using the state marking module; and generates a movement relationship diagram using the movement relationship generation module, enriching the image content displayed on the remote monitoring terminal and effectively marking the movement states of surrounding environmental objects. Furthermore, the risk prediction module can determine whether the monitored target poses a collision risk to the vehicle using the movement relationship diagram. If a collision risk exists, a preset risk marker is added to the monitored target in the image displayed on the remote monitoring terminal, proactively and accurately providing relevant information to the driver on the remote monitoring terminal, further improving driving safety and preventing collisions caused by misjudgments by the remote driver who is not physically present in the vehicle.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A driving safety monitoring and display method, applied to the remote monitoring terminal of a vehicle autonomous driving system, characterized in that, The monitoring and display method includes the following steps: When the vehicle is in motion, the motion state of a preset type of monitoring target within a preset monitoring range around the vehicle is detected by a preset deep learning target detection model and a preset moving object detection model. The moving object detection model is generated by combining the detected state information, position and size of the monitoring target with the optical flow image of the monitoring target obtained by optical flow method. The motion state of the vehicle is marked in the image displayed on the remote monitoring terminal according to the preset optical flow visualization image rules; A motion relationship diagram is generated based on the motion state of the monitored target and the motion state of the vehicle. The motion relationship diagram uses preset optical flow colors and arrows to represent the position, direction, and speed of the monitored target and the driving area and speed direction of the vehicle. The movement relationship diagram is used to determine whether the monitored target poses a collision risk to the vehicle. If there is a risk of collision, a preset risk marker will be added to the monitored target in the image displayed on the remote monitoring terminal.

2. The method according to claim 1, characterized in that, The process of detecting the motion state of a preset type of monitored target within a preset monitoring range around the vehicle when the vehicle is in motion, using a preset deep learning target detection model and a preset moving object detection model, includes the following steps: The state information of the monitored target is determined according to the preset deep learning target detection model; The moving object detection model is generated by combining the state information of the monitored target with the optical flow image of the monitored target obtained by the optical flow method. The motion state of the monitored target is determined based on the moving object detection model.

3. The method according to claim 1, characterized in that, The step of marking the vehicle's motion state according to preset optical flow visualization image rules includes: According to the preset optical flow visualization image marking rules, different marking colors are set for different types of motion states; Based on the vehicle's motion status, the movement area of ​​the vehicle is marked with a corresponding color in the image displayed on the remote monitoring system.

4. The method according to claim 1, characterized in that, Determining whether the monitored target poses a collision risk to the vehicle through the motion relationship diagram includes: Based on the motion relationship diagram, determine and assess whether the movement areas of the monitored target and the vehicle overlap. If overlap occurs, it is determined that there is a risk of collision between the monitored target and the vehicle.

5. The method according to claim 1, characterized in that, The method further includes: If there is no risk of collision, a preset non-risk marker is added to the monitored target in the image displayed on the remote monitoring terminal.

6. A driving safety monitoring and display system, characterized in that, It includes a status detection module, a status marking module, a mobility relationship generation module, and a risk prediction module; among which, The state detection module is used to detect the motion state of a preset type of monitoring target within a preset monitoring range around the vehicle when the vehicle is in motion, by using a preset deep learning target detection model and a preset moving object detection model. The moving object detection model is generated by combining the detected state information, position and size of the monitoring target with the optical flow image of the monitoring target obtained by optical flow method. The status marking module is used to mark the motion status of the vehicle in the image displayed on the remote monitoring terminal according to the preset optical flow visualization image rules; The motion relationship generation module is used to generate a motion relationship diagram based on the motion state of the monitored target and the motion state of the vehicle. The motion relationship diagram uses preset optical flow colors and arrows to represent the position, direction, and speed of the monitored target and the driving area and speed direction of the vehicle, respectively. The risk prediction module is used to determine whether the monitored target poses a collision risk to the vehicle through the movement relationship diagram.

7. The system as described in claim 6, characterized in that, The state detection module includes a target detection submodule, which is used to detect a preset type of monitoring target within a preset monitoring range around the vehicle when the vehicle is in motion.

8. The system as described in claim 6, characterized in that, The state detection module further includes a motion state detection submodule, which is used to detect the motion state of a preset type of monitoring target within a preset monitoring range around the vehicle when the vehicle is in motion.

9. The system as described in claim 6, characterized in that, The status marking module includes: The color setting module is used to set different marking colors for different types of motion states according to preset optical flow visualization image marking rules; The color marking module is used to mark the movement area of ​​the vehicle in the image displayed by remote monitoring according to the vehicle's movement status.

10. The system as described in claim 6, characterized in that, The risk prediction module includes a risk marking submodule, which is used to add a preset risk mark to the monitored target in the image within the monitoring range of the remote monitoring terminal if there is a collision risk.

Citation Information

Patent Citations

  • Lane auxiliary method for controlling steering torque based on vehicle-mounted blind area visual scene analysis

    CN110775057A

  • Intelligent driving vehicle testing method and device and equipment

    CN111795832A