A data processing method and system of a vehicle-mounted monitoring system

Abstract

The application discloses a kind of data processing method and system of vehicle monitoring system, the vehicle monitoring system includes multiple vehicle recorders and multiple edge computing terminals;One of the vehicle recorder can communicate with multiple edge computing terminals interaction;The edge computing terminal is used to realize the data processing method.The data processing method includes the following steps: obtaining the video data to be detected uploaded by vehicle recorder;From the video data to be detected, a first target object is identified, and a detection frame of the first target object is generated;Wherein, the first target object is motor vehicle;Using multi-target tracking algorithm to track the first target object;From the video data to be detected, lane information is extracted;Based on the lane information, the first target object is detected for illegal driving behavior.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing information, in particular to a data processing method and system of a vehicle-mounted monitoring system. BACKGROUND

[0002] At present, the supervision of traffic violations is mainly monitored by setting up illegal photographing cameras on road sections. However, it is impossible to cover a large area by relying on fixed-point cameras for illegal photographing and human supervision.

[0003] Therefore, the present application provides a vehicle-mounted monitoring system which can increase the supervision of traffic and save costs, and is convenient to maintain, to detect traffic violations. SUMMARY

[0004] In order to overcome the above technical defects, the present application provides a data processing method and system of a vehicle-mounted monitoring system.

[0005] In order to solve the above problems, the present application is implemented according to the following technical scheme:

[0006] In a first aspect, the present application provides a data processing method of a vehicle-mounted monitoring system, wherein the vehicle-mounted monitoring system comprises a plurality of vehicle event data recorders and a plurality of edge computing terminals; one vehicle event data recorder can communicate and interact with a plurality of edge computing terminals; the edge computing terminal is used to implement the data processing method, and the data processing method comprises the following steps:

[0007] Obtaining video data uploaded by the vehicle event data recorder for detection;

[0008] Identifying a first target object from the video data for detection, and generating a detection box of the first target object; wherein the first target object is a motor vehicle;

[0009] Tracking the first target object by using a multi-target tracking algorithm;

[0010] Extracting lane information from the video data for detection;

[0011] Detecting the first target object for irregular driving behavior based on the lane information.

[0012] Preferably, the detection of the first target object for irregular driving behavior comprises a motor vehicle real line lane changing irregularity detection step:

[0013] Generating a left fitting point and a right fitting point based on the detection box of the first target object; the left fitting point and the right fitting point are used to represent the position of the first target object in the video data;

[0014] The left fitting point and the right fitting point of the first target object and the lane information are used to determine whether the first target object has a rule violation of changing lanes with a solid line.

[0015] Preferably, the left fitting point is a left lower corner coordinate of the detection frame, and the right fitting point is a right lower corner coordinate of the detection frame.

[0016] The calculation formula of the left fitting point is:

[0017] The calculation formula of the left fitting point is:

[0018] wherein (cx, cy) represents a center point coordinate of the detection frame, (w, h) represents a width and a height of the detection frame, (x1, y1) represents a coordinate of the left fitting point, and (x2, y2) represents a coordinate of the right fitting point.

[0019] Preferably, the detection of the rule violation of driving of the first target object specifically includes a motor vehicle reverse driving detection step.

[0020] The right lane in the lane information is identified.

[0021] The first target object in the detection frame located in the right lane is identified.

[0022] The vehicle head of the first target object is identified to determine whether the first target object has a rule violation of reverse driving.

[0023] Preferably, the data processing method further includes:

[0024] When it is detected that the first target object has a rule violation of driving;

[0025] The license plate number information of the first target object is identified.

[0026] The to-be-detected video data, the rule violation detection result, and the license plate number information are sent to a target device.

[0027] Preferably, the data processing method further includes:

[0028] The first target object and the second target object in the to-be-detected video data are identified to respectively generate a first target detection frame and a second target detection frame; wherein the first target object is a motor vehicle, and the second target object is a non-motor vehicle.

[0029] A multi-target tracking algorithm is used to track the first target object and the second target object.

[0030] A trajectory prediction model is used to predict trajectories of the first target object and the second target object to obtain a first predicted trajectory and a second predicted trajectory.

[0031] determine whether there is an accident conflict point between the plurality of first predicted trajectories, between the first predicted trajectory and a second predicted trajectory;

[0032] generate a road accident warning information when there is an accident conflict point;

[0033] based on the road accident warning information, instruct a corresponding dashcam to shoot potential accident video data;

[0034] send the road accident warning information, the to-be-detected video data and the potential accident video data to a target device.

[0035] In a second aspect, the present application provides a data processing system of a vehicle-mounted monitoring system, the vehicle-mounted monitoring system comprising a plurality of dashcams and a plurality of edge computing terminals; one of the dashcams is capable of communicating and interacting with a plurality of edge computing terminals; the data processing system is arranged in the edge computing terminal, and the data processing system comprises:

[0036] an acquisition module for acquiring to-be-detected video data uploaded by a dashcam;

[0037] an identification module for identifying a first target object from the to-be-detected video data and generating a detection box of the first target object; wherein the first target object is a motor vehicle;

[0038] a tracking module for tracking the first target object using a multi-target tracking algorithm;

[0039] an extraction module for extracting lane information from the to-be-detected video data;

[0040] a detection module for detecting a violation driving behavior of the first target object based on the lane information.

[0041] Preferably, the detection module detects the violation driving behavior of the first target object, specifically including a motor vehicle solid line lane changing violation detection step:

[0042] generating a left fitting point and a right fitting point based on the detection box of the first target object; the left fitting point and the right fitting point are used to represent the position of the first target object in the video data;

[0043] judging whether the first target object has a solid line lane changing violation behavior based on the left fitting point and the right fitting point of the first target object and the lane information.

[0044] Preferably, the left fitting point is the lower left corner coordinate of the detection box, and the right fitting point is the lower right corner coordinate of the detection box;

[0045] the calculation formula of the left fitting point is:

[0046] The calculation formula of the left fitting point is:

[0047] Wherein, (cx, cy) represents the center point coordinates of the detection frame, (w, h) represents the width and height of the detection frame, (x1, y1) represents the coordinates of the left fitting point, and (x2, y2) represents the coordinates of the right fitting point.

[0048] Compared with the prior art, the beneficial effects of the present application are:

[0049] The present application provides a kind of data processing method of vehicle monitoring system, the vehicle monitoring system includes multiple vehicle recorders and multiple edge computing terminals;One of the vehicle recorders can communicate with multiple edge computing terminals;The edge computing terminal is used to realize the data processing method.The data processing method includes the following steps: obtaining the video data to be detected uploaded by vehicle recorder;From the video data to be detected, a first target object is identified, and a detection frame of the first target object is generated;Wherein, the first target object is motor vehicle;Multi-target tracking algorithm is used to track the first target object;From the video data to be detected, lane information is extracted;Based on the lane information, the first target object is detected for illegal driving behavior.

[0050] The vehicle monitoring system of the present application, the vehicle recorder as an important component, is responsible for capturing video data during vehicle driving, and these video data are then uploaded to the edge computing terminal for further processing.The present application realizes a vehicle violation detection system for vehicle recorder video.

[0051] The present application solves the limitation of traditional violation capture system that is fixed in place and can only capture the illegal driving information of vehicles in the fixed place provided with violation capture system.The present application overcomes the problems existing in traditional monitoring mode, such as damaging highway buried coil and weak anti-interference, reduces the cost of monitoring system, and improves the monitoring efficiency.The present application not only improves the flexibility and accuracy of violation detection, but also reduces the monitoring cost, improves road safety, and has important practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0052] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings, in which:

[0053] Figure 1 is a perspective view of a mounting bracket of the present application;

[0054] Figure 2 is an assembly view of a mounting bracket of the present application.

[0055] Figure 3 is a cross-sectional view of a second clamping arm of the present application;

[0056] Figure 4 is a flowchart of a data processing method of the present application;

[0057] Figure 5 is a schematic diagram of vehicle reverse driving recognition of the present application;

[0058] in the figure:

[0059] 10 - connecting piece;

[0060] 20 - first clamping arm, 21 - clamping jaw;

[0061] 30 - second clamping arm, 31 - counterweight;

[0062] 40 - rubber part;

[0063] 50 - placement plate, 51 - flexible pad, 52 - plastic base. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present application will be described below with reference to the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0065] At present, the supervision of traffic violations is mainly carried out by setting up illegal photographing cameras on road sections for monitoring, and it is impossible to cover a large area by relying only on fixed-point cameras for illegal photographing and human supervision.

[0066] Referring to Figure 4 , the present application provides a flowchart of a data processing method of a vehicle monitoring system. The vehicle monitoring system of the present application, as an important component, is responsible for capturing video data during vehicle driving, and the video data is then uploaded to an edge computing terminal for further processing. The present application realizes a vehicle violation detection system for the vehicle recorders.

[0067] In the present application, the vehicle monitoring system includes a plurality of vehicle recorders and a plurality of edge computing terminals; one of the vehicle recorders can communicate and interact with a plurality of edge computing terminals; and the edge computing terminal is used to realize the data processing method.

[0068] It can be understood that the edge computing terminal is arranged in the road, such as the control box of the intelligent street lamp. The vehicle can communicate with the edge computing terminal through the vehicle. For example, the vehicle networking V2X platform based on the TMEC (Tencent Edge Computing Platform) allows the vehicle to directly communicate with the roadside wireless base station, and realizes the data interaction of the vehicle with the edge computing terminal through the vehicle.

[0069] The data processing method of the application can be executed by a data processing system which can be realized in the form of hardware and / or software, and the system can be configured in an electronic device, such as an edge computing terminal in a vehicle monitoring system. Figure 4 As shown in the figure, the data processing method comprises:

[0070] S100: acquiring video data uploaded by a vehicle event data recorder for detection.

[0071] S200: identifying a first target object from the video data for detection, and generating a detection frame of the first target object; wherein the first target object is a motor vehicle.

[0072] S300: tracking the first target object by using a multi-target tracking algorithm.

[0073] S400: extracting lane information from the video data for detection.

[0074] S500: detecting a violation driving behavior of the first target object based on the lane information.

[0075] The application solves the limitation of the traditional violation capture system which is fixed in place and can only capture the violation driving information of the vehicle in the fixed place provided with the violation capture system. The application overcomes the problems existing in the traditional monitoring mode, such as damaging the buried coil of the road and weak anti-interference, reduces the cost of the monitoring system, and improves the monitoring efficiency. The application not only improves the flexibility and accuracy of the violation detection, but also reduces the monitoring cost, improves the road safety, and has important practical application value. Specifically, the steps of the data processing method are described in detail.

[0076] S100: acquiring video data uploaded by a vehicle event data recorder for detection.

[0077] In a specific implementation, the controller integrated with the vehicle event data recorder is used for collecting and uploading the video data. Specifically, the vehicle event data recorder transmits the video raw data to the vehicle terminal system through the vehicle WiFi, and the vehicle terminal system and the edge computing terminal perform data transmission and interaction. The video data is transmitted to the edge computing terminal for real-time processing.

[0078] S200: Identify the first target object from the video data to be detected, and generate a detection box of the first target object; wherein the first target object is a motor vehicle.

[0079] In this step, different types of traffic participants can be accurately identified and distinguished, providing accurate target information for subsequent tracking and trajectory prediction.

[0080] In specific implementation, deep learning algorithms such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector) can be used for real-time analysis of video streams to identify non-motor vehicles and motor vehicles. After the algorithm detects target objects, it will mark the location of the target objects in the video frame and generate a bounding box to identify these objects. The system will classify the detected target objects into non-motor vehicles (first target objects) and motor vehicles (second target objects) according to the preset classification criteria.

[0081] In specific implementation, the target detection model is Yolov5s model, which has identified the first target object and the second target object. In this invention, the first object and the second object are motor vehicles and non-motor vehicles respectively.

[0082] S300: Use a multi-target tracking algorithm to track the first target object.

[0083] In this step, the multi-target tracking algorithm can provide the motion information of the target object over time, which is crucial for understanding the dynamics of traffic flow and the trajectory of vehicle travel.

[0084] In specific implementation, suitable multi-target tracking algorithms such as DeepSORT or MOT (Multi-Object Tracking) algorithm are selected, which can handle problems such as target occlusion, intersection and reappearance. The algorithm will track each target object, even if the target temporarily leaves the field of view or reappears after being occluded, it can maintain the continuity of tracking. The system will update the position and motion state of each target object in real time, providing data support for trajectory prediction.

[0085] In one specific implementation, during the target detection process, non-motor vehicles and motor vehicles in each frame are detected. In order to unify the information of the same target, an unchanged ID needs to be assigned to the same target in different frames. DeepSORT multi-target tracking algorithm is used, which adds DeepAssociationMetric and appearance image information based on the original Sort algorithm target tracking to solve the tracking of occluded targets for a long time, which is a tracking framework based on deep appearance feature model and motion information model.

[0086] S400: Extract lane information from the video data to be detected.

[0087] In a specific implementation, the lane information includes the number of lanes, the type of lane lines (solid, dashed), and the specific implementation is as follows:

[0088] S410: Edge detection and preprocessing: using image processing techniques such as grayscale, Gaussian blur, and Canny edge detection to extract the edge information of the lane lines.

[0089] By performing grayscale and Gaussian blur processing on the video frame, and then applying the Canny edge detection algorithm, the edges of the lane lines are extracted.

[0090] S420: Region of Interest (ROI) mask: by defining a Region of Interest (ROI) mask, only the part of the image containing the lane lines is focused on, and other irrelevant areas are ignored.

[0091] In this step, a ROI mask is defined, and only the area below the image is focused on, because the lane lines are usually located in the lower half of the road.

[0092] S430: Straight line detection: using Hough Transform to detect straight lines in the image, and the straight lines represent the lane lines.

[0093] In a specific implementation, Hough Transform is applied to detect straight lines in the image, and according to the length and continuity of the straight lines, the straight lines are classified as solid or dashed. For example, if a straight line is long and continuous, it is classified as a solid line; if the straight line is short and intermittent, it is classified as a dashed line

[0094] S440: Lane line classification: according to the length and continuity of the straight lines, the detected straight lines are classified as solid or dashed. Long and continuous straight lines are identified as solid lines, while short and intermittent straight lines are identified as dashed lines.

[0095] S450: Lane number and position estimation: by using a clustering algorithm, specifically a clustering algorithm based on discrete Frechet distance, the lane trajectories are segmented into lane trajectory segments, and the road trajectory segment clusters are generated. The least squares estimation algorithm is used to fit the road trajectory segment clusters, and the estimated road feature parameters are extracted. Based on the expectation maximization algorithm and the mixture Gaussian model, the number of lanes and the corresponding target lane information of each lane are determined.

[0096] S500: Based on the lane information, detect the illegal driving behavior of the first target object.

[0097] In a specific implementation, in combination with the lane information and the tracking data of the target object, it is determined whether there is a violation by analyzing whether the vehicle crosses the lane line and the like.

[0098] In a specific implementation of the present application, the detection of the first target object performing a violation driving behavior specifically includes a motor vehicle solid line lane change violation detection step:

[0099] S511: Based on the detection frame of the first target object, a left fitting point and a right fitting point are generated; the left fitting point and the right fitting point are used to represent the position of the first target object in the video data.

[0100] Specifically, the left fitting point is the lower left corner coordinate of the detection frame, and the right fitting point is the lower right corner coordinate of the detection frame.

[0101] In a specific implementation, for each detected motor vehicle, the lower left corner coordinate of its detection frame is extracted as the left fitting point, and the lower right corner coordinate is extracted as the right fitting point. These coordinates are used to represent the position of the motor vehicle in the video data.

[0102] In a specific implementation, the calculation formula of the left fitting point is:

[0103] The calculation formula of the left fitting point is:

[0104] Where (cx, cy) represents the center point coordinate of the detection frame, (w, h) represents the width and height of the detection frame, (x1, y1) represents the coordinate of the left fitting point, and (x2, y2) represents the coordinate of the right fitting point.

[0105] S512: Based on the left fitting point and the right fitting point of the first target object, and the lane information, it is determined whether the first target object has a solid line lane change violation behavior.

[0106] In a specific implementation, the left fitting point and the right fitting point of the motor vehicle are compared with the detected lane line. If the detection frame of the motor vehicle moves from one side lane line to another side lane line in consecutive video frames and crosses the solid line, it can be determined as a solid line lane change.

[0107] Further, by analyzing the position change of the motor vehicle in the video frame, in combination with the position of the lane line, it is determined whether the vehicle crosses the solid line. If the left fitting point or the right fitting point of the vehicle crosses the solid line, it can be regarded as a violation of the solid line lane change.

[0108] It can be understood that, due to the particularity of the vehicle and the accuracy of the illegal judgment, it is not feasible to directly use the detection frame of the detection network for the vehicle target in the video, because the detection frame is always larger than the area of the car itself in each frame of image, and for illegal judgment, some judgments are made according to whether the wheels of the car are pressed or crossed the line. Taking the example of the car changing lanes with solid lines, because the detection frame of the car has a certain height on the picture, simply from the picture, it will cause the car body to have exceeded the lane line, but the wheels of the car have not pressed the line, if the boundary of the detection frame or the center point of the detection frame is simply used for judgment, the result will be very inaccurate and easy to misjudge.

[0109] For such a situation, through analysis of the data, it is found that the position of the wheels is the best for judging the vehicle, but it is difficult to realize the detection of the wheels. Since it is judged on a two-dimensional picture, the present application uses two fitted points, one on the left and one on the right, to represent the position of a car, which are the lower left corner and the lower right corner of the detection frame respectively, to reduce the occurrence of misjudgment as much as possible.

[0110] Through the above-mentioned technology, the vehicle-mounted monitoring system can effectively detect and judge whether the motor vehicle has the illegal behavior of changing lanes with solid lines, and improve the efficiency of road safety and traffic management.

[0111] In a preferred implementation, the detection of the illegal driving behavior of the first target object specifically includes a motor vehicle reverse driving detection step:

[0112] S521: identifying a right lane in the lane information.

[0113] In a specific implementation, the right lane line is identified by the following method:

[0114] S5211: image preprocessing: converting the video frame into a grayscale image to reduce the computational complexity and focus on the brightness information. This is because the lane line is easier to be detected in the grayscale image.

[0115] S5212: edge detection: applying the Canny edge detection algorithm to identify the edges in the image. The Canny algorithm is a popular edge detection method that can reduce noise and preserve edge information.

[0116] S5213: lane line detection: using Hough Transform to detect straight lines in the image. Hough Transform can detect straight line segments from the image, even if these straight line segments are disturbed by noise or partially occluded.

[0117] S5214: Lane line classification: Differentiate left lane and right lane according to the slope of the detected straight line. Generally, the slope of the left lane is negative, while the slope of the right lane is positive. By calculating the slope of each straight line, they can be classified as left lane or right lane.

[0118] In a specific implementation, straight lines with a slope greater than 0 are to be identified. In the image, these straight lines are usually located on the right side of the image, and their x coordinates will increase as the image moves down. For the identified right lane line, the exact position of the lane line can be determined by a fitting algorithm such as least squares.

[0119] S522: Identify the first target object whose detection box is located in the right lane.

[0120] S523: Identify the vehicle head of the first target object to determine whether the first target object has a reverse driving violation.

[0121] It can be understood that when the first target object is identified to have a vehicle head, it can be determined that the first target object has a reverse driving violation. In a specific implementation, based on the AI vision intelligent analysis algorithm, the system can identify the vehicle head or tail of the target vehicle. When identified as a vehicle head, it is determined that there is a reverse driving behavior.

[0122] In a preferred implementation, the data processing method further comprises:

[0123] S600: When detecting that the first target object has a violation driving behavior, identifying the license plate number information of the first target object.

[0124] In a specific implementation, the PyTesseract OCR (Optical Character Recognition) tool is used to recognize the text on the license plate, which can convert the text in the image into a machine-readable text format. Before performing OCR, the license plate image needs to be preprocessed, including denoising, binarization, morphological operation, etc., to improve the accuracy of recognition.

[0125] S700: Send the to-be-detected video data, violation detection result and license plate number information to the target device.

[0126] In a specific implementation, the detailed information of the violation event is sent to the monitoring center of the traffic management department in real time, so as to take corresponding law enforcement actions. The violation detection result and the license plate number information are integrated with the original video data to form a complete violation event record. The detailed information of the violation event is sent to the monitoring center of the traffic management department in real time, so as to take corresponding law enforcement actions.

[0127] In a preferred implementation, the data processing method further comprises:

[0128] S1000: Identify the first target object and the second target object in the video data to be detected, and generate corresponding first target detection frame and second target detection frame respectively; wherein the first target object is a motor vehicle, and the second target object is a non-motor vehicle.

[0129] In this step, different types of traffic participants can be accurately identified and distinguished, providing accurate target information for subsequent tracking and trajectory prediction.

[0130] In specific implementation, deep learning algorithms such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector) can be used for real-time analysis of video streams to identify non-motor vehicles and motor vehicles. After detecting target objects, the algorithm will mark the location of the target objects in the video frame and generate a bounding box to identify these objects. The system will classify the detected target objects into non-motor vehicles (second target objects) and motor vehicles (first target objects) according to the preset classification standard.

[0131] S2000: Use multi-target tracking algorithm to track the first target object and the second target object respectively.

[0132] In this step, the multi-target tracking algorithm can provide the motion information of the target objects over time, which is crucial for understanding the dynamics of traffic flow and predicting potential conflicts.

[0133] In specific implementation, suitable multi-target tracking algorithms such as DeepSORT or MOT (Multi-Object Tracking) algorithm are selected, which can handle problems such as target occlusion, intersection and reappearance. The algorithm will track each target object, even if the target temporarily leaves the field of view or reappears after being occluded, it can maintain the continuity of tracking. The system will update the position and motion state of each target object in real time to provide data support for trajectory prediction.

[0134] In one specific implementation, during target detection, pedestrians, non-motor vehicles and motor vehicles in each frame are detected. In order to unify the information of the same target, the same target needs to be assigned an unchanged ID in different frames. DeepSORT multi-target tracking algorithm is used, which adds DeepAssociationMetric and appearance image information to the original Sort algorithm target tracking to solve the tracking of occluded targets over a long period of time. It is a tracking framework based on deep appearance feature model and motion information model.

[0135] DeepAssociationMetric and appearance image information are added to the original Sort algorithm target tracking to solve the tracking of occluded targets over a long period of time. It is a tracking framework based on deep appearance feature model and motion information model.

[0136] S3000: The trajectory prediction model is used to predict the trajectories of the first target object and the second target object respectively, and the first predicted trajectory and the second predicted trajectory are obtained.

[0137] In a specific implementation, the future position and speed of each target object are predicted according to historical trajectory data by using a machine learning or deep learning method, such as a recurrent neural network (RNN) or a long short-term memory network (LSTM).

[0138] In a specific implementation, the future position and speed of each target object are predicted according to historical trajectory data by using an LSTM (Long Short-Term Memory) network prediction model. The position and speed data obtained by the tracking algorithm are input into the trajectory prediction model. The model will output the future trajectory of each target object, i.e., the first predicted trajectory (motor vehicle) and the second predicted trajectory (non-motor vehicle).

[0139] In a specific implementation, the trajectory prediction model uses a Kalman filter, and an extended Kalman filter predictor can also be used.

[0140] In a specific implementation, when a target object is detected, the trajectory of the object is predicted by a Kalman filter, and then the Hungarian algorithm is used to combine and match the trajectory data of the predicted frame with the trajectory detections of the current frame, and then the Kalman filter is updated.

[0141] Specifically, the DeepSORT algorithm takes the intersection ratio of the target detection box obtained by the YOLO algorithm and the previously predicted target tracking box as a combination input into the Hungarian algorithm, and uses linear assignment to associate the IDs of each frame. In addition, the appearance and position information of the target can be added in the process of inter-frame matching calculation to realize correct matching in the case that the target object is occluded but then appears again, reduce the frequent jumping of the target ID, and achieve the purpose of continuous tracking.

[0142] S4000: Whether there is an accident conflict point between the plurality of first predicted trajectories or between the first predicted trajectory and the second predicted trajectory is calculated.

[0143] The present application calculates whether the trajectories of the non-motor vehicle and the motor vehicle exist in the intersection or proximity, i.e., the accident conflict point, by analyzing the predicted trajectories. This step can identify potential collision risks and provide a basis for early warning.

[0144] In a specific implementation, taking a road intersection without traffic signals as an example, in the dynamic interaction process between electric bicycles and non-motor vehicles, electric bicycles and motor vehicles, etc., collision problems may occur. By predicting whether there is an accident conflict point between the trajectories. Specifically, the predicted trajectory of the target object can be matched to the background of the image data (the calibration points along the ground in the image are marked, and a predicted trajectory line is formed), and then it is judged whether there is an overlapping intersection point between the predicted trajectories to determine whether there is an accident conflict point.

[0145] S5000: When there is an accident conflict point, generate road accident warning information.

[0146] In a specific implementation, once the accident conflict point is detected, the system will trigger the warning logic to generate warning information. The warning information includes the location, time, and target object type involved in the conflict point.

[0147] In a specific implementation, the road accident warning information includes the accident conflict level, the location information of the road, and the positioning information of the dashcam. Specifically, the accident conflict level can be calibrated according to the conflict relationship between the target objects, such as electric bicycles and non-motor vehicles being a first-level accident conflict level; electric bicycles and motor vehicles being a second-level accident conflict level.

[0148] S6000: Based on the road accident warning information, instruct the corresponding dashcam to shoot potential accident video data.

[0149] In a specific implementation, according to the warning information, the dashcam is instructed to record the video data of the potential accident scene for subsequent analysis and evidence preservation.

[0150] S7000: Send the road accident warning information, the to-be-detected video data, and the potential accident video data to the target device.

[0151] In a specific implementation, through wireless communication technology such as 4G / 5G network, the warning information and related video data are transmitted in real time to the target device, such as the smart phone of the vehicle owner's family, or the monitoring system of the traffic management center, etc. This step ensures that the warning information can be timely conveyed to the personnel who need to take action, improving the response and timely rescue of the accident.

[0152] Through S1000-S7000 of the present application, by monitoring and analyzing the road situation in real time, potential traffic accident risks can be discovered in time and preventive measures can be taken. Through the prediction model and multi-target tracking algorithm, the system can predict and identify possible accident conflict points, and issue warnings in advance to provide enough time for drivers or traffic management departments to take risk avoidance measures.

[0153] The application further provides a data processing system of a vehicle-mounted monitoring system, the vehicle-mounted monitoring system comprising a plurality of vehicle recorders and a plurality of edge computing terminals; one vehicle recorder is capable of communicating with a plurality of edge computing terminals; the data processing system is arranged on the edge computing terminal, and the data processing system comprises:

[0154] an acquisition module configured to acquire video data uploaded by the vehicle recorder and to be detected;

[0155] an identification module configured to identify a first target object from the video data to be detected and to generate a detection frame of the first target object; wherein the first target object is a motor vehicle;

[0156] a tracking module configured to track the first target object by using a multi-target tracking algorithm;

[0157] an extraction module configured to extract lane information from the video data to be detected;

[0158] a detection module configured to detect a violation driving behavior of the first target object based on the lane information.

[0159] Preferably, the detection of the violation driving behavior of the first target object specifically comprises a motor vehicle real line lane changing violation detection step:

[0160] based on the detection frame of the first target object, left fitting points and right fitting points are generated; the left fitting points and the right fitting points are used to represent the position of the first target object in the video data;

[0161] based on the left fitting points and the right fitting points of the first target object and the lane information, it is determined whether the first target object has a real line lane changing violation behavior.

[0162] Preferably, the left fitting points are the lower left corner coordinates of the detection frame, and the right fitting points are the lower right corner coordinates of the detection frame;

[0163] the calculation formula of the left fitting points is as follows:

[0164] the calculation formula of the right fitting points is as follows:

[0165] wherein (cx, cy) represents the center point coordinates of the detection frame, (w, h) represents the width and height of the detection frame, (x1, y1) represents the coordinates of the left fitting points, and (x2, y2) represents the coordinates of the right fitting points.

[0166] The application further provides a vehicle recorder, which is used to deploy a vehicle-mounted monitoring system to implement the above-mentioned data processing method for a data processing system.

[0167] The dashcam of the present invention includes a housing and a mounting bracket, the housing housing housing a circuit board module such as a camera and a processor.

[0168] like Figure 1 As shown, the product of this invention has superior versatility and practicality, and can be applied to motorcycles, electric bicycles, and automobiles. Specifically, the mounting bracket includes a connector 10, a first clamping arm 20, a second clamping arm 30, and an elastic element. The connector is used to connect a camera; the first clamping arm is connected to the connector; the second clamping arm is hinged to the first clamping arm and is located below the first clamping arm. The second clamping arm has a planar structure; the elastic element is connected to the first and second clamping arms, and the elastic element is used to drive the first and second clamping arms to clamp each other. The mounting bracket can form a clamping mechanism through the first and second clamping arms for clamping onto an external carrier; the mounting bracket can also be stably placed on an external carrier through the planar structure of the second clamping arm.

[0169] In this technology, the connector is the basic part of the mounting bracket for connecting the camera. The elastic element provides elastic force, allowing the first and second clamping arms to clamp together, thereby securing it to the relevant parts of the motorcycle or electric bicycle.

[0170] In one specific embodiment, the connector is hinged to the first clamping arm, and the connector can swing around the hinge point. The connector has a T-shaped structure, and the first clamping arm is provided with a mounting groove structure adapted to the connector.

[0171] This patent utilizes a first clamping arm and a second clamping arm to form a clamping mechanism. Under the action of an elastic element, the clamping mechanism can stably clamp onto gripper parts such as the handlebars of motorcycles and electric bicycles, thus meeting the needs of motorcycle / electric bicycle use. To adapt to use in family cars, the planar structure of the second clamping arm allows it to be stably placed on the car's interior, such as on the passenger-side dashboard.

[0172] In one specific implementation, the first clamping arm, the second clamping arm, and the connector are all plastic parts formed by plastic injection molding.

[0173] The first and second clamping arms are both relatively flat. The first clamping arm has two bent grippers 21, which are spaced apart and their ends extend towards the second clamping arm. The bent grippers form a claw-like clamping mechanism, which can accommodate more external carriers and improve the practicality of the mounting bracket.

[0174] In a preferred embodiment, the end of the clamping jaw is provided with a first rubber piece; the end of the second clamping arm adjacent to the clamping jaw is provided with a second rubber piece; wherein the first rubber piece and the second rubber piece are both rubber blocks with an elliptical cross section.

[0175] Through this design, the rubber piece with an elliptical cross section can provide better friction when compressed, which helps the clamping jaw to hold the motorcycle or electric bicycle more firmly on the structure, preventing it from slipping during use. On the other hand, the rubber piece can adapt to different contact surface shapes and maintain good contact.

[0176] In a specific embodiment of the present patent, the elastic member is a torsion spring. It can be understood that the first clamping arm and the second clamping arm of the present application are like a clamp, providing a stable clamping force through the torsion spring, so that the first clamping arm and the second clamping arm can achieve the effect of clamping. Specifically, the use of a torsion spring, a first clamping arm and a second clamping arm to form a clamping mechanism is a common knowledge in the art and can be easily realized by those skilled in the art, and therefore will not be described here.

[0177] In a preferred embodiment, the second clamping arm includes a plastic shell and a plurality of counterweights arranged in the plastic shell.

[0178] Among them, the counterweight increases the weight of the mounting bracket itself, mainly used to stabilize the center of gravity of the mounting bracket, increase the weight of the mounting bracket itself, to ensure the stability when placed on the automotive interior, can keep stable.

[0179] In a preferred embodiment, the mounting bracket further includes a placement plate 50, the placement plate is provided with a plurality of magnets, and the placement plate is used to be placed on an external carrier; wherein the counterweight of the second clamping arm is an iron block, and the second clamping arm can be magnetically connected with the placement plate through the counterweight.

[0180] Magnetic connection allows the second clamping arm and the placement plate to be quickly and easily installed or removed without complex alignment or fastening operations, improving the convenience of use. The design of the placement plate is to use it in the car, and the placement plate serves as the base of the mounting bracket, providing stable placement. As a base, the placement plate has a larger area to be placed more stably on the automotive interior. Then through the magnetic connection between the second clamping arm and the placement plate, the entire mounting bracket is placed stably.

[0181] In a specific embodiment, the placement plate includes a flexible pad 51 and a plastic base 52 fixedly connected to the flexible pad; wherein the magnets are arranged in the plastic base.

[0182] In the present technology, the flexible pad can adapt to different carrier surfaces, including curved or uneven surfaces, improving applicability. The surface of the flexible pad generally has good friction characteristics, which can increase the friction between the placement plate and the carrier, preventing the camera from sliding or shifting.

[0183] Other structures of the data processing method and system of the vehicle-mounted monitoring system described in the embodiment refer to the prior art.

[0184] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Therefore, any modification, equivalent change and modification of the above embodiment based on the technical essence of the present application, without departing from the technical solution content of the present application, still belongs to the scope of the technical solution of the present application.

Claims

1. A data processing method of a vehicle-mounted monitoring system, characterized by, The vehicle-mounted monitoring system comprises a plurality of vehicle recorders and a plurality of edge computing terminals; one vehicle recorder can communicate with a plurality of edge computing terminals; the edge computing terminal is used to realize the data processing method, and the data processing method comprises the following steps: Obtain the video data uploaded by the vehicle recorder for detection; From the video data for detection, a first target object is identified, and a detection box of the first target object is generated; wherein the first target object is a motor vehicle; A multi-target tracking algorithm is used to track the first target object; Lane information is extracted from the video data for detection; Based on the lane information, the first target object is detected for illegal driving behavior; A deep learning algorithm, YOLO or SSD, is used to analyze the video stream in real time to identify the first target object and the second target object in the video data for detection, and generate corresponding first target detection box and second target detection box respectively; DeepSORT multi-target tracking algorithm is used to track the first target object and the second target object respectively, which adds DeepAssociationMetric and appearance image information to the original Sort algorithm to solve the tracking of the occluded target for a long time; The Kalman filter is used for trajectory prediction, and when the target object is detected, the Kalman filter is used to predict the trajectory of the object, and then the Hungarian algorithm is used to combine and match the trajectory data of the predicted frame with the trajectory detections of the current frame, and then the Kalman filter is updated. The interaction ratio of the target detection box obtained by the YOLO algorithm and the target tracking box predicted previously is input into the Hungarian algorithm as a combination, and linear assignment is used to associate the IDs of each frame. In addition, the appearance and position information of the target can be added in the process of inter-frame matching calculation to realize correct matching when the target object is occluded but appears again, reduce the frequent jumping of the target ID, and achieve the purpose of continuous tracking. Identify the first target object and the second target object in the video data for detection, and generate corresponding first target detection box and second target detection box respectively; wherein the first target object is a motor vehicle, and the second target object is a non-motor vehicle; A multi-target tracking algorithm is used to track the first target object and the second target object respectively; A trajectory prediction model is used to predict the trajectories of the first target object and the second target object respectively to obtain first predicted trajectory and second predicted trajectory; Calculate whether there is an accident conflict point between a plurality of first predicted trajectories and between the first predicted trajectory and the second predicted trajectory; When there is an accident conflict point, generate road accident warning information; Based on the road accident warning information, instruct the corresponding vehicle recorder to shoot potential accident video data; Send the road accident warning information, the video data for detection and the potential accident video data to the target device; Detect the illegal driving behavior of the first target object, specifically including the motor vehicle real line lane changing violation detection step: generate a left fitting point and a right fitting point based on the bounding box of the first target object; the left fitting point and the right fitting point are used to represent the position of the first target object in the video data; compare the left fitting point and the right fitting point of the motor vehicle with the detected lane line based on the left fitting point and the right fitting point of the first target object and the lane information; if the bounding box of the motor vehicle moves from one lane line to another lane line and crosses a solid line in consecutive video frames, it can be judged that the solid line changes lane; detect the first target object for a violation driving behavior, specifically including a motor vehicle reverse driving detection step: identify a right lane in the lane information; identify the first target object in the right lane of the bounding box; identify the head of the first target object to determine whether the first target object has a reverse violation behavior.

2. The data processing method of the vehicle-mounted monitoring system according to claim 1, characterized in that: the left fitting point is the left lower corner coordinate of the bounding box, and the right fitting point is the right lower corner coordinate of the bounding box; The calculation formula of the left fitting point is: The calculation formula of the left fitting point is: wherein (cx, cy) represents the center point coordinate of the bounding box, (w, h) represents the width and height of the bounding box, (x1, y1) represents the coordinate of the left fitting point, and (x2, y2) represents the coordinate of the right fitting point.

3. The data processing method of the vehicle monitoring system according to claim 1, wherein, The data processing method further comprises: when detecting that the first target object has a violation driving behavior; identify the license plate number information of the first target object; send the to-be-detected video data, the violation detection result, and the license plate number information to a target device.

4. A data processing system of a vehicle monitoring system, characterized by The vehicle-mounted monitoring system comprises a plurality of vehicle recorders and a plurality of edge computing terminals; one vehicle recorder can communicate and interact with a plurality of edge computing terminals; the data processing system is arranged in the edge computing terminal, and the data processing system comprises: an acquisition module for acquiring to-be-detected video data uploaded by a vehicle recorder; an identification module for identifying a first target object from the to-be-detected video data and generating a bounding box of the first target object; wherein the first target object is a motor vehicle; a tracking module for tracking the first target object using a multi-target tracking algorithm; an extraction module for extracting lane information from the to-be-detected video data; a detection module for detecting a violation driving behavior of the first target object based on the lane information; using a deep learning algorithm, YOLO or SSD, to perform real-time analysis on a video stream to identify a first target object and a second target object in the to-be-detected video data and generate corresponding first target bounding boxes and second target bounding boxes respectively; use the DeepSORT multi-target tracking algorithm to track the first target object and the second target object respectively; the algorithm adds DeepAssociationMetric and appearance image information to the original Sort algorithm to solve the tracking of occluded targets for a long time. The trajectory prediction model adopts a Kalman filter. When a target object is detected, the trajectory of the object is predicted by the Kalman filter, the trajectory data of the predicted frame is matched with the trajectory detections of the current frame by using the Hungarian algorithm, then the Kalman filter is updated, and the interaction ratio of the target detection frame obtained by the YOLO algorithm and the target tracking frame obtained by the previous prediction is input into the Hungarian algorithm as combination, linear distribution is used to associate the IDs of the frames, the appearance and position information of the target is added in the inter-frame matching calculation process, so that the correct matching of the target object is realized in the case that the target object is blocked and then appears again, the frequent jumping of the target ID is reduced, and the purpose of continuous tracking is achieved. A first target object and a second target object in to-be-detected video data are identified, and corresponding first and second target detection boxes are generated respectively; wherein the first target object is a motor vehicle, and the second target object is a non-motor vehicle; A multi-target tracking algorithm is used to track the first target object and the second target object respectively; A trajectory prediction model is used to predict the trajectories of the first target object and the second target object respectively, and first and second predicted trajectories are obtained; Whether there is an accident conflict point between the first predicted trajectories and between the first and second predicted trajectories is calculated; When there is an accident conflict point, road accident warning information is generated; Based on the road accident warning information, a corresponding drive recorder is instructed to shoot potential accident video data; The road accident warning information, the to-be-detected video data, and the potential accident video data are sent to a target device; The detection module detects the first target object for a violation driving behavior, specifically including a motor vehicle real line lane changing violation detection step: Based on the detection box of the first target object, left and right fitting points are generated; the left and right fitting points are used to represent the position of the first target object in the video data; Based on the left and right fitting points of the first target object and the lane information, whether the first target object has a real line lane changing violation is judged; The detection module detects the first target object for a violation driving behavior, specifically including a motor vehicle reverse driving detection step: The right lane in the lane information is identified; The first target object in the detection box located in the right lane is identified; The vehicle head of the first target object is identified to judge whether the first target object has a reverse driving violation.

5. The data processing system of the vehicle-mounted monitoring system according to claim 4, characterized in that: The left fitting point is the lower left corner coordinate of the detection box, and the right fitting point is the lower right corner coordinate of the detection box; The calculation formula of the left fitting point is: The calculation formula of the left fitting point is: wherein (cx, cy) represents the center point coordinate of the detection box, (w, h) represents the width and height of the detection box, (x1, y1) represents the coordinate of the left fitting point, and (x2, y2) represents the coordinate of the right fitting point.

Images