A Vehicle Intelligent Driving Control Method and System for Traffic Scene Analysis

Abstract

The present invention relates to the technical field of vehicle intelligent control, and particularly to a vehicle intelligent driving control method and system for traffic scene analysis. The method includes the following steps: performing traffic scene monitoring and analysis processing based on in-vehicle sensors of the vehicle to generate traffic scene monitoring data; performing visual object behavior analysis and trend accuracy optimization processing based on the traffic scene monitoring data to generate optimized visual object behavior data; obtaining historical vehicle driving control data; designing an intelligent analysis engine for visual object behavior and vehicle driving control based on the historical vehicle driving control data to generate a vehicle intelligent driving control engine; performing vehicle intelligent driving control analysis on the optimized visual object behavior data through the vehicle intelligent driving control engine to generate intelligent vehicle driving control parameters; and performing intelligent vehicle driving control operations for the traffic scene based on the intelligent vehicle driving control parameters. The present invention improves the response level of vehicle intelligent driving control.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle intelligent control, and particularly to a vehicle intelligent driving control method and system for traffic scene analysis. Background Art

[0002] With the development of autonomous driving technology, intelligent vehicles and related driving assistance systems have gradually become the main direction of future traffic development. Through in-vehicle sensors, radars, cameras and other devices, intelligent vehicles can obtain real-time traffic environment information around them, so as to realize the perception and analysis of traffic scenes. Based on this information, the vehicle can make a series of intelligent decisions such as path planning, obstacle avoidance, and lane keeping. However, autonomous driving technology still faces many technical challenges. Especially in complex traffic scenarios, how to accurately and quickly perceive and understand the dynamic behaviors in traffic scenes and perform safe and reliable vehicle driving control has become the key bottleneck in the current technological development. However, when the existing vehicle intelligent driving control methods for traffic scene analysis handle complex scenarios, the reaction speed and decision accuracy of the system are limited. Especially in traffic scenes with multiple targets and multiple dynamic objects, it is difficult to efficiently and accurately predict and optimize the behavior trends of traffic objects, making the vehicle prone to delays or misjudgments when facing emergencies or complex driving environments, unable to ensure the real-time nature and safety of driving decisions, and often relying on historical experience data, but lacking in-depth correlation analysis of historical data and the current scene, and lacking corresponding adjustment mechanisms when dealing with atypical scenes, thus affecting the intelligent level and adaptability of vehicle control. Summary of the Invention

[0003] Based on this, the present invention provides a vehicle intelligent driving control method for traffic scene analysis to solve at least one of the above technical problems.

[0004] To achieve the above object, a vehicle intelligent driving control method for traffic scene analysis includes the following steps:

[0005] Step S1: Based on the in-vehicle sensors of the vehicle, traffic scene monitoring and analysis processing is performed to generate traffic scene monitoring data, where the traffic scene monitoring data includes traffic scene visual data, traffic scene radar data, and traffic scene V2X data;

[0006] Step S2: Traffic scene visual object analysis is performed according to the traffic scene visual data to generate traffic scene visual object data;

[0007] Step S3: Visual object behavior analysis is performed on the traffic scene visual object data according to the traffic scene radar data to generate visual object behavior data;

[0008] Step S4: Optimize the visual object behavior trajectory trend of the visual object behavior data based on the traffic scene visual data and the traffic scene V2X data to generate optimized visual object behavior data;

[0009] Step S5: Obtain historical vehicle drive control data; Design an intelligent analysis engine for visual object behavior and vehicle drive control based on the historical vehicle drive control data to generate a vehicle intelligent drive control engine;

[0010] Step S6: Perform an optimized visual object behavior monitoring and updating operation for the vehicle drive traffic scene according to the optimized visual object behavior data; And collect and update the optimized visual object behavior data according to the optimized visual object behavior monitoring and updating operation of the vehicle drive traffic scene; Transmit the updated optimized visual object behavior data to the vehicle intelligent drive control engine for vehicle intelligent drive control analysis to generate intelligent vehicle drive control parameters; Perform an intelligent vehicle drive control operation for the traffic scene based on the intelligent vehicle drive control parameters.

[0011] Further, step S1 includes the following steps:

[0012] Perform preliminary traffic scene monitoring based on the vehicle's on-vehicle sensors to generate preliminary traffic scene monitoring data;

[0013] Perform internal and external parameter calibration processing for traffic scene monitoring according to the vehicle's on-vehicle sensors to generate internal and external parameter calibration data; Design a coordinate transformation matrix for traffic scene monitoring based on the internal and external parameter calibration data;

[0014] Perform spatial calibration processing for traffic scene monitoring on the preliminary traffic scene monitoring data based on the coordinate transformation matrix for traffic scene monitoring to generate traffic scene monitoring data.

[0015] Further, step S2 includes the following steps:

[0016] Step S21: Perform gradient edge detection processing on the traffic scene visual data to generate traffic scene visual gradient edge data;

[0017] Step S22: Perform traffic scene visual area analysis processing according to the traffic scene visual gradient edge data to generate traffic scene visual area data;

[0018] Step S23: Perform candidate object area evaluation processing on the traffic scene visual data based on the traffic scene visual area data to generate candidate object area evaluation data;

[0019] Step S24: Perform visual object attribute analysis processing according to the candidate object area evaluation data to generate visual object attribute data;

[0020] Step S25: Perform visual object identification tracking processing on the candidate object area evaluation data to generate visual object identification tracking data;

[0021] Step S26: Based on the visual object attribute data and the visual object identification tracking data, perform traffic scene visual object analysis to generate traffic scene visual object data.

[0022] Further, step S23 includes the following steps:

[0023] Step S231: Extract region multi-scale visual features from the traffic scene visual data according to the traffic scene visual region data to generate region multi-scale visual feature data;

[0024] Step S232: Perform sliding window traversal processing on the region multi-scale visual feature data to generate sliding window traversal visual feature data;

[0025] Step S233: Perform candidate object overlap analysis processing according to the sliding window traversal visual feature data to generate candidate object overlap data;

[0026] Step S234: Perform candidate object area evaluation processing according to the candidate object overlap data to generate candidate object area evaluation data.

[0027] Further, step S3 includes the following steps:

[0028] Step S31: Analyze the lidar point cloud features according to the traffic scene lidar data to generate lidar point cloud feature data; Based on the lidar point cloud feature data, perform visual object positioning processing on the traffic scene visual object data to generate visual object positioning data;

[0029] Step S32: Analyze the millimeter-wave radar point cloud features according to the traffic scene radar data to generate millimeter-wave radar point cloud feature data; Based on the millimeter-wave radar point cloud feature data, perform visual object trajectory modeling processing on the traffic scene visual object data to generate visual object trajectory model data;

[0030] Step S33: Perform visual object behavior modeling processing according to the visual object positioning data and the visual object trajectory model data to generate a visual object behavior model;

[0031] Step S34: Perform visual object behavior analysis on the traffic scene visual object data according to the visual object behavior model to generate visual object behavior data.

[0032] Further, step S4 includes the following steps:

[0033] Step S41: Analyze the visual dynamic description features of the traffic scene based on the visual data of the traffic scene to generate traffic scene visual dynamic description feature data;

[0034] Step S42: Analyze the visual data of the trajectory trend of the visual object based on the traffic scene visual dynamic description feature data and the visual object identification tracking data corresponding to the traffic scene visual object data to generate visual object trajectory trend visual data;

[0035] Step S43: Collect the traffic scene visual object communication data for the traffic scene visual object data according to the traffic scene V2X data to generate traffic scene visual object communication data;

[0036] Step S44: Perform optimization processing on the visual object behavior data based on the visual object trajectory trend visual data and the traffic scene visual object communication data to generate optimized visual object behavior data, where the optimized visual object behavior data includes visual object real-time behavior data and visual object predicted behavior data.

[0037] Further, step S5 includes the following steps:

[0038] Step S51: Design a preferred priority vehicle driving decision based on the visual object real-time behavior data;

[0039] Step S52: Design an alternative priority vehicle driving decision based on the visual object predicted behavior data;

[0040] Step S53: Design the decision mapping relationship between the visual object behavior and the vehicle driving control based on the decision tree algorithm and the preferred priority vehicle driving decision to generate an initial vehicle driving control decision tree model;

[0041] Step S54: Perform adjustment processing on the vehicle driving control decision nodes of the initial vehicle driving control decision tree model for the evolution of the visual object warning behavior according to the alternative priority vehicle driving decision to generate an adjusted vehicle driving control decision tree model;

[0042] Step S55: Obtain the historical vehicle driving control sample set;

[0043] Step S56: Perform model decision update training processing on the adjusted vehicle driving control decision tree model based on the historical vehicle driving control sample set to generate a vehicle driving control decision tree model;

[0044] Step S57: Design an intelligent analysis engine for vehicle intelligent driving control according to the vehicle driving control decision tree model to generate a vehicle intelligent driving control engine.

[0045] Further, step S52 includes the following steps:

[0046] Step S521: Extract visual object anomaly-related behavior features based on the predicted behavior data of the visual object to generate visual object anomaly-related behavior feature data;

[0047] Step S522: Perform visual object anomaly behavior-related probability analysis based on the visual object anomaly-related behavior feature data to generate visual object anomaly behavior-related probability data;

[0048] Step S523: Screen the warning anomaly behavior-related probability data in the visual object anomaly behavior-related probability data based on a preset anomaly behavior warning threshold, and design an alternative priority vehicle drive decision according to the warning anomaly behavior-related probability data.

[0049] Furthermore, the step of transmitting the updated and optimized visual object behavior data to the vehicle intelligent drive control engine for vehicle intelligent drive control analysis in Step S6 includes the following steps:

[0050] Transmit the updated and optimized visual object behavior data to the vehicle intelligent drive control engine: Perform vehicle drive control behavior state analysis on the updated and optimized visual object behavior data through the preferred priority vehicle drive decision of the vehicle intelligent drive control engine to generate vehicle drive control behavior state data; Perform vehicle drive control behavior state inference analysis on the updated and optimized visual object behavior data through the alternative priority vehicle drive decision of the vehicle intelligent drive control engine to generate vehicle drive control behavior state inference data;

[0051] Perform vehicle drive control change state analysis on the vehicle drive control behavior state inference data based on a preset traffic scene limit change condition to generate vehicle drive control change state data;

[0052] Perform vehicle intelligent drive control analysis according to the vehicle drive control change state data and the vehicle drive control behavior state data to generate intelligent vehicle drive control parameters.

[0053] This specification provides a vehicle intelligent drive control system for traffic scene analysis, which is used to execute the vehicle intelligent drive control method for traffic scene analysis as described above. The vehicle intelligent drive control system for traffic scene analysis includes:

[0054] A traffic scene monitoring module, which is used to perform traffic scene monitoring analysis and processing based on the vehicle's on-vehicle sensors to generate traffic scene monitoring data, where the traffic scene monitoring data includes traffic scene visual data, traffic scene radar data, and traffic scene V2X data;

[0055] A traffic scene visual object analysis module, which is used to perform traffic scene visual object analysis according to the traffic scene visual data to generate traffic scene visual object data;

[0056] A visual object behavior analysis module, which is used to perform visual object behavior analysis on traffic scene visual object data according to traffic scene radar data and generate visual object behavior data;

[0057] A visual object behavior trend accuracy optimization module, which is used to perform visual object behavior trajectory trend optimization processing on visual object behavior data based on traffic scene visual data and traffic scene V2X data and generate optimized visual object behavior data;

[0058] A vehicle intelligent drive control engine design module, which is used to obtain historical vehicle drive control data; design an intelligent analysis engine for visual object behavior and vehicle drive control based on historical vehicle drive control data and generate a vehicle intelligent drive control engine;

[0059] An intelligent vehicle drive control analysis module, which is used to execute an optimized visual object behavior monitoring and updating operation for the vehicle drive traffic scene according to the optimized visual object behavior data; collect and update the optimized visual object behavior data according to the optimized visual object behavior monitoring and updating operation for the vehicle drive traffic scene; transmit the updated optimized visual object behavior data to the vehicle intelligent drive control engine for vehicle intelligent drive control analysis and generate intelligent vehicle drive control parameters; execute an intelligent vehicle drive control operation for the traffic scene based on the intelligent vehicle drive control parameters.

[0060] The beneficial effects of this application are as follows. By integrating visual data, radar data, and V2X data, and comprehensively utilizing data sources of different types of sensors, the present invention overcomes the limitations of relying on a single data source and significantly improves the perception accuracy of multiple targets and multiple dynamic objects in complex traffic scenarios. Especially in traffic scenarios, the combination of lidar and millimeter-wave radar makes the detection, tracking, and modeling of object behaviors more accurate, and the system can identify and analyze the behavior trends of target objects faster. Through data fusion, the blind spots of scene perception are reduced, and the real-time performance of decision-making is improved. By modeling and trend prediction of the behavior data of visual objects, combined with the fusion analysis of trajectory modeling and communication data, the accuracy of behavior prediction of visual objects can be effectively optimized. Through the visual optimization of trajectory trends and the assistance of V2X data, the system can not only accurately predict the real-time behaviors of visual objects and anticipate their future behaviors, ensuring that the vehicle has stronger response capabilities in complex dynamic traffic scenarios and reducing the occurrence of delays and misjudgments. Through the design of an intelligent driving control engine, combined with the real-time data and predicted data of visual object behaviors, an intelligent vehicle driving decision tree model is constructed. Based on the decision tree algorithm, this model can dynamically adjust the vehicle's driving control strategy in real time, enabling the vehicle to flexibly respond to various complex environments according to changes in traffic scenarios. By combining the patterns in historical vehicle driving control data with the behavior trends of visual objects in the current scenario, the system can quickly adjust the vehicle's control strategy when facing atypical scenarios, improving the flexibility and adaptability of vehicle control. This intelligent adjustment mechanism enables the vehicle to make more reasonable and safe driving decisions in different traffic scenarios. It enables the engine to have the ability of autonomous learning. By continuously optimizing the decision model and simulating the corresponding vehicle driving control responses in traffic scenarios made by the real driving experience of drivers, the intelligent level of vehicle control is improved, and the emergency response ability of the system is enhanced. The optimized prediction of visual object behavior data and the rapid response of the intelligent driving control engine significantly improve the vehicle's response ability in emergencies and reduce misjudgments and lags in vehicle control decisions. Based on the fusion analysis of multi-source data, the vehicle can anticipate potential risk factors in complex traffic environments in advance and take corresponding measures in a timely manner to ensure the real-time performance and safety of driving decisions. Thus, it improves the perception and analysis accuracy of dynamic targets in traffic scenarios, and through the visual dynamic features and V2X communication data, trend prediction and optimization of the visual object behaviors obtained by lidar analysis are realized, achieving multi-source data fusion, ensuring that the vehicle can quickly respond and make reasonable control in changing traffic scenarios, enhancing the vehicle's adaptability in complex scenarios, and through the combination of historical data and real-time data and applying them to the decision tree model for learning and analysis of vehicle driving control, realizing intelligent vehicle driving control decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the step flow of a vehicle intelligent drive control method for traffic scene analysis according to the present invention;

[0062] Figure 2 is Figure 1 Detailed implementation step flow diagram of step S4 in;

[0063] Figure 3 is Figure 1 Detailed implementation step flow diagram of step S5 in;

[0064] The realization, functional characteristics and advantages of the object of the present invention will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific embodiments

[0065] The technical method of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0066] In addition, the accompanying drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0067] It should be understood that although the terms "first", "second", etc. may be used here to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed related items.

[0068] To achieve the above object, please refer to Figures 1 to 3 , the present invention provides a vehicle intelligent drive control method for traffic scene analysis. In the embodiments of the present invention, please refer to Figure 1 shown, which is a schematic diagram of the step flow of the vehicle intelligent drive control method for traffic scene analysis according to the present invention. The vehicle intelligent drive control method for traffic scene analysis includes the following steps:

[0069] Based on this, the present invention provides a vehicle intelligent driving control method for traffic scene analysis to solve at least one of the above technical problems.

[0070] To achieve the above object, a vehicle intelligent driving control method for traffic scene analysis includes the following steps:

[0071] Step S1: Based on the in-vehicle sensors of the vehicle, traffic scene monitoring and analysis are performed to generate traffic scene monitoring data, where the traffic scene monitoring data includes traffic scene visual data, traffic scene radar data, and traffic scene V2X data;

[0072] In an embodiment of the present invention, a high-definition camera is equipped in the front of the vehicle to collect visual data in the traffic scene, including image information of roads, pedestrians, vehicles, and other objects. The parameters of the camera are, for example: resolution: 1920x1080, frame rate: 30FPS, field of view angle: 120 degrees. Millimeter-wave radars and lidars installed in the front and rear of the vehicle respectively collect the distance, speed, and trajectory information of the objects in the front and rear. Typical parameters of the millimeter-wave radar are, for example: detection distance: 250 meters, detection accuracy: <5 cm; lidar parameters are, for example: point cloud density: 5000 points / second, distance accuracy: 2 cm. The vehicle obtains real-time information of the traffic scene, such as signal light status, traffic congestion ahead, etc. data from surrounding vehicles, infrastructure, etc. through V2X communication equipment. Based on the acquisition results of multi-source data, the vehicle control system generates traffic scene monitoring data including visual data, radar data, and V2X data as the basis for subsequent analysis.

[0073] Step S2: Perform traffic scene visual object analysis according to the traffic scene visual data to generate traffic scene visual object data;

[0074] In an embodiment of the present invention, edge detection processing is performed on the visual data (image) of the traffic scene to extract the contours and edges in the image and generate visual gradient edge data. The Sobel algorithm or Canny algorithm is used for edge detection. Detection parameter examples: Sobel convolution kernel size: 3x3, threshold: 0.1. Region segmentation analysis is performed on the extracted edge data to identify visual objects, such as pedestrians, vehicles, obstacles, etc. Region growing method or a deep learning-based segmentation network (such as Mask R-CNN) is used for region segmentation. The segmented visual regions are evaluated to filter out irrelevant regions. The evaluation criteria are based on object size, shape, color features, and the attributes of the regional objects are identified based on a pre-designed object recognition model. According to the evaluation results and the object attribute recognition results, traffic scene visual object data is generated, including the bounding box, classification label (vehicle, pedestrian, etc.) and the preliminary position of each object.

[0075] Step S3: Perform visual object behavior analysis on the traffic scene visual object data according to the traffic scene radar data to generate visual object behavior data;

[0076] In the embodiment of the present invention, three-dimensional positioning of visual objects is performed on the point cloud data collected by lidar, and combined with the visual object data to generate accurate visual object positioning data. The data of millimeter-wave radar is used for speed and trajectory analysis of moving objects to generate the motion trajectory data of visual objects, and the behavior pattern of the object is determined by combining the positioning data. Based on the positioning and trajectory data of visual objects, a behavior model of visual objects is established. The motion of the object is modeled using Kalman filtering or Bayesian estimation to predict its future behavior trend. Finally, according to the behavior model, visual object behavior data is generated, including the real-time motion state (speed, acceleration) of the object and future behavior prediction data.

[0077] Step S4: Perform visual object behavior trajectory trend optimization processing on the visual object behavior data based on the traffic scene visual data and the traffic scene V2X data to generate optimized visual object behavior data;

[0078] In the embodiment of the present invention, the trajectory of visual objects is optimized by combining the visual object behavior data with the traffic scene visual data. The motion trajectory prediction of the object is optimized by a trajectory smoothing algorithm (such as Savitzky-Golay filtering) or a deep learning model (such as LSTM). Through V2X communication data, dynamic information from other vehicles, traffic signals, etc. is obtained, and combined with the visual object behavior data to accurately predict future behaviors. For example: sudden stop information from the vehicle ahead, data on upcoming signal changes, and traffic flow conditions on the road ahead. Based on the visual trajectory optimization and V2X data, optimized visual object behavior data is generated, including real-time behavior data and predicted behavior data.

[0079] Step S5: Obtain historical vehicle drive control data; perform intelligent analysis engine design for visual object behavior and vehicle drive control based on the historical vehicle drive control data to generate a vehicle intelligent drive control engine;

[0080] In the embodiments of the present invention, historical driving control data is obtained from the vehicle control system, including control behaviors such as vehicle acceleration, braking, and steering, as well as traffic scene characteristics at that time. Based on the real-time behavior data of visual objects in the optimized visual object behavior data, the preferred priority control decision of the vehicle is designed. And a preliminary control decision model is constructed using a decision tree or random forest algorithm and the preferred priority control decision of the vehicle. Based on the predicted behavior data of visual objects in the optimized visual object behavior data, an alternative priority control decision is designed. Considering abnormal behaviors or emergencies, the decision tree model is optimized by combining abnormal behavior probability data to ensure that driving control emergencies can be timely controlled. Using historical vehicle control data, the control decision tree model is trained and optimized to form a vehicle driving control decision model capable of responding to complex scenarios in real time. An intelligent vehicle driving control engine is generated, which can adjust the control behavior of the vehicle in real time according to visual object behavior and traffic scene data.

[0081] Step S6: Perform an optimized visual object behavior monitoring and updating operation on the vehicle driving traffic scene according to the optimized visual object behavior data; collect and update the optimized visual object behavior data according to the optimized visual object behavior monitoring and updating operation of the vehicle driving traffic scene; transmit the updated optimized visual object behavior data to the vehicle intelligent driving control engine for vehicle intelligent driving control analysis to generate intelligent vehicle driving control parameters; perform an intelligent vehicle driving control operation on the traffic scene based on the intelligent vehicle driving control parameters.

[0082] In the embodiments of the present invention, based on in-vehicle cameras, radars, and V2X communication systems, vehicles continuously monitor objects in the traffic scene (such as the vehicle ahead, pedestrians, and traffic lights). The vehicle updates the visual object behavior data of the traffic scene every 100 milliseconds to ensure the real-time nature of the data. The vehicle detects that the speed of the vehicle ahead suddenly slows down and updates its driving trajectory based on visual and radar data. At the same time, a pedestrian starts to cross the road, and the point cloud analysis of the radar data shows that the pedestrian is gradually approaching the lane. Through the V2X communication system, a warning message that the traffic light turns red is obtained, and the system combines this data to re-optimize the visual object behavior prediction data in the current scene. The deceleration behavior of the vehicle ahead and the behavior of the pedestrian entering the lane, together with the change in the traffic light state, are transmitted to the vehicle control system, generating updated and optimized visual object behavior data. The updated visual object behavior data is transmitted to the intelligent drive control engine of the vehicle, and the intelligent engine comprehensively analyzes the current scene. For example, through the analysis of the intelligent engine, the system predicts that the vehicle ahead is about to stop; the engine evaluates the moving speed and trajectory of the pedestrian and anticipates that the pedestrian will enter the driving path in front of the vehicle within 3 seconds; combined with the traffic light signal in the V2X communication system, the intelligent engine determines that the current red light countdown is 4 seconds. Based on the above analysis results, the intelligent control engine generates a set of vehicle control parameters, such as deceleration: the system determines that it is necessary to gradually decelerate to 0 within 2 seconds to avoid the pedestrian; braking: the vehicle needs to complete a stop before the traffic light turns red. The vehicle starts to perform corresponding operations according to the generated intelligent control parameters: gradually decelerating: the autonomous driving system starts to gradually reduce the speed according to the deceleration behavior of the vehicle ahead to ensure a safe distance from the vehicle ahead. Avoiding pedestrians: At the same time, the system further decelerates to near a stop to avoid the pedestrian gradually entering the lane. Stopping: When the traffic light turns red, the vehicle completes a full stop operation to ensure driving safety and comply with traffic rules.

[0083] Further, step S1 includes the following steps:

[0084] Based on the in-vehicle sensors of the vehicle, perform preliminary traffic scene monitoring to generate preliminary traffic scene monitoring data;

[0085] Perform calibration processing on the internal and external parameters of traffic scene monitoring according to the in-vehicle sensors of the vehicle to generate internal and external parameter calibration data; design a coordinate transformation matrix for traffic scene monitoring based on the internal and external parameter calibration data;

[0086] Perform spatial calibration processing on the preliminary traffic scene monitoring data based on the coordinate transformation matrix of traffic scene monitoring to generate traffic scene monitoring data.

[0087] In the embodiments of the present invention, a vehicle monitors the surrounding traffic environment using multiple in-vehicle sensors (such as cameras, radars, V2X communication devices, etc.). The in-vehicle camera of the vehicle is used to collect image data in the traffic scene, including roads, pedestrians, other vehicles, etc. Resolution: 1920x1080, Frame rate: 30FPS, Field of view: 120 degrees. The lidar and millimeter-wave radar are respectively responsible for collecting information such as the distance and speed of objects in the surrounding environment of the vehicle. The millimeter-wave radar covers a detection range of 250 meters. The vehicle communicates with nearby infrastructure and other vehicles to obtain data such as traffic signal status and information about the road ahead. The images obtained by the camera are used to identify the shape and color information of objects. The radar provides the precise position and distance of objects. Road information obtained through V2X, such as the message that the traffic signal ahead is about to turn red. Through the preliminary data collection of these sensors, preliminary traffic scene monitoring data is generated, including visual data, radar data, and V2X data. The internal parameters of sensors such as in-vehicle cameras are calibrated, including focal length, optical axis offset, distortion coefficient, etc. The internal parameters of the camera are corrected using a known calibration board, and the Zhang Zhengyou calibration method is used to correct parameters such as focal length and optical center position. External parameter calibration is used to determine the installation position and direction of the sensor relative to the vehicle reference system. For example, the relative position between the camera and the lidar is determined. The coordinate position and installation angle of the lidar relative to the vehicle center are determined. The attitude of the sensor is corrected through the relative position between the vehicle movement and the reference point. After calibration, internal and external parameter calibration data is generated for subsequent spatial transformation and data correction. Taking the center of the front axle of the vehicle as the origin, the X-axis points in the forward direction of the vehicle, the Y-axis points to the left side of the vehicle, and the Z-axis points to the sky. Each sensor has its own local coordinate system (such as cameras and lidars), and these local coordinate systems are converted to the vehicle coordinate system through external parameter calibration. According to the internal and external parameter calibration data, a coordinate transformation matrix is designed using a homogeneous transformation matrix. The local coordinate system of each sensor is converted to a unified vehicle coordinate system. Among them, the rotation matrix includes the relative attitude between the sensor and the vehicle, and the translation vector represents the relative position between the sensor and the vehicle center. Through this matrix, the coordinates of multiple sensors such as cameras and radars are unified. This matrix is applied to all preliminary data to ensure that the data of each sensor is uniformly mapped into the vehicle coordinate system, facilitating subsequent multi-source data fusion processing. The distortion of the image data collected by the camera is corrected using the distortion coefficient of the internal parameter calibration to make the image more accurate. Then, the visual data is mapped into the vehicle coordinate system using the coordinate transformation matrix. For example, the coordinates of a pedestrian in the image are calculated through homogeneous transformation to obtain its position 5 meters in front of the vehicle. The point cloud data of the lidar and millimeter-wave radar is uniformly converted into the vehicle coordinate system and fused with the visual data to ensure that the objects seen by the radar and the camera are accurately aligned in the same coordinate system. After spatially calibrating and aligning the radar, visual, and V2X data, multi-source data fusion is performed to obtain consistent traffic scene monitoring data.For example, the radar data of the vehicle ahead shows that it is 10 meters ahead. The visual data confirms its bounding box. After alignment, the exact position and driving speed of the vehicle can be accurately estimated. Finally, traffic scene monitoring data integrating multi-source data such as cameras, radars, and V2X is generated and provided to subsequent steps for processing. These monitoring data include the position information of objects around the vehicle, the motion state, and the interaction data with traffic infrastructure.

[0088] Further, step S2 includes the following steps:

[0089] Step S21: Perform gradient edge detection processing on the traffic scene visual data to generate traffic scene visual gradient edge data;

[0090] In the embodiment of the present invention, in traffic scene monitoring, first, the collected traffic scene visual data (image data) is subjected to gradient edge detection processing. The Sobel operator is used to calculate the luminance gradient of the image to generate gradient edge information in the image. The specific implementation method is as follows: By calculating the change in gray values of the image in the X-axis and Y-axis directions, the gradient magnitude and direction of each pixel point are calculated, and then the edge data in the traffic scene is generated. During the calculation process, the gradient threshold is set to 0.1 to filter out weak gradient information and only retain significant edge features. The image processing tool used is the Sobel operator in the OpenCV library. After edge detection, the data is further processed by morphological operations to remove noise and retain significant edges, generating traffic scene visual gradient edge data.

[0091] Step S22: Perform traffic scene visual area analysis processing based on the traffic scene visual gradient edge data to generate traffic scene visual area data;

[0092] In the embodiment of the present invention, after obtaining the gradient edge data, area analysis processing is performed on this data. The area analysis uses the connected component labeling method to combine connected edge points into independent areas. Each area represents an object or entity. In this step, first, the gradient edge data is binarized to ensure that the edge data can clearly represent the area boundaries. Subsequently, the connected component algorithm is used to process the binarized image, mark all connected areas, and perform statistics and classification on the boundaries of these areas. By calculating the features such as the area, perimeter, and shape of each connected area, traffic scene visual area data is generated. The generated visual area data includes the bounding box information of each object and attributes such as position and area.

[0093] Step S23: Perform candidate object area evaluation processing on the traffic scene visual data based on the traffic scene visual area data to generate candidate object area evaluation data;

[0094] In the embodiments of the present invention, after obtaining the visual area data, candidate object evaluation processing is performed on each visual area. In the evaluation process, the area is initially filtered according to its size, shape, and position. Areas smaller than a predetermined threshold (such as 20 pixels) are regarded as noise and directly filtered out. Among the remaining areas, the aspect ratio and shape features of the area are further analyzed to determine whether they conform to the characteristics of common traffic scene objects (such as pedestrians, vehicles). For example, for a vehicle, the aspect ratio is usually between 1.5 and 3, while for a pedestrian, the aspect ratio is close to 0.5 to 1.5. Through this regularized evaluation method, candidate object area evaluation data is generated. The data of each candidate object includes geometric attributes such as the bounding box, area, and aspect ratio of the area, as well as the evaluation type it passes through.

[0095] Step S24: Perform visual object attribute analysis processing based on the candidate object area evaluation data to generate visual object attribute data;

[0096] In the embodiments of the present invention, after generating the candidate object area evaluation data, attribute analysis is performed on the candidate objects. First, the color features of each candidate object are analyzed through a color histogram to determine whether the color distribution of the object conforms to common traffic object types (such as red and yellow pedestrian sign clothing, white and black vehicle colors). In addition, combined with the bounding box information of the object, the actual physical size of the object is judged by using the size and shape of the object. In this process, a local feature descriptor of the image (such as HOG feature) is used to perform a detailed analysis of the object to obtain more texture and shape information to help identify the category of the object. Based on a pre-designed object recognition model, attribute analysis is performed on the texture and shape information to generate the attribute data of the visual object, which includes the color, size, texture information of each object, and the preliminary category inference result.

[0097] Step S25: Perform visual object identification and tracking processing on the candidate object area evaluation data to generate visual object identification and tracking data;

[0098] In the embodiments of the present invention, based on the candidate object area evaluation data, identification and tracking processing of the object is performed. In order to ensure the continuous tracking of dynamic targets in the traffic scene, a Kalman filter combined with the Hungarian algorithm is used for multi-target tracking. First, the Kalman filter is used to predict the position of each object in the next frame, and the prediction result is matched with the actually observed object position. Through the Hungarian algorithm, the multi-target matching problem is solved to ensure that each object can be correctly associated with the corresponding object in the previous frame. The identification and tracking process assigns a unique ID to each object and records its trajectory and motion state on the time axis. Finally, visual object identification and tracking data is generated, including dynamic information such as the ID, motion trajectory, speed, and acceleration of the object.

[0099] Step S26: Analyze the visual objects in the traffic scene based on the visual object attribute data and the visual object identification tracking data to generate the visual object data of the traffic scene.

[0100] In the embodiment of the present invention, the attribute data of the visual object and the identification tracking data are combined to perform comprehensive analysis of the visual object. This analysis finally determines the category of the object through various feature information (such as color, shape, motion trajectory). For example, if an object has the shape characteristics and motion patterns of a pedestrian, it is finally identified as a pedestrian; if the object has the attributes of a vehicle and maintains a stable speed and trajectory, it is identified as a vehicle. By combining the cumulative analysis of multiple frames of data, short-term errors are eliminated to ensure the accuracy of object classification. After the final analysis and processing, the visual object data in the traffic scene is generated, including key information such as the category (such as vehicle, pedestrian, obstacle), position, speed, and trajectory of each object. These visual object data will be used in subsequent behavior prediction and vehicle control decisions.

[0101] Further, step S23 includes the following steps:

[0102] Step S231: Extract region multi-scale visual features from the traffic scene visual data according to the traffic scene visual region data to generate region multi-scale visual feature data;

[0103] In the embodiment of the present invention, based on the traffic scene visual region data, multi-scale visual features are extracted for each candidate region. The pyramid model is used to decompose the image into image levels of different scales for feature extraction from both the global and local levels. During the feature extraction process, a convolutional neural network (CNN) model is used to process the images at each scale. The convolutional layer of the CNN model extracts features layer by layer from the image, and the extracted features include shape, edge, texture, and color, etc. Through this multi-scale processing, complete feature information can be extracted for candidate objects of different sizes. The data after multi-scale feature extraction includes feature descriptors at different resolutions for each candidate region, generating region multi-scale visual feature data, and the data contains feature vectors of each candidate object at different scales.

[0104] Step S232: Perform a sliding window traversal process on the region multi-scale visual feature data to generate sliding window traversal visual feature data;

[0105] In the embodiment of the present invention, after obtaining the regional multi-scale visual feature data, the data is processed by traversing with a sliding window. The specific implementation is as follows: A sliding window with a fixed size is defined, and it is gradually traversed from left to right and from top to bottom within each candidate region. The size of the sliding window is set to 64×64 pixels, and the step size is 16 pixels to ensure sufficient coverage and appropriate processing speed. Within each sliding window, the local visual features of the region are extracted, and a pre-trained deep learning model (such as YOLO or SSD) is used to classify and identify the candidate objects in each window. During this process, by analyzing the features within each sliding window, it is determined whether the region contains valid traffic objects (such as vehicles, pedestrians, etc.). After the traversal process, the visual feature data of the sliding window traversal is generated, and the data of each window includes the recognized object category and the confidence value.

[0106] Step S233: Analyze and process the overlap degree of candidate objects according to the visual feature data of the sliding window traversal to generate candidate object overlap degree data;

[0107] In the embodiment of the present invention, after the sliding window traversal processing is completed, the overlapping regions between candidate objects are analyzed. First, according to the visual feature data obtained by the sliding window traversal, the bounding boxes of each candidate object are calculated. If there is an overlapping region between the bounding boxes of two candidate objects, their overlap degree is calculated. The calculation formula for the overlap degree is IoU (Intersection over Union), that is, the ratio of the intersection to the union of the two bounding boxes. To ensure the accuracy of the evaluation, the overlap degree threshold is set to 0.5. When the overlap degree of two candidate objects is greater than 0.5, these two objects are considered to belong to the same target and are merged. Finally, the candidate object overlap degree data is generated, including the overlap situation of all candidate objects, the size of the overlapping region, and the overlap degree value.

[0108] Step S234: Evaluate the regions of candidate objects according to the candidate object overlap degree data to generate candidate object region evaluation data.

[0109] In the embodiment of the present invention, according to the candidate object overlap degree data, the regions of candidate objects are further evaluated and optimized. First, the non-maximum suppression (NMS) algorithm is used to process the overlapping regions. This algorithm compares the confidences of overlapping candidate objects, retains the object with the highest confidence, and filters out the remaining overlapping objects to ensure that only the most likely candidate regions are finally retained. Objects with a large overlapping region are merged, and objects with a confidence lower than the set threshold (such as 0.6) are directly removed. After the processing, the final candidate object region evaluation data is generated, and the data includes the bounding box, category, confidence, and overlap situation of each candidate object. This data will serve as an important basis for subsequent visual object analysis and is used for further decision-making in vehicle intelligent driving control.

[0110] Furthermore, step S3 includes the following steps:

[0111] Step S31: Analyze the lidar point cloud features based on the traffic scene radar data to generate lidar point cloud feature data; perform visual object localization processing on the traffic scene visual object data based on the lidar point cloud feature data to generate visual object localization data;

[0112] In the embodiment of the present invention, in a traffic scene, the point cloud data collected by the lidar is used to generate an accurate three-dimensional environment model. First, perform feature analysis on the point cloud data collected by the lidar, and use the ICP (Iterative Closest Point) algorithm to register the point cloud data to eliminate the tiny errors generated during lidar scanning. Then, based on the geometric shape of the point cloud, extract feature points such as planes, corner points, boundaries, etc., and calculate the three-dimensional coordinates of the feature objects in the point cloud. Match these point cloud data with the bounding boxes of the visual objects, and combine the three-dimensional positions of the feature points to complete the three-dimensional localization of the visual objects. The generated visual object localization data contains the accurate three-dimensional coordinates, sizes, and azimuth angles of each object, and is used for subsequent behavior modeling processing.

[0113] Step S32: Analyze the millimeter-wave radar point cloud features based on the traffic scene radar data to generate millimeter-wave radar point cloud feature data; perform visual object trajectory modeling processing on the traffic scene visual object data based on the millimeter-wave radar point cloud feature data to generate visual object trajectory model data;

[0114] In the embodiment of the present invention, the millimeter-wave radar provides accurate long-distance detection, especially suitable for the monitoring of vehicles and other large dynamic targets. The data collected by the millimeter-wave radar contains the speed, distance, and azimuth information of the target object. According to these data, first perform point cloud feature analysis, and use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to cluster the point cloud data to distinguish different target objects. Then, perform trajectory modeling processing on the millimeter-wave radar data through a Kalman filter to track the movement trajectories of each target, including its speed, acceleration, and change direction. The generated visual object trajectory model data records the position information, movement direction, and speed of each object at different time points, forming a continuous trajectory record for subsequent behavior prediction.

[0115] Step S33: Perform visual object behavior modeling processing based on the visual object localization data and the visual object trajectory model data to generate a visual object behavior model;

[0116] In the embodiment of the present invention, after generating the positioning data and trajectory data of the visual object, behavior modeling is performed. First, based on the trajectory data of the object, a Bayesian network is used for behavior modeling to establish the motion pattern of the object. Through the learning of historical trajectories, the Bayesian network can infer the future motion trend of the object. Next, Kalman filtering is performed on the real-time position data of the object, and combined with the precise positioning data of the lidar, the behavior of predicting the future position of the object is carried out, such as vehicle acceleration, turning, or pedestrians crossing the road. The output result of the visual object behavior model is the behavior prediction of each object, including the future moving direction, speed, and sudden behaviors (such as sudden stops or turns).

[0117] Step S34: Perform visual object behavior analysis on the traffic scene visual object data according to the visual object behavior model to generate visual object behavior data.

[0118] In the embodiment of the present invention, based on the visual object behavior model, behavior analysis is performed on all visual objects in the traffic scene. First, according to the results predicted by the model, the behavior pattern of each object is analyzed to determine the actions it will take at a future moment. By analyzing features such as the speed change and path offset of the vehicle, it is judged whether it changes lanes or makes a sudden stop. For pedestrians, it is judged whether they are about to enter the lane according to their walking speed and path. The results of the behavior analysis include the real-time motion state of the object and the future behavior prediction, and the system will also generate a behavior risk assessment for each object. The visual object behavior data includes the current motion state (position, speed, acceleration) of the object and the future behavior estimation result, which is used for further intelligent driving control of the vehicle.

[0119] Further, as an embodiment of the present invention, refer to Figure 2 shown, for Figure 1 the detailed step flow diagram of step S4 in

[0120] Step S41: Perform traffic scene visual dynamic description feature analysis on the traffic scene visual data to generate traffic scene visual dynamic description feature data.

[0121] In an embodiment of the present invention, dynamic feature information is extracted from traffic scene visual data. The motion state of visual objects in the traffic scene is analyzed through an algorithm based on the optical flow method. The optical flow method calculates the moving direction and speed information of visual objects by analyzing the changes of each pixel in consecutive frames. For traffic scenes, the main dynamic features include the driving speed and direction changes of vehicles, as well as the walking speed and pace changes of pedestrians, vehicle turn signals, etc. The Lucas-Kanade optical flow algorithm is used to generate dynamic description feature data reflecting the motion state of each visual object by tracking the pixel motion trajectories of moving objects frame by frame. This data contains key information such as the real-time motion trajectories and trajectory trend information of the objects, forming traffic scene visual dynamic description feature data.

[0122] Step S42: Perform visual data analysis on the trajectory trend of visual objects according to the traffic scene visual dynamic description feature data and the visual object identification tracking data corresponding to the traffic scene visual object data, and generate visual object trajectory trend visual data;

[0123] In an embodiment of the present invention, after obtaining the traffic scene visual dynamic description feature data, it is matched with the visual object identification tracking data to optimize the trajectory trend analysis of the object. First, the historical trajectory data of visual objects is smoothed through a Kalman filter to eliminate the noise and outliers in the trajectory. Combining the dynamic feature data extracted by the optical flow method, the motion trajectory of the visual object is recalculated to predict its future motion trend. By cumulatively analyzing the dynamic feature data of consecutive frames, it is judged whether the visual object has acceleration, deceleration or turning behaviors, and the motion information insufficient for the radar device to analyze the object's motion behavior is optimized through these visual trends. Visual object trajectory trend visual data is generated, and the data content includes the visual future trajectory prediction, speed change trend and motion direction change of each visual object.

[0124] Step S43: Collect traffic scene visual object communication data for the traffic scene visual object data according to the traffic scene V2X data, and generate traffic scene visual object communication data;

[0125] In the embodiments of the present invention, according to the V2X communication data in the traffic scenario, the acquisition of communication data associated with visual objects is carried out. Through the V2X device, the vehicle obtains the real-time communication information of other vehicles and road infrastructure around. For example, the braking signal of the vehicle ahead, the change state of the traffic lights at the intersection, and the road blockage information, etc. For each visual object, the system collects its relevant V2X communication data and synchronizes it. For example, the braking signal, emergency stop warning or lane change information of the vehicle ahead. By associating with the identification tracking data of the visual object, it is ensured that the V2X data is accurately matched with the visual object. The generated traffic scenario visual object communication data includes communication information between vehicles, traffic light signals, road status and other information, which is used to assist in trajectory prediction and behavior analysis.

[0126] Step S44: Based on the visual data of the visual object trajectory trend and the traffic scenario visual object communication data, perform optimization processing on the visual object behavior data to generate optimized visual object behavior data, where the optimized visual object behavior data includes visual object real-time behavior data and visual object predicted behavior data.

[0127] In the embodiments of the present invention, based on the visual data of the visual object trajectory trend and the traffic scenario visual object communication data, further precision optimization processing of the behavior trend is carried out. First, combine the trajectory prediction results in the trajectory trend optimization behavior data with the vehicle braking information, traffic light changes, etc. in the communication data to correct the behavior prediction of the visual object. For example, when the V2X communication data of the vehicle ahead indicates that it is about to brake, the system corrects the trajectory prediction of the vehicle and determines that it will enter a decelerating state. For pedestrians, if the V2X data contains relevant road warning information, the system will consider the brake lights, vehicle turn signals, visual changes of pedestrians and the traffic lights in its visual analysis. Through this multi-source data fusion processing, a more accurate visual object behavior prediction result is generated. The generated optimized visual object behavior data includes the real-time behavior data of the visual object, such as the current speed, direction, acceleration, and predicted behavior data, such as the trajectory, turning, parking or accelerating behavior in the next few seconds.

[0128] Further, as an embodiment of the present invention, refer to Figure 3 shown, for Figure 1 the detailed step flow diagram of step S5 in

[0129] Step S51: Design a preferred priority vehicle driving decision based on the visual object real-time behavior data;

[0130] In the embodiments of the present invention, based on the real-time behavior data of visual objects, the preferred priority-driven decision-making for the vehicle is designed. First, by analyzing the real-time behavior data of each visual object in the traffic scene, including the current speed, position, acceleration, and movement direction of the object. Using the rule engine analysis method, classify the real-time behavior data of visual objects to determine which objects pose a direct threat to the vehicle's driving safety, such as a pedestrian approaching suddenly or a vehicle changing lanes. For each high-risk object, generate a set of emergency handling decisions, such as braking, decelerating, or changing lanes. The design of the decision-making takes safety as the primary goal, giving priority to immediate braking or changing the driving direction to ensure a safe distance from high-risk objects, and completing the design of the preferred priority vehicle driving decision.

[0131] Step S52: Design an alternative priority vehicle driving decision based on the predicted behavior data of visual objects;

[0132] In the embodiments of the present invention, according to the predicted behavior data of visual objects, an alternative priority vehicle driving decision is designed. The predicted behavior data is the future behavior deduced based on the historical trajectory of the object and its movement trend, such as the acceleration, deceleration, or steering behavior of the vehicle. Predict the behavior of visual objects through machine learning models (such as LSTM or Bayesian networks) to obtain the future trajectory. Combining the current driving path of the vehicle and the traffic conditions, the system conducts a risk assessment on these future behaviors and designs corresponding alternative decision-making schemes. For example, when it is predicted that the vehicle in front suddenly brakes, the alternative priority decision is to decelerate in advance or change lanes. The alternative decision and the preferred decision jointly construct a multi-level response plan to ensure that the vehicle can adjust the driving strategy in a timely manner in case of emergencies.

[0133] Step S53: Design the decision mapping relationship between the visual object behavior and the vehicle driving control based on the decision tree algorithm and the preferred priority vehicle driving decision, and generate the initial vehicle driving control decision tree model;

[0134] In the embodiments of the present invention, use the decision tree algorithm to model the preferred priority decision in vehicle driving control to generate the decision mapping relationship between the visual object behavior and the vehicle driving control. Based on the real-time behavior data of visual objects and the preferred priority driving decision, determine the key features in driving behavior, such as the distance, speed, acceleration of visual objects, and the current driving environment (such as road conditions, traffic light status, etc.). These features are used as input variables for the decision tree. Then, construct the structure of the decision tree. Each node of the decision tree represents a judgment condition, such as whether the visual object enters the emergency braking distance, or whether the current object speed exceeds the safety threshold. Each leaf node corresponds to specific vehicle driving control actions, such as braking, steering, or maintaining speed. Use the Gini index as the feature selection criterion to ensure that the branches of each node effectively distinguish different driving decision scenarios, and generate the initial vehicle driving control decision tree model.

[0135] Step S54: According to the alternative priority vehicle driving decision, perform visual object warning behavior evolution-based vehicle driving control decision node adjustment processing on the initial vehicle driving control decision tree model to generate an adjusted vehicle driving control decision tree model;

[0136] In the embodiment of the present invention, by introducing the alternative priority vehicle driving decision, the initial vehicle driving control decision tree model is adjusted. The alternative priority decision is based on visual object prediction behavior data to infer the future scenario, such as whether the vehicle ahead is about to change lanes or brake. For these future possibilities, the nodes in the decision tree need to be further refined. For example, when it is predicted that the vehicle ahead is about to brake, new judgment nodes are added based on the initial model, and corresponding alternative priority decisions are added on this basis, such as decelerating in advance or changing lanes. Using the dynamic node adjustment method, the weights and priorities of each decision node are reconfigured to ensure that the vehicle can preferentially select safer control measures in complex scenarios, and an adjusted vehicle driving control decision tree model is generated.

[0137] Step S55: Obtain the historical vehicle driving control sample set;

[0138] In the embodiment of the present invention, a large number of historical driving control sample sets are extracted from the historical data records in the vehicle control system as the data basis for the optimization and training of the decision tree model. These historical data samples include the real-time reactions and control decision records of the vehicle in different traffic scenarios, such as emergency braking, obstacle avoidance, turning, acceleration and other operations. The data in the sample set mainly comes from sensor information (including cameras, radars, V2X, etc.) and the specific control decisions made by the vehicle at different time points. To ensure the effectiveness and availability of the data, the historical sample set needs to be subjected to data cleaning and denoising processing after acquisition to remove outliers and noise information, ensuring that the data is representative and can be used for subsequent model optimization training.

[0139] Step S56: Based on the historical vehicle driving control sample set, perform model decision update training processing on the adjusted vehicle driving control decision tree model to generate a vehicle driving control decision tree model;

[0140] In the embodiments of the present invention, the adjusted vehicle driving control decision tree model is updated and trained using a historical vehicle driving control sample set. Through a supervised learning algorithm, the feature data in the historical sample set is input into the decision tree model, and the model is trained according to the actual control decisions in the sample set. In each training, the node weights and branch logics in the decision tree model are gradually adjusted according to the performance in the actual driving scenario. The feature importance evaluation is calculated through information gain or Gini index, and the features that can most effectively distinguish different decisions in the actual scenario are selected. After training, an optimized vehicle driving control decision tree model that can adapt to complex traffic scenarios is generated. This model has been verified through a large number of scenarios and can make more accurate control decisions based on real-time data.

[0141] Step S57: Design an intelligent analysis engine for vehicle intelligent driving control according to the vehicle driving control decision tree model, and generate a vehicle intelligent driving control engine.

[0142] In the embodiments of the present invention, according to the optimized vehicle driving control decision tree model, an intelligent analysis engine for vehicle intelligent driving control is designed and generated. This engine includes multiple modules, including a real-time data processing module, a decision-making inference module, an emergency response module, etc. First, the real-time data processing module collects the latest traffic scenario data (such as visual object behavior, radar data, and V2X data) from sensors. Then, the decision-making inference module makes a real-time analysis of the current scenario based on the decision tree model and selects the optimal driving decision. The emergency response module processes emergencies, such as suddenly appearing pedestrians or obstacles, to ensure that the vehicle can respond within the shortest time. Through the collaborative work of these modules, the generated intelligent driving control engine can monitor the traffic environment in real time and adaptively adjust the vehicle's driving control strategy according to the current scenario, improving the vehicle's safety and driving efficiency.

[0143] Further, step S52 includes the following steps:

[0144] Step S521: Extract visual object anomaly-related behavior features based on visual object prediction behavior data to generate visual object anomaly-related behavior feature data;

[0145] In the embodiments of the present invention, based on the predicted behavior data of visual objects, abnormal-related behavior features are extracted. First, a model is constructed using historical behavior data. By analyzing the regular behavior trajectories of visual objects such as vehicles and pedestrians, common behavior patterns (such as straight driving, lane changing, deceleration, etc.) are identified. For the predicted behavior data of each visual object, by comparing its difference with the normal behavior model, abnormal-related behavior features are extracted. The extracted features include sudden acceleration, sharp turns, unexpected deceleration, or unreasonable trajectory deviation, etc. The dynamic time warping (DTW) algorithm is used to compare the trajectories and speed changes of each visual object, so as to identify the features that are significantly different from the regular behavior. The generated abnormal-related behavior feature data of visual objects includes detailed feature information such as the rate of speed change, trajectory deviation amount, acceleration anomaly value, etc., which is used for further probability analysis.

[0146] Step S522: Perform a probability analysis related to the abnormal behavior of visual objects based on the abnormal-related behavior feature data of visual objects, and generate probability data related to the abnormal behavior of visual objects;

[0147] In the embodiments of the present invention, based on the abnormal behavior feature data of visual objects extracted in the previous step, a probability analysis is performed to calculate the possibility of each visual object having an abnormal behavior. First, a probability model based on a Bayesian network is used, combined with the abnormal behavior patterns in the historical data, to analyze the current features of each visual object. Through the conditional probability distribution in the Bayesian network, the abnormal probabilities of each feature in different scenarios are calculated. For example, if the trajectory deviation amount and acceleration of an object exceed the set range, the probability of its having an abnormal behavior will increase significantly. The calculated probability value indicates the risk level of the object having an abnormal behavior. The generated probability data related to the abnormal behavior of visual objects includes the occurrence probability of the abnormal behavior of each object, the influence weights of different behavior features, and the comprehensive risk score, which is used for subsequent warning and decision-making processing.

[0148] Step S523: Screen the probability data related to the warning abnormal behavior in the probability data related to the abnormal behavior of visual objects based on a preset abnormal behavior warning threshold, and design an alternative priority vehicle driving decision according to the probability data related to the warning abnormal behavior.

[0149] In the embodiments of the present invention, visual objects with abnormal behavior probabilities exceeding the warning threshold are screened according to a preset abnormal behavior warning threshold. First, a warning threshold (for example, 0.3) is set, indicating that when the abnormal behavior probability of a certain visual object exceeds this value, the system will mark it as an abnormal behavior that affects driving control. By screening the probability data related to the abnormal behavior of visual objects, the abnormal behavior data exceeding the threshold is extracted. These data indicate that the object will have a sudden behavior, such as suddenly changing lanes or making an emergency stop, within a certain period in the future. Based on these warning abnormal behavior related probability data, alternative priority vehicle driving decisions are designed. The system generates multiple alternative driving schemes according to the current driving state and warning information of the vehicle, including decelerating in advance, maintaining a safe distance, or making an emergency brake, etc. Each decision-making scheme is based on the risk assessment of a specific scenario to ensure that the vehicle can take reasonable countermeasures before the abnormal behavior occurs.

[0150] Further, the step of transmitting the updated and optimized visual object behavior data to the vehicle intelligent driving control engine for vehicle intelligent driving control analysis in step S6 includes the following steps:

[0151] Transmit the updated and optimized visual object behavior data to the vehicle intelligent driving control engine: Analyze the vehicle driving control behavior state of the updated and optimized visual object behavior data through the preferred priority vehicle driving decision of the vehicle intelligent driving control engine to generate vehicle driving control behavior state data; Analyze the vehicle driving control behavior state inference of the updated and optimized visual object behavior data through the alternative priority vehicle driving decision of the vehicle intelligent driving control engine to generate vehicle driving control behavior state inference data;

[0152] Conduct vehicle driving control change state analysis on the vehicle driving control behavior state inference data based on the preset traffic scenario limiting change conditions to generate vehicle driving control change state data;

[0153] Conduct vehicle intelligent driving control analysis based on the vehicle driving control change state data and the vehicle driving control behavior state data to generate intelligent vehicle driving control parameters.

[0154] In the embodiments of the present invention, the updated and optimized visual object behavior data obtained through visual object behavior analysis is transmitted to the vehicle intelligent drive control engine. The intelligent drive control engine analyzes this data based on the preferred priority vehicle drive decision. Specifically, the visual object behavior data includes information such as the movement trajectory, speed, acceleration, and direction of the objects around the vehicle. The intelligent drive control engine determines the control behavior state of the vehicle according to this real-time data and the current traffic conditions. For example, when the system detects that the vehicle in front suddenly decelerates, the preferred priority decision will recommend that the vehicle immediately perform a braking operation. In this way, the intelligent drive control engine generates vehicle drive control behavior state data, which includes the current acceleration, braking, steering, and other operation states of the vehicle, ensuring that the vehicle can make a quick response according to the current traffic scenario. The alternative priority drive decision module of the vehicle intelligent drive control engine performs reasoning analysis on the updated and optimized visual object behavior data. The alternative priority decision focuses on future potential complex scenarios or emergencies. For example, the system infers the behavior trend of the current visual object, combines the historical data in the traffic scenario and other sensor information (such as V2X communication data), and estimates the risk scenarios that may occur in the next few seconds, such as the vehicle in front suddenly stopping or a pedestrian suddenly entering the lane. The alternative priority decision matches the generated reasoning data with the existing vehicle control behavior to form vehicle drive control behavior state reasoning data, which includes the inference of the upcoming behavior and the corresponding drive control strategy. Further analysis is performed on the vehicle drive control behavior state reasoning data, and preset traffic scenario limiting conditions are used to evaluate whether it is necessary to change the current vehicle control state. The preset conditions include important parameters in the scenario, such as the safety distance, road speed limit, traffic signal status, etc. For example, when the deceleration behavior of the vehicle in front exceeds the safety distance threshold preset by the system, the system will activate the change state analysis module. The change state analysis will evaluate whether the vehicle needs to take emergency avoidance measures (such as immediately decelerating or steering) and generate vehicle drive control change state data. This data indicates whether the vehicle needs to change from the current behavior state to the control behavior under the alternative strategy, such as switching from the normal driving state to the deceleration and obstacle avoidance state. The vehicle drive control behavior state data and the change state data are combined to generate the final intelligent vehicle drive control parameters. The intelligent drive control analysis module weighs all the input data, considers the priority between the preferred priority and the alternative priority decision, and generates control parameters in combination with the real-time road conditions and the current state of the vehicle. The parameters include specific vehicle action instructions, such as accelerating, decelerating, braking, or steering, etc. If there are high-risk abnormal behaviors in the current scenario (such as a pedestrian entering the lane or the vehicle in front suddenly braking), the system will give priority to triggering emergency braking; if the reasoning result shows that the risk is reduced, the vehicle will maintain the current speed. The generated intelligent vehicle drive control parameters ensure that the vehicle maintains the best response speed and safety in complex traffic scenarios.

[0155] This specification provides a vehicle intelligent driving control system for traffic scenario analysis, which is used to execute the vehicle intelligent driving control method for traffic scenario analysis as described above. The vehicle intelligent driving control system for traffic scenario analysis includes:

[0156] A traffic scenario monitoring module, which is used to perform traffic scenario monitoring and analysis processing based on the vehicle's on-vehicle sensors, and generate traffic scenario monitoring data, where the traffic scenario monitoring data includes traffic scenario visual data, traffic scenario radar data, and traffic scenario V2X data;

[0157] A traffic scenario visual object analysis module, which is used to perform traffic scenario visual object analysis based on traffic scenario visual data and generate traffic scenario visual object data;

[0158] A visual object behavior analysis module, which is used to perform visual object behavior analysis on the traffic scenario visual object data based on traffic scenario radar data and generate visual object behavior data;

[0159] A visual object behavior trend accuracy optimization module, which is used to perform visual object behavior trajectory trend optimization processing on the visual object behavior data based on traffic scenario visual data and traffic scenario V2X data, and generate optimized visual object behavior data;

[0160] A vehicle intelligent driving control engine design module, which is used to obtain historical vehicle driving control data; perform intelligent analysis engine design on visual object behavior and vehicle driving control based on historical vehicle driving control data, and generate a vehicle intelligent driving control engine;

[0161] An intelligent vehicle driving control analysis module, which is used to execute an optimized visual object behavior monitoring and updating operation for the vehicle driving traffic scenario according to the optimized visual object behavior data; collect updated optimized visual object behavior data according to the optimized visual object behavior monitoring and updating operation for the vehicle driving traffic scenario; transmit the updated optimized visual object behavior data to the vehicle intelligent driving control engine for vehicle intelligent driving control analysis, generate intelligent vehicle driving control parameters; and execute an intelligent vehicle driving control operation for the traffic scenario based on the intelligent vehicle driving control parameters.

[0162] The beneficial effects of this application are as follows. By integrating visual data, radar data, and V2X data, and comprehensively utilizing the data sources of different types of sensors, the present invention overcomes the limitations of relying on a single data source and significantly improves the perception accuracy of multiple targets and multiple dynamic objects in complex traffic scenarios. Especially in traffic scenarios, the combination of lidar and millimeter-wave radar makes the detection, tracking, and modeling of object behaviors more accurate, and the system can identify and analyze the behavior trends of target objects faster. Through data fusion, the blind spots in scene perception are reduced, and the real-time performance of decision-making is improved. By modeling and trend prediction of the behavior data of visual objects, combined with the fusion analysis of trajectory modeling and communication data, the prediction accuracy of visual object behaviors can be effectively optimized. Through the visual optimization of trajectory trends and the assistance of V2X data, the system can not only accurately predict the real-time behaviors of visual objects and anticipate their future behaviors, ensuring that the vehicle has stronger response capabilities in complex dynamic traffic scenarios and reducing the occurrence of delays and misjudgments. Through the design of an intelligent driving control engine, combined with the real-time data and predicted data of visual object behaviors, an intelligent vehicle driving decision tree model is constructed. Based on the decision tree algorithm, this model can dynamically adjust the vehicle's driving control strategy in real time, enabling the vehicle to flexibly respond to various complex environments according to changes in the traffic scenario. By combining the patterns in historical vehicle driving control data with the behavior trends of visual objects in the current scenario, the system can quickly adjust the vehicle's control strategy when facing atypical scenarios, improving the flexibility and adaptability of vehicle control. This intelligent adjustment mechanism enables the vehicle to make more reasonable and safe driving decisions in different traffic scenarios. It enables the engine to have the ability of autonomous learning. By continuously optimizing the decision model and simulating the corresponding vehicle driving control responses in traffic scenarios made by the real driving experience of the driver, the intelligent level of vehicle control is improved, and the emergency response ability of the system is enhanced. The optimized prediction of visual object behavior data and the rapid response of the intelligent driving control engine significantly improve the vehicle's response ability in emergencies and reduce misjudgments and lags in vehicle control decisions. Based on the fusion analysis of multi-source data, the vehicle can anticipate potential risk factors in complex traffic environments in advance and take timely countermeasures to ensure the real-time performance and safety of driving decisions. Thus, it improves the perception and analysis accuracy of dynamic targets in traffic scenarios, and through the visual dynamic features and V2X communication data, trend prediction and optimization of the visual object behaviors obtained from lidar analysis are carried out, realizing multi-source data fusion, ensuring that the vehicle can quickly respond and make reasonable control in changing traffic scenarios, enhancing the vehicle's adaptability in complex scenarios, and through the combination of historical data and real-time data, and applying it to the decision tree model for learning and analysis of vehicle driving control, realizing intelligent vehicle driving control decisions.

[0163] Therefore, in any aspect, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Thus, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be encompassed within the present invention.

[0164] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A vehicle intelligent driving control method for traffic scene analysis, characterized in that Including the following steps: Step S1: Based on the in-vehicle sensors of the vehicle, traffic scene monitoring and analysis processing is carried out to generate traffic scene monitoring data, where the traffic scene monitoring data includes traffic scene visual data, traffic scene radar data, and traffic scene V2X data; Among them, step S1 includes: Based on the in-vehicle sensors of the vehicle, preliminary traffic scene monitoring is carried out to generate preliminary traffic scene monitoring data; according to the in-vehicle sensors of the vehicle, internal and external parameter calibration processing of traffic scene monitoring is carried out to generate internal and external parameter calibration data; based on the internal and external parameter calibration data, a coordinate transformation matrix for traffic scene monitoring is designed; based on the coordinate transformation matrix for traffic scene monitoring, spatial calibration processing of traffic scene monitoring is carried out on the preliminary traffic scene monitoring data to generate traffic scene monitoring data; Step S2: According to the traffic scene visual data, traffic scene visual object analysis is carried out to generate traffic scene visual object data; Among them, step S2 includes: Step S21: Gradient edge detection processing is carried out on the traffic scene visual data to generate traffic scene visual gradient edge data; Step S22: According to the traffic scene visual gradient edge data, traffic scene visual region analysis processing is carried out to generate traffic scene visual region data; Step S23: Based on the traffic scene visual region data, candidate object region evaluation processing is carried out on the traffic scene visual data to generate candidate object region evaluation data; Step S24: According to the candidate object region evaluation data, visual object attribute analysis processing is carried out to generate visual object attribute data; Step S25: Visual object identification tracking processing is carried out on the candidate object region evaluation data to generate visual object identification tracking data; Step S26: Based on the visual object attribute data and the visual object identification tracking data, traffic scene visual object analysis is carried out to generate traffic scene visual object data; Step S3: According to the traffic scene radar data, visual object behavior analysis is carried out on the traffic scene visual object data to generate visual object behavior data; Among them, step S3 includes: Step S31: According to the traffic scene radar data, lidar point cloud feature analysis is carried out to generate lidar point cloud feature data; based on the lidar point cloud feature data, visual object positioning processing is carried out on the traffic scene visual object data to generate visual object positioning data; Step S32: According to the traffic scene radar data, millimeter-wave radar point cloud feature analysis is carried out to generate millimeter-wave radar point cloud feature data; based on the millimeter-wave radar point cloud feature data, visual object trajectory modeling processing is carried out on the traffic scene visual object data to generate visual object trajectory model data; Step S33: According to the visual object positioning data and the visual object trajectory model data, visual object behavior modeling processing is carried out to generate a visual object behavior model; Step S34: According to the visual object behavior model, visual object behavior analysis is carried out on the traffic scene visual object data to generate visual object behavior data; Step S4: Based on the traffic scene visual data and the traffic scene V2X data, visual object behavior trajectory trend optimization processing is carried out on the visual object behavior data to generate optimized visual object behavior data; Step S5: Obtain historical vehicle drive control data; design an intelligent analysis engine for visual object behavior and vehicle drive control based on the historical vehicle drive control data to generate a vehicle intelligent drive control engine; Step S6: Perform an optimized visual object behavior monitoring and update operation for the vehicle drive traffic scene according to the optimized visual object behavior data; collect and update the optimized visual object behavior data according to the optimized visual object behavior monitoring and update operation for the vehicle drive traffic scene; transmit the updated optimized visual object behavior data to the vehicle intelligent drive control engine for vehicle intelligent drive control analysis to generate intelligent vehicle drive control parameters; perform an intelligent vehicle drive control operation for the traffic scene based on the intelligent vehicle drive control parameters.

2. The vehicle intelligent drive control method for traffic scene analysis according to claim 1, characterized in that Step S23 includes the following steps: Step S231: Extract region multi-scale visual features from the traffic scene visual data according to the traffic scene visual region data to generate region multi-scale visual feature data; Step S232: Perform a sliding window traversal process on the region multi-scale visual feature data to generate sliding window traversal visual feature data; Step S233: Perform a candidate object overlap degree analysis process according to the sliding window traversal visual feature data to generate candidate object overlap degree data; Step S234: Perform a candidate object region evaluation process according to the candidate object overlap degree data to generate candidate object region evaluation data.

3. The vehicle intelligent drive control method for traffic scene analysis according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Analyze the traffic scene visual dynamic description features according to the traffic scene visual data to generate traffic scene visual dynamic description feature data; Step S42: Perform visual object trajectory trend visual data analysis according to the traffic scene visual dynamic description feature data and the visual object identification tracking data corresponding to the traffic scene visual object data to generate visual object trajectory trend visual data; Step S43: Collect traffic scene visual object communication data for the traffic scene visual object data according to the traffic scene V2X data to generate traffic scene visual object communication data; Step S44: Perform an optimized process for the visual object behavior trajectory trend on the visual object behavior data based on the visual object trajectory trend visual data and the traffic scene visual object communication data to generate optimized visual object behavior data, where the optimized visual object behavior data includes visual object real-time behavior data and visual object predicted behavior data.

4. The vehicle intelligent drive control method for traffic scene analysis according to claim 1, characterized in that Step S5 includes the following steps: Step S51: Design a preferred priority vehicle drive decision based on the visual object real-time behavior data; Step S52: Design an alternative priority vehicle drive decision based on the visual object predicted behavior data; Step S53: Design a decision mapping relationship between visual object behavior and vehicle drive control based on the decision tree algorithm and the preferred priority vehicle drive decision to generate an initial vehicle drive control decision tree model; Step S54: Perform a vehicle drive control decision node adjustment process for the visual object warning behavior evolution on the initial vehicle drive control decision tree model according to the alternative priority vehicle drive decision to generate an adjusted vehicle drive control decision tree model; Step S55: Obtain a historical vehicle drive control sample set; Step S56: Based on the historical vehicle drive control sample set, perform model decision update training on the adjusted vehicle drive control decision tree model to generate a vehicle drive control decision tree model; Step S57: Design an intelligent analysis engine for vehicle intelligent drive control according to the vehicle drive control decision tree model to generate a vehicle intelligent drive control engine.

5. The vehicle intelligent drive control method for traffic scene analysis according to claim 4, characterized in that, Step S52 includes the following steps: Step S521: Extract visual object anomaly-related behavior features from the visual object prediction behavior data to generate visual object anomaly-related behavior feature data; Step S522: Perform visual object anomaly behavior-related probability analysis based on the visual object anomaly-related behavior feature data to generate visual object anomaly behavior-related probability data; Step S523: Screen the warning anomaly behavior-related probability data in the visual object anomaly behavior-related probability data based on a preset anomaly behavior warning threshold, and design an alternative priority vehicle drive decision according to the warning anomaly behavior-related probability data.

6. The vehicle intelligent drive control method for traffic scene analysis according to claim 1, characterized in that, The steps of transmitting the updated and optimized visual object behavior data to the vehicle intelligent drive control engine for vehicle intelligent drive control analysis in Step S6 include the following steps: Transmit the updated and optimized visual object behavior data to the vehicle intelligent drive control engine: Analyze the vehicle drive control behavior state of the updated and optimized visual object behavior data through the preferred priority vehicle drive decision of the vehicle intelligent drive control engine to generate vehicle drive control behavior state data; Perform vehicle drive control behavior state inference analysis on the updated and optimized visual object behavior data through the alternative priority vehicle drive decision of the vehicle intelligent drive control engine to generate vehicle drive control behavior state inference data; Perform vehicle drive control change state analysis on the vehicle drive control behavior state inference data based on a preset traffic scene limit change condition to generate vehicle drive control change state data; Perform vehicle intelligent drive control analysis based on the vehicle drive control change state data and the vehicle drive control behavior state data to generate intelligent vehicle drive control parameters.

7. A vehicle intelligent drive control system for traffic scene analysis, characterized in that, A vehicle intelligent drive control method for performing traffic scene analysis as described in claim 1, and the vehicle intelligent drive control system for this traffic scene analysis includes: A traffic scene monitoring module for performing traffic scene monitoring analysis based on the vehicle's on-vehicle sensors to generate traffic scene monitoring data, where the traffic scene monitoring data includes traffic scene visual data, traffic scene radar data, and traffic scene V2X data; A traffic scene visual object analysis module for performing traffic scene visual object analysis based on the traffic scene visual data to generate traffic scene visual object data; A visual object behavior analysis module for performing visual object behavior analysis on the traffic scene visual object data based on the traffic scene radar data to generate visual object behavior data; A visual object behavior trajectory trend optimization module for performing visual object behavior trajectory trend optimization on the visual object behavior data based on the traffic scene visual data and the traffic scene V2X data to generate optimized visual object behavior data; Vehicle intelligent drive control engine design module, which is used to obtain historical vehicle drive control data; perform intelligent analysis engine design of visual object behavior and vehicle drive control based on the historical vehicle drive control data, and generate a vehicle intelligent drive control engine; Intelligent vehicle drive control analysis module, which is used to perform an optimized visual object behavior monitoring and updating operation for the vehicle drive traffic scene according to the optimized visual object behavior data; collect and update the optimized visual object behavior data according to the optimized visual object behavior monitoring and updating operation of the vehicle drive traffic scene; transmit the updated optimized visual object behavior data to the vehicle intelligent drive control engine for vehicle intelligent drive control analysis, generate intelligent vehicle drive control parameters; and perform an intelligent vehicle drive control operation for the traffic scene based on the intelligent vehicle drive control parameters.

Images