Vehicle-road collaborative automatic driving decision-making method and device based on artificial intelligence

Abstract

The embodiment of the invention relates to the technical field of automatic driving, and discloses a vehicle-road cooperation automatic driving decision optimization method and device based on artificial intelligence, and the method comprises the steps: building real-time information interaction between a vehicle and a traffic infrastructure through a 5G-A communication protocol, various environmental data including traffic light information and road congestion information are received and processed; image data and radar signal data are collected and preprocessed; performing feature extraction on the preprocessed image data by adopting a multi-layer large kernel convolution structure, and identifying and outputting a front vehicle and a state thereof in combination with radar signal data; and inputting the recognized front vehicle and the state thereof, the radar signal data and the environment data into a vehicle speed control model based on a deep neural network, analyzing the traffic flow and the vehicle state in real time, and adaptively adjusting the automatic driving decision strategy of the intelligent vehicle. According to the invention, the real-time decision-making accuracy and the driving safety of the automatic driving vehicle in the actual traffic environment are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an artificial intelligence-based vehicle-road collaborative autonomous driving decision optimization method and device. Background Art

[0002] With the acceleration of urbanization and the continuous increase in the number of cars, problems such as traffic congestion and traffic safety are becoming increasingly prominent. Traditional traffic management methods and vehicle driving modes are difficult to meet the needs of efficient and safe traffic. The development of intelligent vehicle technology has become one of the key directions to solve these problems. Among them, vehicle-road cooperative technology is considered to be an important development direction in the future field of traffic engineering and vehicle engineering. It can achieve a more intelligent and safer traffic system through mutual communication between vehicles and other traffic participants, infrastructure, and networks. In the vehicle-road cooperative environment, due to the large amount of data generated by the real-time image acquisition of the on-board camera, the traditional single-layer convolution structure will have a reduced ability to extract features from big data, thereby reducing the confidence and accuracy of image recognition. Therefore, the intelligent car may not be able to accurately identify obstacles or other traffic participants ahead in time, and the intelligent car may not slow down or stop in time, increasing the risk of traffic accidents.

[0003] Therefore, an intelligent technical solution is urgently needed to solve the above technical problems. Summary of the invention

[0004] One purpose of the present application is to provide a method and device for optimizing vehicle-road collaborative autonomous driving decisions based on artificial intelligence, so as to solve the technical problem that the current single-vehicle intelligent technology in the field of autonomous driving is not accurate enough in real-time intelligent driving decisions in actual traffic environments, and the intelligent vehicle may not be able to timely and accurately identify obstacles or other traffic participants ahead, resulting in untimely deceleration or stopping, increasing the risk of traffic accidents.

[0005] To achieve the above objectives, some embodiments of the present application provide the following aspects:

[0006] In a first aspect, some embodiments of the present application provide a vehicle-road cooperative autonomous driving decision optimization method based on artificial intelligence, including:

[0007] Establish real-time information exchange between vehicles and traffic infrastructure through 5G-Advanced communication protocols, and receive and process a variety of environmental data including traffic light information and road congestion information;

[0008] The image data and radar signal data are collected by the on-board camera and on-board radar of the smart car respectively;

[0009] Preprocessing the image data collected by the vehicle-mounted camera, wherein the preprocessing process includes sequentially performing Gaussian filtering denoising, sharpness enhancement and grayscale normalization processing;

[0010] A multi-layer large-kernel convolution structure is used to extract features from the preprocessed image data, and combined with the radar signal data, the vehicle ahead and its status are identified and output;

[0011] The identified front vehicle and its status, the radar signal data and the environmental data are input into a vehicle speed control model based on a deep neural network (DNN) structure to analyze traffic flow and vehicle status in real time and adaptively adjust the autonomous driving decision strategy of the smart car.

[0012] Furthermore,

[0013] Environmental data includes traffic light status S, road congestion status C, traffic participant information, traffic accidents, traffic incidents, road construction, and road maintenance;

[0014] The image data X is collected by the vehicle camera capture ;

[0015] The radar signal data collected by the vehicle-mounted radar includes the distance d of the front vehicle and the speed v of the front vehicle front 、Smart car speed v self .

[0016] Furthermore, the use of a multi-layer large-kernel convolution structure to extract features from the preprocessed image data and combining the radar signal data to identify and output the vehicle ahead and its status includes:

[0017] The preprocessed image data x is subjected to feature extraction through multi-layer large-kernel convolution operations to obtain a feature map cv(x);

[0018] The extracted feature map cv(x) is input into the classification head and the vehicle recognition result y is output.

[0019] Furthermore, the step of performing feature extraction on the preprocessed image data x through multi-layer large kernel convolution operations to obtain a feature map cv(x) includes:

[0020] cv(x)=Conv(Conv(Conv(x,ch,3),ch,3),4×reg max ,1)

[0021] The input is the preprocessed image data x, and the output is the feature map cv(x) after multi-layer large kernel convolution operation; ch is the number of input channels; reg max is the maximum number of parameters for bounding box regression; Conv is the large kernel convolution function.

[0022] Furthermore, the step of inputting the extracted feature map cv(x) into a classification head and outputting a vehicle recognition result y comprises:

[0023] y=[Concat(box,ReLU(cls)),Concat(box2,ReLU(cls2))]

[0024] box,cls=Split(Concat(Flatten(cv(x)),(reg max ×4,nc))

[0025] box2,cls2=Split(Concat(Flatten(cv(x)),(reg max ×4,nc))

[0026] Where: ReLU is the activation function; Flatten(A) = reshape(A, [N, -1]), where A is the input feature map, N is the number of samples, and the result is to flatten the multidimensional array into a two-dimensional array; Concat(P, Q) = [P, Q], where P and Q are the feature maps to be concatenated, and the result is to merge them in a specific dimension; Split(C, (d1, d2)) = (C1, C2), where C is the input feature map, d1 and d2 are the sizes to be split, and the result is to divide C into two parts C1 and C2; box is the bounding box coordinates obtained from the prediction, which is used to locate the position of the target; cls: classification result, indicating whether the target is contained in the anchor box and its category; box2 and cls2 are similar to box and cls, and are used for another set of anchor boxes or detection branches, used when processing multi-category targets; nc is the number of categories.

[0027] Furthermore, the identified front vehicle and its status, the radar signal data and the environmental data are input into a vehicle speed control model based on a deep neural network (DNN) structure, the traffic flow and vehicle status are analyzed in real time, and the autonomous driving decision strategy is adaptively adjusted, including:

[0028] According to the vehicle recognition result y, radar signal data (d, v front ,v self ) and environmental data (S, C) to control vehicle speed.

[0029] Furthermore, the identified front vehicle and its status, the radar signal data and the environmental data are input into a vehicle speed control model based on a deep neural network (DNN) structure, the traffic flow and vehicle status are analyzed in real time, and the autonomous driving decision strategy is adaptively adjusted, including:

[0030] Constructing a vehicle speed control model based on a deep neural network (DNN) structure; wherein the deep neural network (DNN) structure includes an input layer, a feature fusion layer, a feature extraction layer, a time series information processing layer and an output layer;

[0031] The input layer is used to receive the vehicle recognition result y and radar signal data (d, v front ,v self ) and environmental data;

[0032] Feature fusion layer, used to fuse different types of input data together using fully connected layers or concatenation operations;

[0033] Feature extraction layer, used to extract features using multiple fully connected layers or convolutional layers;

[0034] The time series information processing layer is used to process the radar signal data time series information in the input data using LSTM or GRU recurrent neural network;

[0035] The output layer is used to output the speed control data of the smart car using a fully connected layer, wherein the output layer uses a linear activation function or a Softmax activation function.

[0036] Furthermore, the training and optimization process of the vehicle speed control model includes the following process:

[0037] Performing data processing on the vehicle identification results, radar signal data and environmental data, wherein the data processing includes: normalizing, standardizing or encoding the input data; slicing the radar signal data according to the time step to form input-output pairs required for supervised learning;

[0038] Constructing training set data according to the processed data for training the vehicle speed control model;

[0039] Deploy the trained vehicle speed control model to the intelligent vehicle system for real-time vehicle speed control;

[0040] Monitor the performance of the vehicle speed control model in actual operation, and update or retrain the vehicle speed control model online as needed, and continuously optimize the model structure and parameters.

[0041] In a second aspect, some embodiments of the present application further provide a vehicle-road cooperative automatic driving decision optimization device based on artificial intelligence, including:

[0042] The vehicle-road collaboration module is used to establish real-time information interaction between vehicles and traffic infrastructure through the 5G-Advanced communication protocol, and receive and process a variety of environmental data including traffic light information and road congestion information;

[0043] The data acquisition module includes a vehicle-mounted camera and a vehicle-mounted radar, which are used to collect image data and radar signal data respectively;

[0044] The data preprocessing module preprocesses the image data collected by the vehicle camera, wherein the preprocessing process includes Gaussian filtering denoising, sharpness enhancement and grayscale normalization processing in sequence;

[0045] The vehicle recognition module uses a multi-layer large-kernel convolution structure to extract features from the preprocessed image data, and combines the radar signal data to identify and output the vehicle ahead and its status;

[0046] The decision optimization module is used to input the identified front vehicle and its status, the radar signal data and the environmental data into a vehicle speed control model based on a deep neural network (DNN) structure, analyze the traffic flow and vehicle status in real time, and adaptively adjust the autonomous driving decision strategy of the smart car.

[0047] Furthermore, the vehicle-road collaboration module also includes:

[0048] The communication protocol submodule is responsible for the implementation and maintenance of the 5G-Advanced communication protocol;

[0049] The data receiving and processing submodule is used to receive and pre-process information from the transportation infrastructure, including filtering, classifying and formatting data.

[0050] In a third aspect, some embodiments of the present application further provide an electronic device comprising: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor performs the steps of the method described in the first aspect.

[0051] Compared with the related art, the embodiment of the present application provides a method and device for optimizing vehicle-road cooperative automatic driving decision based on artificial intelligence. By proposing a multi-layer large-kernel convolution structure to extract features from the on-board camera image, and combining the radar signal data, it realizes the recognition of the vehicle in front and its status, and combines the on-board radar data and road environment data to obtain the final vehicle speed control decision. This multimodal data fusion method improves the accuracy and robustness of recognition. And by constructing a vehicle speed control model based on a deep neural network (DNN) structure, integrating various types of input data such as the status of the vehicle in front, radar signal data, and environmental data, the comprehensive data fusion helps the model to understand the driving environment more comprehensively, thereby making more accurate automatic driving decisions, ensuring the driving safety and comfort of the smart car in various situations. The present application significantly improves the automatic driving capability and driving safety of smart cars, and provides strong support for the development of vehicle-road cooperative automatic driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0053] Figure 1 A flowchart of a vehicle-road cooperative autonomous driving decision optimization method based on artificial intelligence provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a vehicle-road cooperative autonomous driving decision optimization method based on artificial intelligence provided in an embodiment of the present application;

[0055] Figure 3 A unit module diagram of a vehicle-road cooperative automatic driving decision optimization device based on artificial intelligence provided in an embodiment of the present application;

[0056] Figure 4 A subunit module diagram of the vehicle-road collaboration module provided in an embodiment of the present application;

[0057] Figure 5 An exemplary structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0059] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor represent the necessary logical order between them. It should also be understood that in the embodiments of the present disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, in the absence of explicit limitation or contrary revelation given in the context, it can generally be understood as one or more. In addition, the term "and / or" in the present disclosure is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are an "or" relationship. It should also be understood that the description of each embodiment in the present disclosure emphasizes the differences between the embodiments, and the same or similar parts can refer to each other. For the sake of brevity, they will not be repeated one by one.

[0060] At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. The techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices should be considered part of the specification. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The present application provides a vehicle-road cooperative automatic driving decision optimization method and device based on artificial intelligence, which recognizes the vehicle in front of the vehicle camera image by proposing a multi-layer large-core convolution structure, and combines the vehicle radar data and road environment data, and constructs a speed control model based on a deep neural network (DNN) structure to adaptively adjust the automatic driving decision strategy of the smart car. The speed control model integrates various types of input data such as the state of the vehicle in front, radar signal data, and environmental data. The comprehensive data fusion helps the model to understand the driving environment more comprehensively, so as to make more accurate automatic driving decisions, ensuring the driving safety and comfort of the smart car in various situations. As a result, the present invention improves the confidence and accuracy of image recognition, thereby improving the accuracy and formal safety of the real-time decision-making of the automatic driving vehicle in the actual traffic environment.

[0063] First embodiment

[0064] Figure 1 A flowchart of a vehicle-road cooperative automatic driving decision optimization method based on artificial intelligence provided in an embodiment of the present application. Figure 1 As shown, a vehicle-road cooperative automatic driving decision optimization method 100 based on artificial intelligence includes:

[0065] S110: Establishes real-time information exchange between vehicles and traffic infrastructure through 5G-Advanced communication protocols, and receives and processes a variety of environmental data including traffic light information and road congestion information;

[0066] Environmental data includes but is not limited to the signal light status S and the road congestion status C. For example, it may also include information about traffic participants, traffic accidents, traffic incidents, road construction, road maintenance, etc. Among them, the signal light status S is one of the important rules that must be followed by smart cars during driving. By receiving the signal light status data, the smart car can understand the traffic light status of the current intersection in real time, so as to make corresponding driving decisions, such as speeding up, slowing down and stopping, etc. The road congestion status C reflects the traffic flow and capacity of the current road. By receiving the road congestion status data, the smart car can timely understand the congestion of the road ahead, so as to choose a better driving route and avoid getting stuck in congestion.

[0067] In the above step S110, an efficient communication protocol is established between vehicles and traffic infrastructure based on 5G-A to achieve real-time sharing and feedback of information, including traffic light information, road congestion information, traffic participant information, etc. These data are crucial for smart cars because they directly reflect the current traffic conditions and driving rules, thereby building an intelligent traffic management system to improve overall traffic mobility. Through the 5G-Advanced communication protocol, real-time information interaction between vehicles and traffic infrastructure is achieved, allowing smart cars to instantly obtain and process a variety of environmental data including traffic light information and road congestion information. This greatly improves the perception and response speed of smart cars to the surrounding environment.

[0068] S120: collecting image data and radar signal data through a vehicle-mounted camera and a vehicle-mounted radar of the smart car respectively;

[0069] In step S120, the image data X is acquired by the vehicle-mounted camera. capture ; The radar signal data collected by the vehicle-mounted radar includes the distance d of the front vehicle and the speed v of the front vehicle front 、Smart car speed v self .

[0070] S130: Preprocessing the image data collected by the vehicle-mounted camera;

[0071] In step S130, the preprocessing includes Gaussian filtering denoising, sharpness enhancement and grayscale normalization processing to improve the image quality.

[0072] Specifically, the original image data X acquired in step S120 capture The preprocessing process includes the following steps:

[0073] S131: Gaussian filtering denoising:

[0074] Input original image X capture , output the denoised image X denoise :

[0075]

[0076] Among them: G(m,n) is a function of the offset (m,n), which is the kernel function of the Gaussian filter or called the Gaussian kernel. It represents the weighted average coefficient of each pixel value and the surrounding pixel values ​​during the Gaussian filtering process. (m,n) is the offset relative to the center pixel, specifically the horizontal and vertical offset relative to the center pixel; (i,j) represents the coordinate position of the pixel in the image; σ is the standard deviation of the Gaussian distribution, which controls the degree of blur.

[0077] By convolving the image with the Gaussian function, the influence of random noise in the image can be reduced, making the image smoother. The effect of Gaussian filtering depends on the size of the standard deviation σ. The larger the σ, the more obvious the filtering effect, but it may also cause the loss of image details. Therefore, it is necessary to select a suitable σ value to strike a balance between removing noise and retaining image details.

[0078] S132: Sharpness Enhancement

[0079] The image X after step S131 denoise As input, sharpness enhancement is performed and the sharpened image X is output sharp :

[0080] X sharp (i,j)=X denoise (i,j)+α·Lap(X denoise )(i,j)

[0081]

[0082] Among them: α is a control parameter used to adjust the strength of sharpness enhancement. The larger the value, the more obvious the sharpness enhancement effect is, and the details and edges of the image will be more strongly highlighted. The smaller the value, the weaker the sharpness enhancement effect is. The image will maintain a more natural appearance, but the details may not be as obvious as when using a larger value.

[0083] The above sharpness enhancement process aims to improve the clarity and contrast of the image and make the details and edges of the image more prominent. The sharpness of the image can be enhanced by calculating the Laplacian of the image and adding it to the original image. The control parameter α is used to adjust the intensity of the sharpness enhancement. The larger the α, the more obvious the sharpness enhancement effect, but it may also cause the image to be over-enhanced or artifacts. Therefore, the α value needs to be carefully selected to obtain the best sharpness enhancement effect.

[0084] S132: Grayscale Normalization

[0085] After step S132, the image X sharpTake grayscale normalization as input and output the normalized image x:

[0086]

[0087] Where: x(i,j) is the normalized image pixel value, (i,j) represents the coordinate position of the pixel in the image; μ x is the mean value of the image pixels; σ x is the standard deviation of the image pixel values; H is the number of rows (height) of the image, that is, the number of vertical pixels of the image; W is the number of columns (width) of the image, that is, the number of horizontal pixels of the image.

[0088] The above normalization process converts the grayscale value of the image to a standard range (usually 0 to 1 or -1 to 1) for better comparison and analysis. x and divided by the standard deviation σ x , the gray value distribution of the image can be standardized to make the brightness and contrast of the image more uniform. This helps to improve the visual effect of the image and improve the efficiency and accuracy of image processing.

[0089] S140: extracting features from the preprocessed image data using a multi-layer large-kernel convolution structure, and identifying and outputting the front vehicle and its status in combination with the radar signal data;

[0090] Specifically, step S140 includes:

[0091] S141: Perform feature extraction on the preprocessed image data x through multi-layer large kernel convolution operations to obtain a feature map cv(x), including:

[0092] cv(x)=Conv(Conv(Conv(x,ch,3),ch,3),4×reg max ,1)

[0093] The input is the preprocessed image data x, and the output is the feature map cv(x) after multi-layer large kernel convolution operation; ch is the number of input channels, which can be 4; reg max is the maximum number of parameters for bounding box regression, which can be 16. Of course, it can also be other numbers, which can be selected according to actual conditions; Conv is a large-kernel convolution function, and the convolution kernel is (7,7).

[0094] S142: Input the extracted feature map cv(x) into the classification head and output the vehicle recognition result y, which specifically includes:

[0095] y=[Concat(box,ReLU(cls)),Concat(box2,ReLU(cls2))]

[0096] box,cls=Split(Concat(Flatten(cv(x)),(reg max ×4,nc))

[0097] box2,cls2=Split(Concat(Flatten(cv(x))),(reg max ×4,nc))

[0098] Wherein: ReLU is the activation function; Flatten(A) = reshape(A, [N, -1]), where A is the input feature map, N is the number of samples, and the result is to flatten the multidimensional array into a two-dimensional array; Concat(P, Q) = [P, Q], where P and Q are the feature maps to be concatenated, and the result is to merge them in a specific dimension; Split(C, (d1, d2)) = (C1, C2), where C is the input feature map, d1 and d2 are the sizes to be split, and the result is to divide C into two parts C1 and C2; box is the bounding box coordinates obtained from the prediction, which is used to locate the position of the target; cls: classification result, indicating whether the target is contained in the anchor box and its category; box2 and cls2 are similar to box and cls, and are used for another set of anchor boxes or detection branches, and are used when processing multi-category targets; nc is the number of categories, which is specifically 80 in this embodiment.

[0099] In the above step S142, the output vehicle recognition result y includes whether there is a vehicle ahead and the type of vehicle. Specifically, the result y includes the following information:

[0100] Bounding box coordinates (box, box2): These coordinates are used to locate the position of the vehicle in front. They are usually given as points in the image coordinate system (such as the coordinates of the upper left and lower right corners) or the coordinates of the center point plus width and height. Box and box2 represent the results of different detection branches or anchor boxes, which are used to handle multi-category targets or improve the robustness of detection.

[0101] Classification result (cls, cls2): The classification result indicates whether the anchor box contains a target (i.e., a vehicle) and its category (such as a car, truck, etc.). cls and cls2 also represent the classification results of different detection branches or anchor boxes.

[0102] Information after feature map (cv(x)): Although cv(x) itself is not a direct component of the result y, it is a necessary feature representation for generating y. These feature maps contain key information such as the shape, texture, and edges of the vehicle in the image, which are used for subsequent vehicle recognition and state judgment.

[0103] Therefore, the output vehicle recognition result y includes the positioning information (bounding box coordinates) and classification information (vehicle type and existence) of the front vehicle, etc.

[0104] Furthermore, based on the vehicle recognition result y and combined with the radar signal data, the comprehensive result of the vehicle in front and its status is finally output, where the status information includes speed, distance, etc. obtained through radar data fusion. This information is crucial for the subsequent autonomous driving decision-making and control of the smart car, thereby improving the accuracy and robustness of recognition.

[0105] In step S140, the pre-processed image data is feature extracted by a multi-layer large-kernel convolution structure, and combined with the radar signal data, the front vehicle and its status are accurately identified. This multi-modal data fusion method improves the accuracy and robustness of identification.

[0106] S150: Utilize the identified front vehicle and its status, the radar signal data and the environmental data to input into a vehicle speed control model based on a deep neural network (DNN) structure, analyze traffic flow and vehicle status in real time, and adaptively adjust the autonomous driving decision strategy of the smart car to optimize driving safety and efficiency.

[0107] Specifically, in step S150, the traffic flow and vehicle status are analyzed in real time by using the identified front vehicle and its status, radar signal data, and the environmental data, and input into a vehicle speed control model based on a deep neural network (DNN) structure, and the autonomous driving decision strategy is adaptively adjusted, including:

[0108] S151: constructing a vehicle speed control model based on a deep neural network (DNN) structure; wherein the deep neural network (DNN) structure includes an input layer, a feature fusion layer, a feature extraction layer, a time series information processing layer and an output layer;

[0109] 1. Input layer: used to receive the following input data:

[0110] (1) The vehicle ahead and its status: including whether there is a vehicle ahead (Boolean value, represented by 1 or 0) and the type of vehicle (represented by one-hot encoding or embedding vector);

[0111] (2) Radar signal data: including the distance d (value) to the vehicle ahead and the speed v of the vehicle ahead front (value), smart car speed v self (value);

[0112] (3) Environmental data:

[0113] Traffic light status S (represented by one-hot encoding, such as red light, green light, yellow light, etc.).

[0114] Road congestion status C (expressed using numerical values ​​or classification codes, such as unimpeded traffic, slight congestion, severe congestion, etc.).

[0115] Traffic participant information (may include the location and speed of pedestrians and other vehicles, represented by vectors or matrices).

[0116] Traffic accidents, traffic incidents, road construction, road maintenance, etc. (represented by Boolean values ​​or categorical codes).

[0117] Since the input data are of different types and dimensions, proper preprocessing and encoding are required so that they can be uniformly input into the neural network.

[0118] 2. Feature fusion layer: used to fuse different types of input data together using a dense layer or concatenation operation; and use the attention mechanism to dynamically focus on the importance of different input data. This means that in complex driving scenarios, the model can automatically adjust the attention paid to various types of information to meet driving needs in different situations.

[0119] 3. Feature extraction layer: used to extract features using multiple fully connected layers or convolutional layers; and activation functions (such as ReLU) are used after each layer to increase nonlinearity. Using deep learning techniques such as fully connected layers, convolutional layers, and activation functions, the model can efficiently extract features from input data, which helps the model understand the driving environment more accurately and output more accurate vehicle speed control signals.

[0120] 4. Timing information processing layer: used to use LSTM or GRU recurrent neural network to process the timing information of radar signal data in the input data. This step can be placed before or after the feature extraction layer, depending on the importance of the timing information. The timing information processing layer can process the timing information in the input data, which is of great significance for understanding the changes of radar signal data over time and predicting future driving environments, and helps the model make more forward-looking autonomous driving decisions.

[0121] 5. Output layer: used to output the speed control data of the smart car using a fully connected layer, wherein the output layer uses a linear activation function (if the speed is a continuous value) or a Softmax activation function (if the speed is a classification value, such as low speed, medium speed, high speed, etc.).

[0122] For network parameters:

[0123] (1) Optimizer: Select an adaptive learning rate optimizer such as Adam or RMSprop.

[0124] (2) Loss function: According to the characteristics of the vehicle speed control task, you can choose mean square error (MSE), mean absolute error (MAE) or a custom loss function.

[0125] (3) Evaluation indicators: MAE, MSE, R 2 Scores are used to evaluate the predictive performance of the model.

[0126] (4) Batch size: Adjust according to the size of the dataset and computing resources, usually 32, 64, or 128.

[0127] (5) Number of training rounds: This number is determined by the model’s performance on the validation set and usually requires dozens to hundreds of rounds of training.

[0128] (6) Learning rate: The initial learning rate can be set to 0.001 and then adjusted according to the training situation.

[0129] (7) Regularization: Use L1, L2 regularization or Dropout to prevent overfitting.

[0130] S152: Model training and optimization, including the following processes:

[0131] S1521: performing data processing on the vehicle recognition result, radar signal data and environmental data, wherein the data processing includes: normalizing, standardizing or encoding the input data; slicing the radar signal data according to the time step to form input-output pairs required for supervised learning;

[0132] S1522: constructing training set data according to the processed data for training the vehicle speed control model;

[0133] S1523: deploying the trained vehicle speed control model to the intelligent vehicle system to perform real-time vehicle speed control;

[0134] S1524: Monitor the performance of the vehicle speed control model in actual operation, and update or retrain the vehicle speed control model online as needed, and continuously optimize the model structure and parameters.

[0135] This step S150 constructs a vehicle speed control model based on a deep neural network (DNN) structure, integrating various types of input data such as the state of the vehicle in front, radar signal data, and environmental data. The comprehensive data fusion helps the model to understand the driving environment more comprehensively, thereby making more accurate autonomous driving decisions. The model uses an adaptive learning rate optimizer and adjustable model parameters, so that the model can automatically adjust the learning rate and parameters according to the data characteristics during the training process to achieve better training results. At the same time, the model structure can also be optimized and adjusted according to actual needs to adapt to different driving scenarios and needs.

[0136] According to the above-mentioned embodiments of the present invention, the above-mentioned technical scheme significantly improves the autonomous driving capability and driving safety of smart vehicles through real-time information interaction, image data preprocessing, feature extraction and vehicle identification, adaptive decision optimization, and a vehicle speed control model based on a deep neural network (DNN) structure, providing strong support for the development of vehicle-road collaborative autonomous driving technology.

[0137] Second embodiment

[0138] Figure 3 This is a unit module diagram of a vehicle-road cooperative automatic driving decision optimization device based on artificial intelligence provided in the second embodiment of the present application. Figure 3 As shown, a vehicle-road cooperative automatic driving decision optimization device 200 based on artificial intelligence includes:

[0139] The vehicle-road collaboration module 210 is used to establish real-time information interaction between vehicles and traffic infrastructure through the 5G-Advanced communication protocol, and receive and process various environmental data including traffic light information and road congestion information;

[0140] The data acquisition module 220 includes a vehicle-mounted camera and a vehicle-mounted radar, which are used to collect image data and radar signal data respectively;

[0141] The data preprocessing module 230 preprocesses the image data collected by the vehicle-mounted camera, wherein the preprocessing process includes sequentially performing Gaussian filtering denoising, sharpness enhancement and grayscale normalization processing to improve image quality;

[0142] The vehicle identification module 240 uses a multi-layer large-kernel convolution structure to extract features from the pre-processed image data, and combines the radar signal data to identify and output the vehicle ahead and its status;

[0143] The decision optimization module 250 is used to input the identified front vehicle and its status, the radar signal data and the environmental data into a vehicle speed control model based on a deep neural network (DNN) structure, analyze traffic flow and vehicle status in real time, and adaptively adjust the autonomous driving decision strategy of the smart car.

[0144] Further, in some embodiments, Figure 4 As shown, the vehicle-road collaboration module 210 also includes:

[0145] The communication protocol submodule 211 is responsible for the implementation and maintenance of the 5G-Advanced communication protocol;

[0146] The data receiving and processing submodule 212 is used to receive and pre-process information from the traffic infrastructure, including filtering, classifying and formatting data.

[0147] In this embodiment, the communication protocol submodule 211 is a core component of the vehicle-road collaboration module 210, and is mainly responsible for implementing and maintaining the 5G-Advanced communication protocol. This submodule ensures that a stable, efficient and secure communication connection can be established between the vehicle and the traffic infrastructure. It contains all necessary protocol stacks related to 5G-Advanced technology, such as the physical layer, data link layer, network layer, and application layer, to support real-time information interaction between vehicles and infrastructure. The data reception and processing submodule 212 is a key component in the vehicle-road collaboration module 210 that is responsible for receiving and pre-processing information from the traffic infrastructure. This submodule can receive a variety of environmental data including traffic light information, road congestion information, traffic participant information, etc., and filter, classify and format these data for subsequent analysis and use.

[0148] It is not difficult to find that this embodiment is a device embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the first embodiment.

[0149] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0150] Third embodiment

[0151] Figure 5 This is a schematic diagram of an exemplary structure of an electronic device provided in the third embodiment of the present application. Figure 5 As shown, an electronic device, which may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device may also be various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0152] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the steps of the method provided in any one or more of the above embodiments. Figure 5 An exemplary structural diagram of the electronic device is disclosed. Figure 5 As shown, the electronic device includes: one or more processors 1101, memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0153] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 2 The example of connecting through bus is taken in the following.

[0154] The input device 1103 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1104 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0155] To provide interaction with a user, the electronic device may be a computer. The computer has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0156] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium, and when the computer program / instruction is executed by a processor, the steps of the method provided by any one or more of the above embodiments are implemented. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0157] The memory 1102 can be used as a non-transient computer-readable storage medium, which can be used to store non-transient software programs, non-transient computer executable programs and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 1102, so as to implement the program instructions / modules corresponding to the method provided by any one or more embodiments in the embodiments of the present application.

[0158] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely arranged relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0160] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0161] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] In the above-described embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. For example, it can be implemented by using an application specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented by hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0163] The computer program product provided in the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0164] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0165] The scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim may also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are only used to distinguish the description, and do not indicate any particular order, nor can they be understood as indicating or implying relative importance.

[0166] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily mention changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-restrictive.

Claims

1. A vehicle-road cooperative automatic driving decision optimization method based on artificial intelligence, characterized in that: include: Establish real-time information exchange between vehicles and traffic infrastructure through 5G-Advanced communication protocols, and receive and process a variety of environmental data including traffic light information and road congestion information; The image data and radar signal data are collected by the on-board camera and on-board radar of the smart car respectively; Preprocessing the image data collected by the vehicle-mounted camera, wherein the preprocessing process includes sequentially performing Gaussian filtering denoising, sharpness enhancement and grayscale normalization processing; A multi-layer large-kernel convolution structure is used to extract features from the preprocessed image data, and combined with the radar signal data, the vehicle ahead and its status are identified and output; The identified front vehicle and its status, the radar signal data and the environmental data are input into a vehicle speed control model based on a deep neural network (DNN) structure to analyze traffic flow and vehicle status in real time and adaptively adjust the autonomous driving decision strategy of the smart car.

2. The method according to claim 1, characterized in that in, Environmental data includes traffic light status S, road congestion status C, traffic participant information, traffic accidents, traffic incidents, road construction, and road maintenance; The image data X is collected by the vehicle camera capture ; The radar signal data collected by the vehicle-mounted radar includes the distance d of the front vehicle and the speed v of the front vehicle front 、Smart car speed v self .

3. The method according to claim 2, characterized in that in, The multi-layer large-kernel convolution structure is used to extract features from the pre-processed image data, and combined with the radar signal data, the front vehicle and its status are identified and output, including: The preprocessed image data x is subjected to feature extraction through multi-layer large-kernel convolution operations to obtain a feature map cv(x); The extracted feature map cv(x) is input into the classification head and the vehicle recognition result y is output.

4. The method according to claim 3, characterized in that in, The feature extraction of the preprocessed image data x is performed through multi-layer large kernel convolution operations to obtain a feature map cv(x), including: cv(x)=Conv(Conv(Conv(x,ch,3),ch,3),4×reg max ,1) The input is the preprocessed image data x, and the output is the feature map cv(x) after multi-layer large kernel convolution operation; ch is the number of input channels; reg max is the maximum number of parameters for bounding box regression; Conv is the large kernel convolution function.

5. The method according to claim 4, characterized in that in, The extracted feature map cv(x) is input into the classification head, and the vehicle recognition result y is output, including: y=[Concat(box,ReLU(cls)),Concat(box2,ReLU(cls2))] box,cls=Split(Concat(Flatten(cv(x)),(reg max ×4,nc)) box2,cls2=Split(Concat(Flatten(cv(x))),(reg max ×4,nc)) Where: ReLU is the activation function; Flatten(A) = reshape(A, [N, -1]), where A is the input feature map, N is the number of samples, and the result is to flatten the multidimensional array into a two-dimensional array; Concat(P, Q) = [P, Q], where P and Q are the feature maps to be concatenated, and the result is to merge them in a specific dimension; Split(C, (d1, d2)) = (C1, C2), where C is the input feature map, d1 and d2 are the sizes to be split, and the result is to divide C into two parts C1 and C2; box is the bounding box coordinates obtained from the prediction, which is used to locate the position of the target; cls: classification result, indicating whether the target is contained in the anchor box and its category; box2 and cls2 are similar to box and cls, and are used for another set of anchor boxes or detection branches, used when processing multi-category targets; nc is the number of categories.

6. The method according to claim 5, characterized in that in, The identified front vehicle and its status, the radar signal data and the environmental data are input into a vehicle speed control model based on a deep neural network (DNN) structure, the traffic flow and vehicle status are analyzed in real time, and the autonomous driving decision strategy is adaptively adjusted, including: Constructing a vehicle speed control model based on a deep neural network (DNN) structure; wherein the deep neural network (DNN) structure includes an input layer, a feature fusion layer, a feature extraction layer, a time series information processing layer and an output layer; The input layer is used to receive the vehicle recognition result y and radar signal data (d, v front ,v self ) and environmental data; Feature fusion layer, used to fuse different types of input data together using fully connected layers or concatenation operations; Feature extraction layer, used to extract features using multiple fully connected layers or convolutional layers; The time series information processing layer is used to process the radar signal data time series information in the input data using LSTM or GRU recurrent neural network; The output layer is used to output the vehicle speed control data of the smart car using a fully connected layer, wherein the output layer uses a linear activation function or a Softmax activation function.

7. The method according to claim 6, characterized in that in, The training and optimization process of the vehicle speed control model includes the following steps: Performing data processing on the vehicle identification results, radar signal data and environmental data, wherein the data processing includes: normalizing, standardizing or encoding the input data; slicing the radar signal data according to the time step to form input-output pairs required for supervised learning; Constructing training set data according to the processed data for training the vehicle speed control model; Deploy the trained vehicle speed control model to the intelligent vehicle system for real-time vehicle speed control; The performance of the vehicle speed control model in actual operation is monitored, and the vehicle speed control model is updated or retrained online as needed, and the model structure and parameters are continuously optimized.

8. An artificial intelligence-based vehicle-road cooperative automatic driving decision optimization device, characterized in that include: The vehicle-road collaboration module is used to establish real-time information interaction between vehicles and traffic infrastructure through the 5G-Advanced communication protocol, and receive and process a variety of environmental data including traffic light information and road congestion information; The data acquisition module includes a vehicle-mounted camera and a vehicle-mounted radar, which are used to collect image data and radar signal data respectively; The data preprocessing module preprocesses the image data collected by the vehicle camera, wherein the preprocessing process includes Gaussian filtering denoising, sharpness enhancement and grayscale normalization processing in sequence; The vehicle recognition module uses a multi-layer large-kernel convolution structure to extract features from the preprocessed image data, and combines the radar signal data to identify and output the vehicle in front and its status; The decision optimization module is used to input the identified front vehicle and its status, the radar signal data and the environmental data into a vehicle speed control model based on a deep neural network (DNN) structure, analyze the traffic flow and vehicle status in real time, and adaptively adjust the autonomous driving decision strategy of the smart car.

9. The device according to claim 8, characterized in that The vehicle-road collaboration module also includes: The communication protocol submodule is responsible for the implementation and maintenance of the 5G-Advanced communication protocol; The data receiving and processing submodule is used to receive and pre-process information from the transportation infrastructure, including filtering, classifying and formatting data.

10. An electronic device, comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method according to any one of claims 1 to 7.

Images