Intelligent driving assistance method, system and device, storage medium and vehicle
By acquiring the image of the vehicle ahead and using multi-layer models to predict position information and taillight status, the problem of low accuracy in prediction of the motion trajectory of the vehicle ahead in traditional technology is solved, achieving higher road safety.
Patent Information
- Application Number
- CN202411283468.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional intelligent driving assistance technology has low accuracy in predicting the motion trajectory of the vehicle ahead in an environment with large traffic, fast speed or poor vision, resulting in an increase in driver judgment errors and traffic accidents.
By obtaining the image of the vehicle in front, using the trained model to predict the vehicle position information, taillight status and motion trajectory, combined with the synergistic influence of multiple factors, the accuracy of motion trajectory prediction is improved and reminder information is generated to reduce accidents.
It improves the accuracy of predicting the movement trajectory of vehicles ahead, reduces the driver's judgment errors, reduces the risk of traffic accidents, and improves road driving safety.
Smart Images

Figure CN119068461A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent driving assistance method, system, device, storage medium and vehicle. Background Art
[0002] With the development of Internet of Vehicles and intelligent driving technology, many intelligent connected cars have the ability to actively make corresponding decisions based on the surrounding environment during the driving process, thereby assisting the driver to change the driving strategy and achieve safe driving. Traditional processing methods can only meet simple image recognition and detection. In the case of heavy traffic, high speed or poor visibility, the accuracy of predicting the movement trajectory of the vehicle in front is very low, which cannot achieve the effect of reducing the driver's misjudgment and reducing the occurrence of traffic accidents. Summary of the invention
[0003] In view of this, the purpose of the embodiments of the present invention is to provide an intelligent driving assistance method, system, device, storage medium and vehicle, which can improve the accuracy of predicting the motion trajectory of the vehicle ahead and improve road driving safety.
[0004] On the one hand, an embodiment of the present invention provides an intelligent driving assistance method, including:
[0005] Acquire a first image of a front vehicle; the first image includes the taillights and the rear of the front vehicle;
[0006] Predicting the position information of the vehicle ahead according to the first image and the trained first model;
[0007] Predicting a taillight state of the vehicle ahead according to the first image and the trained second model;
[0008] The movement trajectory of the vehicle ahead is predicted according to the position information, the taillight status and the trained third model, and the reminder information is determined according to the movement trajectory.
[0009] Optionally, predicting the position information of the front vehicle according to the first image and the trained first model includes:
[0010] Adjusting the size of the first image, predicting the taillight status of the front vehicle, retaining the rear of the front vehicle in the first image, and obtaining a second image;
[0011] The second image is input into the trained first model to predict the position information of the vehicle ahead.
[0012] Optionally, the first model includes a convolutional layer, a pooling layer, and a fully connected layer; inputting the second image into the trained first model to predict the position information of the front vehicle includes:
[0013] Inputting the second image into the convolutional layer to extract image features;
[0014] Inputting the extracted image features into the pooling layer to obtain key features;
[0015] The key features are input into the fully connected layer to predict the position information of the vehicle ahead.
[0016] Optionally, predicting the taillight state of the vehicle ahead according to the first image and the trained second model includes:
[0017] Adjusting the size of the first image, removing the portion of the first image except the taillights of the front vehicle, retaining the taillights of the front vehicle in the first image, and obtaining a third image;
[0018] The third image is input into the trained second model to predict the taillight status of the vehicle ahead.
[0019] Optionally, the second model includes a fully connected layer, a softmax layer and a classifier, and the inputting the third image into the trained second model to predict the taillight state of the vehicle ahead includes:
[0020] Inputting the third image into the fully connected layer to obtain a distribution matrix;
[0021] Inputting the distribution matrix into the softmax layer to obtain a distribution vector;
[0022] The distribution vector is input into the classifier to predict the taillight status of the leading vehicle.
[0023] Optionally, predicting the movement trajectory of the vehicle ahead according to the position information, the taillight state and a trained third model, and determining the reminder information according to the movement trajectory includes:
[0024] Inputting the position information and the taillight status into a trained third model to predict the motion trajectory of the vehicle ahead;
[0025] If the movement trajectory is safe, use the movement trajectory as reminder information;
[0026] If there is a safety risk in the motion trajectory, operation prompt information is generated according to the motion trajectory, and the operation prompt information is used as reminder information.
[0027] On the other hand, an embodiment of the present invention provides an intelligent driving assistance system, including:
[0028] A first module is used to obtain a first image of a front vehicle; the first image includes a taillight and a rear of the front vehicle;
[0029] A second module is used to predict the position information of the front vehicle according to the first image and the trained first model;
[0030] A third module is used to predict the taillight status of the vehicle ahead according to the first image and the trained second model;
[0031] The fourth module is used to predict the movement trajectory of the vehicle in front according to the position information, the taillight status and the trained third model, and determine the reminder information according to the movement trajectory.
[0032] On the other hand, an embodiment of the present invention provides an intelligent driving assistance device, including:
[0033] at least one processor;
[0034] at least one memory for storing at least one program;
[0035] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned intelligent driving assistance method.
[0036] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned intelligent driving assistance method.
[0037] On the other hand, an embodiment of the present invention provides a vehicle, which includes the above-mentioned intelligent driving assistance system or the above-mentioned intelligent driving assistance device.
[0038] Implementation of the embodiments of the present invention includes the following beneficial effects: first, a first image including the taillights and the rear of the vehicle in front is acquired; then, the position information of the vehicle in front is predicted based on the first image and the first model; and the taillight status of the vehicle in front is predicted based on the first image and the second model; and then, the motion trajectory of the vehicle in front is predicted based on the position information of the vehicle in front, the taillight status and the third model; and the motion trajectory of the vehicle in front is predicted based on the position information and the taillight status of the vehicle in front. That is, when predicting the motion trajectory of the vehicle in front, the multi-factor synergistic influence of the position information and the taillight status of the vehicle in front is simultaneously considered, thereby improving the accuracy of predicting the motion trajectory of the vehicle in front, reducing the driver's misjudgment, reducing traffic accidents, and improving road driving safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is a structural schematic diagram of an intelligent driving assistance system provided by an embodiment of the present invention;
[0040] Figure 2 is a flowchart of the steps of an intelligent driving assistance method provided by an embodiment of the present invention;
[0041] Figure 3 is a flowchart of another intelligent driving assistance method provided by an embodiment of the present invention;
[0042] Figure 4 It is a schematic diagram of a process flow of predicting the position information of a vehicle ahead provided by an embodiment of the present invention;
[0043] Figure 5 is a structural schematic diagram of a first model provided by an embodiment of the present invention;
[0044] Figure 6 is a flowchart of steps for predicting the state of the taillights of a vehicle ahead provided by an embodiment of the present invention;
[0045] Figure 7 is a structural schematic diagram of a second model provided by an embodiment of the present invention;
[0046] Figure 8 is a flowchart of steps for generating reminder information provided by an embodiment of the present invention;
[0047] Fig. 9 is a structural block diagram of another intelligent driving assistance system provided by an embodiment of the present invention;
[0048] Fig.10 It is a structural block diagram of an intelligent driving assistance device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0049] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0050] It should be noted that, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification and claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0052] One-Hot encoding is a commonly used feature encoding method, which is mainly used to convert the values of categorical variables into binary vector form. The basic principle is to establish a one-hot matrix for each categorical variable according to the value range, and then set the row vector corresponding to each value to 1 and the other row vectors to 0, so as to convert the categorical variable into a binary vector. For example, for a categorical variable with 3 values, it can be converted into 3 binary vectors, each vector corresponding to one value. One-Hot encoding expands the values of discrete features to Euclidean space, making the distance calculation between features more reasonable. By converting the values of each categorical variable into a binary vector, the features can be expanded to a certain extent, providing more information to the machine learning model.
[0053] See also Figure 1 The intelligent driving assistance system provided in this embodiment at least includes an image acquisition device and a processor. The image acquisition device is used to acquire real-time images of the vehicle in front; the processor is used to process data based on the real-time images acquired by the image acquisition device to obtain the accurate motion trajectory of the vehicle in front, and determine the reminder information based on the motion trajectory of the vehicle in front. The intelligent driving assistance system may also include hardware devices such as a display, a loudspeaker, or a warning light, such as displaying reminder information through a display, broadcasting reminder information through a loudspeaker, and reminding warning information through a warning light.
[0054] See also Figure 2 and Figure 3, an embodiment of the present invention provides an intelligent driving assistance method, used in a processor, comprising:
[0055] S100, acquiring a first image of a vehicle in front; the first image includes a taillight and a rear of the vehicle in front.
[0056] The first image of the vehicle in front is captured by an image capture device and sent to an image processing element, such as a processor. It should be noted that the image capture device is used to capture video or images of the vehicle in front, including but not limited to a camera or a still camera. The position of the image capture device is determined according to the actual application, and this embodiment does not make specific restrictions. For example, if the camera is installed in front of the roof of the vehicle, the capture range of the camera includes the range illuminated by the camera.
[0057] It should be noted that after obtaining the first image of the vehicle in front, the first image can be further denoised. Image denoising refers to the process of reducing noise in digital images. In reality, digital images are often affected by the interference of imaging equipment and external environmental noise during the digitization and transmission process, which is called a noisy image or a noisy image. Gaussian filtering is a linear smoothing filter that is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of image processing. Gaussian filtering is the process of weighted averaging the entire image. The value of each pixel is obtained by weighted averaging itself and other pixel values in the neighborhood. The specific operation of Gaussian filtering is: use a template (or convolution, mask) to scan each pixel in the image, and replace the value of the central pixel of the template with the weighted average gray value of the pixels in the neighborhood determined by the template.
[0058] For each frame of the first image of the vehicle in front, the image is convolved with a Gaussian filter template to perform image denoising. The GaussianBlur function in OpenCV is used in the specific implementation to obtain the image denoised by Gaussian filtering before the next step. In a specific embodiment, the image denoising process is as follows:
[0059] dst=cv2.GaussianBlur(src, ksize, sigmaX, sigmaY, borderType)
[0060] Among them, src represents the original unprocessed image; ksize represents the size of the Gaussian filter kernel. The larger the size, the greater the filtering degree and the greater the time complexity; sigmaX represents the standard deviation of the Gaussian kernel function in the X direction, and the specific value is determined by the Gaussian filter kernel; sigmaY represents the standard deviation of the Gaussian kernel function in the Y direction, and the specific value is determined by the Gaussian filter kernel; borderType uses the default value. dst represents the output processed image, and the output image dst has the same size as the input image src.
[0061] S200: predicting position information of a leading vehicle according to the first image and the trained first model.
[0062] It should be noted that before the first model is used to predict the position information of the vehicle ahead, it is necessary to use the first sample data for training, and the first sample data includes a sample image of the vehicle ahead and the position information of the vehicle ahead corresponding to the sample image. The condition for the end of the first model training is determined according to the actual application, and this embodiment does not impose specific restrictions, for example, the number of trainings reaches a preset number of times or the training accuracy reaches a preset accuracy. The input data of the first model is an image, and the output data of the first model is the position information of the vehicle. In order to improve the training accuracy of the first model, the sample images in the first sample data of the training are expanded, and key elements are extracted. The method of data expansion is determined according to the actual application, and this embodiment does not impose specific restrictions, such as data expansion methods include but are not limited to horizontal flipping, random cropping, color jitter technology, etc. The key elements are determined according to the actual application, and this embodiment does not impose specific restrictions, such as key elements including the tail range of the vehicle ahead and the range of the signal light, etc. The method for extracting key elements is determined according to the actual application, and this embodiment does not make any specific restrictions. For example, the SIFT (Scale-Invariant Feature Transform) feature extraction method is adopted. The SIFT feature extraction method is an image local feature extraction algorithm, which extracts features by finding the precise positioning and main direction of extreme points (feature points, key points) in different scale spaces and constructing key point descriptors. The key points extracted by SIFT are scale-invariant and rotation-invariant, and will not be disturbed by factors such as illumination, affine transformation and noise.
[0063] It should be noted that the position information of the vehicle in front is determined according to actual application, and this embodiment does not impose any specific restrictions. For example, the position information of the vehicle in front includes but is not limited to the coordinate information and distance information of the vehicle in front.
[0064] S300: predicting the state of the taillight of the vehicle ahead according to the first image and the trained second model.
[0065] It should be noted that before the second model is used to predict the taillight status of the vehicle ahead, it needs to be trained with the second sample data, and the second sample data includes a sample image of the vehicle ahead and the taillight status of the vehicle ahead corresponding to the sample image. The condition for the end of the second model training is determined according to the actual application, and this embodiment does not impose specific restrictions, for example, the number of trainings reaches a preset number of times or the training accuracy reaches a preset accuracy. The input data of the second model is an image, and the output data of the second model is the taillight status of the vehicle.
[0066] It should be noted that the taillight status of the vehicle ahead indicates the lighting condition of the headlights of the vehicle ahead. The taillight status of the vehicle ahead is determined according to actual application and is not specifically limited in this embodiment. For example, the taillight status of the vehicle ahead includes but is not limited to six states: no light on, left turn light on, right turn light on, brake light on, left turn and brake lights on at the same time, and right turn and brake lights on at the same time.
[0067] S400: predicting a movement trajectory of the vehicle ahead according to the position information, the state of the taillights and the trained third model, and determining reminder information according to the movement trajectory.
[0068] It should be noted that before the third model is used to predict the motion trajectory of the vehicle ahead, it is necessary to use the third sample data for training, and the third sample data includes the position information and taillight status of the vehicle ahead and the corresponding motion trajectory of the vehicle ahead. The conditions for the end of the third model training are determined according to the actual application, and this embodiment does not impose specific restrictions, for example, the number of trainings reaches a preset number of times or the accuracy of the training reaches a preset accuracy. The input data of the third model is the position information and the taillight status, and the output data of the third model is the motion trajectory of the vehicle.
[0069] See also Figure 4 , predicting the position information of the front vehicle according to the first image and the trained first model, including:
[0070] S210, adjusting the size of the first image, predicting the state of the taillights of the vehicle ahead, retaining the tail of the vehicle ahead in the first image, and obtaining a second image;
[0071] S220: Input the second image into the trained first model to predict the position information of the vehicle ahead.
[0072] The redundant factors in step S210 refer to factors other than the rear of the front vehicle in the first image, such as environmental information other than the front vehicle, color information of the front vehicle, lighting information of the front vehicle, etc. First, the first image is resized to a required size, and the redundant factors in the resized first image are removed, and only the rear part of the front vehicle in the first image is retained to obtain a second image; then, the second image is input into the trained first model to predict the position information of the front vehicle.
[0073] It should be noted that the specific structure of the first model is determined according to actual applications and is not specifically limited in this embodiment.
[0074] In a specific embodiment, Kalman filtering is used to predict the position information of the vehicle in front. Kalman filter is a highly efficient recursive filter. In target tracking applications, Kalman filtering describes the transfer of target state through the state transfer equation, uses the optimal value of the state at the previous moment (t-1) to predict the state value at the current moment (t), and uses the observed value at the current moment (t) to correct the predicted value at the current moment (t) to obtain the optimal value at the current moment (t). This process is continuously iterated, so that the predicted value is constantly close to the true value, thereby achieving accurate prediction of the position. Therefore, using a series of continuous images, the state of the vehicle in front at the next moment (t+1) can be predicted; after a large amount of data input and training, the model is corrected according to the prediction results, which can greatly improve the prediction accuracy. The predicted value refers to the physical quantity that describes the target state that you want to obtain, such as the position quantity (x, y, z) of the target of interest in target positioning. Since the target is usually in motion, the position of the target at time k (x[k], y[k], z[k]) is different from the position at time k-1 (x[k-1], y[k-1], z[k-1]). That is, the state of the target changes from time k-1 to time k. The state transfer equation is used in Kalman filtering to describe this change in the target state. The target position can be measured directly. For example, in the field of computer vision, the target position (x, y) can be directly obtained by using the target detection method. The state transfer equation and the measurement equation only provide an ideal physical relationship description. Due to the existence of noise, the estimated value of the target state predicted at the previous moment (k-1), that is, the predicted value, has a large deviation. Therefore, the observed value at the current moment (k) can be used to correct the predicted value. This observed value can provide new information for the estimation of the target state at the current moment. After correction, the predicted value is promoted to the optimal estimate.
[0075] Kalman filtering has the following advantages: it can be used to process noise and errors in positioning technology to improve positioning accuracy; it can be used to estimate hidden states in positioning technology, such as the speed and acceleration of the target object; it can be used to fuse information from multiple positioning technologies to improve positioning accuracy.
[0076] Optionally, the first model includes a convolution layer, a pooling layer, and a fully connected layer, and the second image is input into the trained first model to predict the position information of the front vehicle, including:
[0077] Input the second image into the convolution layer to extract image features;
[0078] Input the extracted image features into the pooling layer to obtain the key features;
[0079] The key features are input into the fully connected layer to predict the position information of the vehicle ahead.
[0080] See also Figure 5 In a specific embodiment, the first model includes a convolution layer, a pooling layer and a fully connected layer. The processed image of the vehicle in front is input as the input layer, and the input layer divides the input image into an n*n grid (depending on the resolution of the captured image), and can also be input as coordinates to the convolution layer. Convolution layer: The image is feature extracted through convolution operations to extract local features in the image. Each convolution layer is composed of several convolution units, and the parameters of each convolution unit are optimized through the back propagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first convolution layer may only be able to extract some low-level features such as edges, lines, and corners. More layers of the network can iteratively extract more complex features from low-level features.
[0081] Pooling layer: By performing dimensionality reduction and feature selection on the features output by the convolution layer, the dimension and size of the image are reduced, and the hierarchy of key features is retained. Max pooling and average pooling are the two main types, which extract the maximum value and average value in the region respectively. The pooling layer is placed after the convolution layer to reduce the amount of data and computational complexity, which helps to improve the model training speed and generalization ability, and prevent problems such as overfitting. The pooling layer consists of multiple filters, each of which performs a convolution operation on the input data to obtain an output feature map. The convolution operation of the pooling layer usually does not use weight parameters, but uses a fixed pooling function, such as max pooling, average pooling, etc. Max pooling refers to the pooling operation of the maximum value in each region of the input data to obtain the output feature map. Average pooling refers to the pooling operation of the average value in each region of the input data to obtain the output feature map.
[0082] Fully connected layer: The feature maps extracted by the convolution layer and the pooling layer are used for classification and regression prediction to obtain the results. The purpose of the first model of this structure after multiple convolution layers and pooling layers is to continuously reduce the dimension and size of the image, improve the accuracy, and reduce the amount of calculation and save storage space. The fully connected layer can perform matrix multiplication and bias addition operations on the connection weights between the input features and each neuron to obtain the output result. In the fully connected layer, each neuron is connected to all neurons in the previous layer, and each input feature has a certain connection weight with each neuron. During the training process, the neural network optimizes the weight and bias of each neuron through the back propagation algorithm, so that the output result can better fit the training data. The function of the fully connected layer is to map the input features to the output results. It is usually used in the last layer of the neural network for tasks such as classification and regression. The output result of the fully connected layer can be regarded as a nonlinear transformation of the input features. This transformation can map the input feature space to the output result space, thereby realizing the complexity and nonlinear fitting ability of the model. The number of parameters of the fully connected layer is very large, so it is easy to overfit. To avoid overfitting, some regularization methods can be used, such as Dropout, L1 / L2 regularization, etc. Overfitting refers to the phenomenon that a machine learning model performs well on the training data set, but performs poorly on new, unseen data sets.
[0083] In a specific embodiment, taking a 32*32*3 (3 is the number of channels, RGB three-channel color image) image as an example, a 5*5*3 convolution kernel is used for convolution, and finally a 28*28*1 image can be obtained; using maximum pooling is to select the maximum value of each channel to obtain an N-dimensional vector, thereby reducing the dimension while retaining the main parameters.
[0084] See also Figure 6 , predicting the taillight status of the vehicle ahead according to the first image and the trained second model, including:
[0085] S310, adjusting the size of the first image, removing the portion of the first image except the taillights of the front vehicle, retaining the taillights of the front vehicle in the first image, and obtaining a third image;
[0086] S320: Input the third image into the trained second model to predict the taillight status of the vehicle ahead.
[0087] The redundant factors in step S310 refer to factors other than the taillights of the vehicle in front in the first image, such as environmental information other than the vehicle in front, color information of the vehicle in front, outline information of the vehicle in front, etc. First, the first image is resized to a required size, and the redundant factors in the resized first image are removed, and only the taillight information of the vehicle in front in the first image is retained to obtain a third image; then, the third image is input into the trained second model to predict the taillight status of the vehicle in front.
[0088] It should be noted that the specific structure of the second model is determined according to actual applications and is not specifically limited in this embodiment.
[0089] Optionally, the second model includes a fully connected layer, a softmax layer and a classifier, and the third image is input into the trained second model to predict the taillight state of the vehicle ahead, including:
[0090] Input the third image into the fully connected layer to obtain a distribution matrix;
[0091] Input the distribution matrix into the softmax layer to obtain the distribution vector;
[0092] The distribution vector is input into the classifier to predict the taillight status of the vehicle ahead.
[0093] See also Figure 7 ,The second model includes a fully connected layer, a softmax layer and a classifier.
[0094] The image of the rear lights is used as the basic input, and a multi-label network structure is used to encode the labels in one-hot. Different weights are assigned according to the probability of the lights being on. Multiple softmax layers are used in the network output to improve the classification accuracy. After obtaining the output of the fully connected layer for the headlight classification, the softmax layer is entered. The softmax function is a commonly used activation function that converts the classification elements into probability values between 0 and 1. The softmax loss function is a measurement method to measure the difference between the predicted probability distribution and the true value distribution. It is used to optimize the training model and improve the classification accuracy through repeated training with optimization algorithms such as gradient descent.
[0095] It should be noted that the number of softmax layers after the fully connected layer is determined according to actual applications and is not specifically limited in this embodiment.
[0096] See also Figure 8 , predict the movement trajectory of the vehicle ahead based on the location information, taillight status and the trained third model, and determine the reminder information based on the movement trajectory, including:
[0097] S410: Input the position information and the taillight status into the trained third model to predict the movement trajectory of the vehicle ahead.
[0098] The position information of the vehicle in front and the state of the taillights jointly affect the motion trajectory of the vehicle in front. The motion trajectory of the vehicle in front includes but is not limited to forward straight-line motion, changing lanes to the left, changing lanes to the right, turning around in front or turning right, etc. The specific network results of the third model are determined according to the actual application, and this embodiment does not make specific restrictions. For example, the third model can be a graph neural network model, and each node in the graph neural network model has edges related to other nodes, which can be used to capture the interdependence between objects. The third model can also be a generative model, and the generative model includes a generative adversarial network and a conditional variational autoencoder. The generative adversarial network includes a generator and a discriminator. The generator is used to generate random samples similar to real samples, and the discriminator is used to judge the true or false of the data. Through the continuous game evolution of the generator and the discriminator, the generative adversarial network can obtain a generator with higher quality and a discriminator with stronger judgment ability. The autoencoder compresses the data into a low-dimensional vector representation through the encoder, and uses a decoder to decode the low-dimensional vector to obtain a reconstructed output, but the ability to generate data is very poor. The conditional variational autoencoder is a deep generative model for generating and reconstructing data, and the generation process can be controlled by conditional variables.
[0099] S420A: If the motion trajectory is safe, use the motion trajectory as reminder information.
[0100] The safe movement trajectory means that there is no risk of collision or friction between the vehicle ahead and the running vehicle, and the reminder information indicates important information that the driver needs to pay attention to. The reminder information includes but is not limited to the movement trajectory, movement speed and time of the vehicle ahead.
[0101] S420B: If there is a safety risk in the motion trajectory, generate operation prompt information according to the motion trajectory, and use the operation prompt information as reminder information.
[0102] Safety risk indicates the risk of collision or friction between the vehicle in front and the running vehicle. Operation prompt information indicates the safe operation method information that the driver can choose according to the movement status of the vehicle in front and the vehicle. The display method of prompt information is determined according to the actual application, and this embodiment does not make specific restrictions. For example, the prompt information is issued by voice broadcast, or displayed on a display screen, etc. The specific display method of the prompt information can be set by the driver or by the system default setting.
[0103] Implementation of the embodiments of the present invention includes the following beneficial effects: first, a first image including the taillights and the rear of the vehicle in front is acquired; then, the position information of the vehicle in front is predicted based on the first image and the first model; and the taillight status of the vehicle in front is predicted based on the first image and the second model; and then, the motion trajectory of the vehicle in front is predicted based on the position information of the vehicle in front, the taillight status and the third model; and the motion trajectory of the vehicle in front is predicted based on the position information and the taillight status of the vehicle in front. That is, when predicting the motion trajectory of the vehicle in front, the multi-factor synergistic influence of the position information and the taillight status of the vehicle in front is simultaneously considered, thereby improving the accuracy of predicting the motion trajectory of the vehicle in front, reducing the driver's misjudgment, reducing traffic accidents, and improving road driving safety.
[0104] See also Fig. 9 , an embodiment of the present invention provides an intelligent driving assistance system, including:
[0105] The first module is used to obtain a first image of a front vehicle; the first image includes a taillight and a rear of the front vehicle;
[0106] The second module is used to predict the position information of the front vehicle according to the first image and the trained first model;
[0107] A third module is used to predict the taillight status of the vehicle ahead according to the first image and the trained second model;
[0108] The fourth module is used to predict the movement trajectory of the vehicle ahead based on the position information, the taillight status and the trained third model, and determine the reminder information based on the movement trajectory.
[0109] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0110] See also Fig.10 , an embodiment of the present invention provides an intelligent driving assistance device, comprising:
[0111] at least one processor;
[0112] at least one memory for storing at least one program;
[0113] When at least one program is executed by at least one processor, the at least one processor implements the above-mentioned intelligent driving assistance method.
[0114] Among them, the memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. The memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a remote memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0115] It can be seen that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0116] In addition, the embodiment of the present application also discloses a computer program product or a computer program, which is stored in a computer-readable storage medium. The processor of the computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above method.
[0117] The embodiment of the present invention further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above method when executed by the processor. Similarly, the contents of the above method embodiment are all applicable to the present storage medium embodiment, and the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method embodiment.
[0118] It is understood that all or some steps and systems in the disclosed method above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0119] The embodiment further provides a vehicle, which includes an electric drive assembly of the above-mentioned intelligent driving assistance system (or device) or computer device (or electronic device). Specifically, the vehicle can be a private car, such as a sedan, SUV, MPV or pickup truck. The vehicle can also be an operating vehicle, such as a van, bus, small truck or large trailer. The vehicle can be a fuel vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it can be a hybrid vehicle or a pure electric vehicle.
[0120] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0123] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. An intelligent driving assistance method, It is characterized in that include: Acquire a first image of a front vehicle; the first image includes the taillights and the rear of the front vehicle; Predicting the position information of the vehicle ahead according to the first image and the trained first model; Predicting a taillight state of the vehicle ahead according to the first image and the trained second model; The movement trajectory of the vehicle ahead is predicted according to the position information, the taillight status and the trained third model, and the reminder information is determined according to the movement trajectory.
2. The method according to claim 1, It is characterized in that The predicting the position information of the front vehicle according to the first image and the trained first model includes: Adjusting the size of the first image, removing the portion of the first image except the rear of the front vehicle, retaining the rear of the front vehicle in the first image, and obtaining a second image; The second image is input into the trained first model to predict the position information of the vehicle ahead.
3. The method according to claim 2, It is characterized in that The first model includes a convolution layer, a pooling layer and a fully connected layer; inputting the second image into the trained first model to predict the position information of the front vehicle, including: Inputting the second image into the convolutional layer to extract image features; Inputting the extracted image features into the pooling layer to obtain key features; The key features are input into the fully connected layer to predict the position information of the vehicle ahead.
4. The method according to claim 1, It is characterized in that The predicting the taillight state of the vehicle ahead according to the first image and the trained second model includes: Adjusting the size of the first image, removing the portion of the first image except the taillights of the vehicle in front, retaining the taillights of the vehicle in front in the first image, and obtaining a third image; The third image is input into the trained second model to predict the taillight status of the vehicle ahead.
5. The method according to claim 4, It is characterized in that The second model includes a fully connected layer, a softmax layer and a classifier; the step of inputting the third image into the trained second model to predict the taillight state of the vehicle ahead includes: Inputting the third image into the fully connected layer to obtain a distribution matrix; Inputting the distribution matrix into the softmax layer to obtain a distribution vector; The distribution vector is input into the classifier to predict the taillight status of the leading vehicle.
6. The method according to claim 1, It is characterized in that The predicting the movement trajectory of the vehicle ahead according to the position information, the taillight state and the trained third model, and determining the reminder information according to the movement trajectory includes: Inputting the position information and the taillight status into a trained third model to predict the motion trajectory of the vehicle ahead; If the movement trajectory is safe, use the movement trajectory as reminder information; If there is a safety risk in the motion trajectory, operation prompt information is generated according to the motion trajectory, and the operation prompt information is used as reminder information.
7. An intelligent driving assistance system, It is characterized in that include: A first module is used to obtain a first image of a front vehicle; the first image includes a taillight and a rear of the front vehicle; A second module is used to predict the position information of the front vehicle according to the first image and the trained first model; A third module is used to predict the taillight status of the vehicle ahead according to the first image and the trained second model; The fourth module is used to predict the movement trajectory of the vehicle in front according to the position information, the taillight status and the trained third model, and determine the reminder information according to the movement trajectory.
8. An intelligent driving assistance device, It is characterized in that include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a processor-executable program stored therein, It is characterized in that The processor-executable program is used to perform the method according to any one of claims 1 to 6 when executed by the processor.
10. A vehicle, It is characterized in that The vehicle includes the intelligent driving assistance system as claimed in claim 7 or the intelligent driving assistance device as claimed in claim 8.
Citation Information
Patent Citations
Vehicle part feature-based vehicle detection and tracking method
CN102867416A
Vehicle trajectory predicting method based on hybrid dynamic bayesian networks and gaussian process
CN110304075A
System and method for vehicle taillight state recognition
US20190370574A1
