Vehicle early warning method, device and equipment based on AI vision and medium

Abstract

The invention relates to an artificial intelligence technology, and discloses a vehicle early warning method and device based on AI vision, and the method comprises the steps: collecting a surrounding image of a current vehicle, recognizing a real-time vehicle driving data set, and building a real-time road model according to the surrounding image of a vehicle body, predicting a driving track set of surrounding vehicles according to the real-time vehicle driving data set, predicting a future driving track of the current vehicle, performing vehicle driving simulation in the real-time road model based on the driving track set and the future driving track, and if vehicle collision does not occur in the vehicle driving simulation, executing the vehicle driving simulation. And if yes, returning to the step of acquiring the surrounding images in real time by using the cameras arranged around the current vehicle in advance, and otherwise, generating early warning information according to the driving data corresponding to the collision vehicle in the real-time vehicle driving data set. The invention further provides a vehicle early warning device based on the AI vision, electronic equipment and a storage medium. According to the invention, the timeliness and accuracy of vehicle early warning can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a vehicle early warning method, device, equipment and medium based on AI vision. Background Art

[0002] With the acceleration of urbanization and the sharp increase in traffic volume, traffic safety issues have become increasingly prominent. Traditional vehicle warning methods are unable to cope with the complex and changing traffic environment. Frequent traffic accidents and severe traffic congestion have become important factors restricting urban development.

[0003] Most existing vehicle warning methods use infrared sensors to detect the distance to surrounding vehicles to determine whether there is a risk of collision. This method has low timeliness, making it difficult for users to respond in a timely manner, and the accuracy of the warning is affected in complex traffic environments. Summary of the invention

[0004] The present invention provides a vehicle warning method, device, equipment and medium based on AI vision, the main purpose of which is to solve the problem of low timeliness and low accuracy of vehicle warning in the prior art.

[0005] To achieve the above purpose, the present invention provides a vehicle early warning method based on AI vision, comprising:

[0006] The camera pre-set around the current vehicle is used to collect the image around the vehicle body in real time;

[0007] Identify vehicle information in the image around the vehicle body to obtain a real-time vehicle travel data set;

[0008] Establishing a real-time road model based on the image around the vehicle body;

[0009] Predicting the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set;

[0010] Generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory;

[0011] If no vehicle collision occurs in the vehicle driving simulation, returning to the step of collecting surrounding images in real time using cameras pre-set around the current vehicle;

[0012] If a vehicle collision occurs in the vehicle driving simulation, warning information is generated according to the driving data of the collision vehicle corresponding to the real-time vehicle driving data set.

[0013] Optionally, the identifying vehicle information in the image around the vehicle body to obtain a real-time vehicle travel data set includes:

[0014] Performing grayscale processing on each of the images around the vehicle body to obtain a set of grayscale images;

[0015] Performing edge detection on each grayscale image in the grayscale image set using an edge detection algorithm to obtain an edge contour of an object in each image;

[0016] Compare the object edge contour of each image with each vehicle edge contour in a pre-established vehicle contour database to obtain contour similarity;

[0017] If the contour similarity is greater than a preset similarity threshold, the position of the edge contour of the object is used as the vehicle position;

[0018] After confirming the vehicle positions of all images, a vehicle position image set is obtained;

[0019] By comparing the changes in the vehicle position in the continuous images in the vehicle position image set, a real-time vehicle travel data set is obtained.

[0020] Optionally, comparing the object edge contour information of each image with each vehicle edge contour in a pre-established vehicle contour database to obtain contour similarity includes:

[0021] Enlarging the contour information of the object identified in each image to a preset scale to obtain enlarged contour information;

[0022] The following formula is used to calculate the contour similarity between each enlarged contour information and each vehicle edge contour in the vehicle contour database:

[0023]

[0024] Wherein, N is the contour similarity, I 1 is the image matrix of the amplified contour information, μ 1 is the average value of the image matrix of the amplified contour information, i, j represents the horizontal coordinate and vertical coordinate of the pixel in the preset coordinate system, I 2 is the image matrix of the vehicle edge contour, μ 2 is the average value of the image matrix of the vehicle edge contour.

[0025] Optionally, establishing a real-time road model according to the vehicle body surrounding image includes:

[0026] Performing denoising processing on each of the images around the vehicle body to obtain a denoised image set;

[0027] Using an edge detection algorithm to identify road edge information in each denoised image in the denoised image set;

[0028] Obtaining road set data by continuously identifying road edge information of each denoised image in the denoised image set;

[0029] A real-time road model is established according to the road set data.

[0030] Optionally, the performing denoising processing on each of the vehicle body surrounding images includes:

[0031] Each image is denoised using the following formula:

[0032]

[0033] Wherein, L is the denoised image, M is the size of the preset mean filter, x and y are the coordinate points of the original image, I(x+i, y+j) represents the pixel value of the coordinate point (x+i, y+j) in the original image, and i and j represent the offset of the mean filter.

[0034] Optionally, the predicting driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set includes:

[0035] Extracting the time series characteristics of the real-time vehicle driving data to obtain driving time series characteristics;

[0036] Extracting local features of the driving time sequence features using a preset convolution kernel in a convolution layer of a pre-trained vehicle trajectory analysis model to obtain a convolution feature;

[0037] Using the pooling layer of the vehicle trajectory analysis model to extract the local average of the convolutional features, to obtain pooling features;

[0038] Using the activation function preset in the activation layer of the vehicle trajectory analysis model to add a nonlinear factor to the pooling feature, so as to obtain an activation feature;

[0039] The fully connected layer of the vehicle trajectory analysis model is used to output the predicted driving trajectory of each surrounding vehicle according to the activation feature to obtain a driving trajectory set.

[0040] Optionally, generating the future driving trajectory of the current vehicle according to the current driving speed of the vehicle includes:

[0041] Obtain the latest past travel route of the current vehicle with a preset travel length in the traveled path;

[0042] Using the past driving route as the future driving route of the current vehicle;

[0043] constructing checkpoints on the future driving route at preset time intervals based on the current vehicle speed;

[0044] The location information and time information corresponding to each checkpoint on the future driving route are integrated to obtain the future driving trajectory.

[0045] In order to solve the above problems, the present invention also provides a vehicle warning device based on AI vision, the device comprising:

[0046] A data acquisition module, used for acquiring images of the surroundings of the vehicle body in real time by using cameras pre-set around the current vehicle, and if no vehicle collision occurs in the vehicle driving simulation, returning to the step of acquiring images of the surroundings in real time by using cameras pre-set around the current vehicle;

[0047] A data recognition module, used to recognize vehicle information in the image around the vehicle body, obtain a real-time vehicle driving data set, and predict the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set;

[0048] A driving simulation module, used to establish a real-time road model according to the image around the vehicle body, generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory;

[0049] The collision warning module generates warning information according to the driving data corresponding to the collision vehicle in the real-time vehicle driving data set if a vehicle collision occurs in the vehicle driving simulation.

[0050] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0051] at least one processor;

[0052] and, a memory communicatively coupled to the at least one processor;

[0053] Among them, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned AI vision-based vehicle warning method.

[0054] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned AI vision-based vehicle warning method.

[0055] The embodiment of the present invention improves the efficiency of identifying vehicle driving data by using a camera pre-set around the current vehicle to collect images around the vehicle body in real time, identifies vehicle information in the images around the vehicle body, obtains a real-time vehicle driving data set, establishes a real-time road model based on the images around the vehicle body, predicts the driving trajectories of surrounding vehicles based on the real-time vehicle driving data set, obtains a driving trajectory set, improves the efficiency of subsequent prediction of driving trajectories, generates the future driving trajectory of the current vehicle based on the driving speed of the current vehicle, performs vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory, and if no vehicle collision occurs in the vehicle driving simulation, returns to the step of using the camera pre-set around the current vehicle to collect surrounding images in real time, and if a vehicle collision occurs in the vehicle driving simulation, generates warning information based on the driving data corresponding to the collision vehicle in the real-time vehicle driving data set, thereby improving the efficiency and accuracy of vehicle collision warning. Therefore, the vehicle warning method, device, electronic device and computer-readable storage medium based on AI vision proposed in the present invention can solve the problem of low timeliness and low accuracy in vehicle warning in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of a process flow of a vehicle early warning method based on AI vision provided by an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of a process for obtaining a vehicle driving data set provided by an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of a process for establishing a real-time road model provided by an embodiment of the present invention;

[0059] Figure 4 A functional module diagram of an AI vision-based vehicle warning device provided in one embodiment of the present invention;

[0060] Figure 5 A schematic diagram of the structure of an electronic device for implementing the AI ​​vision-based vehicle warning method provided in one embodiment of the present invention.

[0061] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0063] The embodiment of the present application provides a vehicle early warning method based on AI vision. The execution subject of the vehicle early warning method based on AI vision includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the vehicle early warning method based on AI vision can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0064] Reference Figure 1 FIG. 1 is a flow chart of a vehicle warning method based on AI vision according to an embodiment of the present invention. In this embodiment, the vehicle warning method based on AI vision includes:

[0065] S1. Using cameras pre-installed around the current vehicle to collect images around the vehicle body in real time.

[0066] In the embodiment of the present invention, the current vehicle refers to the vehicle driven by the user.

[0067] In the embodiment of the present invention, the cameras pre-arranged around the current vehicle may be arranged on the top of the current vehicle, the front and rear of the vehicle body, and the sides of the vehicle body.

[0068] In the embodiment of the present invention, the vehicle body surrounding image is a continuous image set based on a time series.

[0069] In the embodiment of the present invention, the real-time acquisition of the image around the vehicle body by using the camera pre-set around the current vehicle is performed according to a pre-set acquisition frequency, and the pre-set acquisition frequency may be once every 0.1 seconds.

[0070] In the embodiment of the present invention, since the image around the vehicle body is acquired in real time, the latest image data in the image around the vehicle body is continuously updated. At the same time, the earliest acquired image data in the image around the vehicle body is deleted so that the total amount of data in the image around the vehicle body remains unchanged.

[0071] In the embodiment of the present invention, by using the camera pre-set around the current vehicle to collect the image around the vehicle body in real time, the efficiency and accuracy of subsequent recognition of surrounding vehicle information are improved.

[0072] S2. Identify vehicle information in the image around the vehicle body to obtain a real-time vehicle driving data set.

[0073] In the embodiment of the present invention, the identifying of vehicle information in the image surrounding the vehicle body refers to identifying the number information of surrounding vehicles and the driving information of surrounding vehicles.

[0074] In the embodiment of the present invention, the vehicle information in the image around the vehicle body is identified by identifying the vehicle position in each image of the image around the vehicle body, and the vehicle's driving trajectory information is obtained by comparing the changes in the vehicle position in consecutive images.

[0075] In the embodiment of the present invention, refer to Figure 2 As shown, a schematic diagram of a process of obtaining a vehicle driving data set is provided in an embodiment of the present invention. The process of identifying vehicle information in the image around the vehicle body to obtain a real-time vehicle driving data set includes:

[0076] S21, performing grayscale processing on each of the images around the vehicle body to obtain a grayscale image set;

[0077] S22, performing edge detection on each grayscale image in the grayscale image set using an edge detection algorithm to obtain an edge contour of an object in each image;

[0078] S23, comparing the object edge contour of each image with each vehicle edge contour in a pre-established vehicle contour database to obtain contour similarity;

[0079] S24, if the contour similarity is greater than a preset similarity threshold, taking the position of the object edge contour as the vehicle position;

[0080] S25, after confirming the vehicle positions of all images, a vehicle position image set is obtained;

[0081] S26. Obtain a real-time vehicle travel data set by comparing changes in vehicle positions in consecutive images in the vehicle position image set.

[0082] In the embodiment of the present invention, the pre-established vehicle profile database is a database containing a large amount of profile data of vehicles of different types and at different angles.

[0083] In an embodiment of the present invention, the edge detection is performed on each grayscale image in the grayscale image set using an edge detection algorithm to obtain the object edge contour of each image. This is achieved by identifying the position in each grayscale image where the grayscale value is obviously different from the adjacent area, thereby obtaining the object edge contour of each image.

[0084] In detail, the object edge contour information of each image is compared with each vehicle edge contour in a pre-established vehicle contour database to obtain contour similarity, including:

[0085] Enlarging the contour information of the object identified in each image to a preset scale to obtain enlarged contour information;

[0086] The following formula is used to calculate the contour similarity between each enlarged contour information and each vehicle edge contour in the vehicle contour database:

[0087]

[0088] Wherein, N is the contour similarity, I 1 is the image matrix of the amplified contour information, μ 1 is the average value of the image matrix of the amplified contour information, i, j represents the horizontal coordinate and vertical coordinate of the pixel in the preset coordinate system, I 2 is the image matrix of the vehicle edge contour, μ 2 is the average value of the image matrix of the vehicle edge contour.

[0089] In detail, the enlarging the object contour information identified in each image to a preset scale refers to enlarging the object contour information identified in each image to the same scale as the vehicle edge contour in the vehicle contour database, so as to facilitate subsequent similarity calculation.

[0090] In detail, the image matrix of the enlarged contour information and the image matrix of the vehicle edge contour are both binary matrices, which are obtained by marking the pixels containing contour information in the enlarged contour information and the vehicle edge contour as 1 and the blank parts as 0.

[0091] In the embodiment of the present invention, the object edge contour information of each image is compared with each vehicle edge contour in the pre-established vehicle contour database, and the vehicle edge contour of the corresponding angle is selected from the vehicle contour database for comparison according to the location of the camera that acquires the image. For example, after the object contour in the image is identified, the vehicle edge contour at the side angle is selected from the vehicle contour database for comparison.

[0092] In the embodiment of the present invention, by identifying the vehicle information in the image around the vehicle body, a real-time vehicle driving data set is obtained, thereby improving the efficiency and accuracy of subsequent prediction of the driving trajectory of surrounding vehicles.

[0093] S3. Establishing a real-time road model according to the image around the vehicle body.

[0094] In the embodiment of the present invention, the real-time road model is a 3D model.

[0095] In the embodiment of the present invention, since the image around the vehicle body is continuously updated, the action of establishing the real-time road model is continuously performed, and the content of the real-time road model is also continuously updated. At the same time, the earliest established model part in the real-time road model is deleted so that the length of the real-time road model remains unchanged.

[0096] In the embodiment of the present invention, refer to Figure 3 As shown, a schematic diagram of a process of establishing a real-time road model is provided in an embodiment of the present invention. The process of establishing a real-time road model according to the image around the vehicle body includes:

[0097] S31, performing denoising processing on each of the images around the vehicle body to obtain a denoised image set;

[0098] S32, using an edge detection algorithm to identify road edge information in each denoised image in the denoised image set;

[0099] S33, obtaining road set data by continuously identifying road edge information of each denoised image in the denoised image set;

[0100] S34: Establish a real-time road model according to the road set data.

[0101] In detail, the denoising process is performed on each image in the vehicle body surrounding images, and mean filtering can be used to denoise each image.

[0102] In detail, the denoising process is performed on each of the vehicle body surrounding images, including:

[0103] Each image is denoised using the following formula:

[0104]

[0105] Wherein, L is the denoised image, M is the size of the preset mean filter, x and y are the coordinate points of the original image, I(x+i, y+j) represents the pixel value of the coordinate point (x+i, y+j) in the original image, and i and j represent the offset of the mean filter.

[0106] In detail, the preset mean filter is a simple linear filter used to remove noise in image or signal processing. It takes the average value of all points in a small area at each point of the image or signal to replace the value of the point, thereby achieving smoothing and denoising effects.

[0107] In detail, the road edge information in each denoised image in the denoised image set is identified by using an edge detection algorithm, and the road edge information is obtained by identifying a contour in the image that is similar to a preset road curvature.

[0108] In the embodiment of the present invention, by establishing a real-time road model according to the image around the vehicle body, the efficiency of subsequent driving simulation is improved.

[0109] S4. Predicting driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set.

[0110] In the embodiment of the present invention, the predicting of the driving trajectories of the surrounding vehicles based on the real-time vehicle driving data set is to predict the driving trajectories of the surrounding vehicles through a pre-trained vehicle trajectory analysis model.

[0111] In the embodiment of the present invention, the step of predicting the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set includes:

[0112] Extracting the time series characteristics of the real-time vehicle driving data to obtain driving time series characteristics;

[0113] Extracting local features of the driving time sequence features using a preset convolution kernel in a convolution layer of a pre-trained vehicle trajectory analysis model to obtain a convolution feature;

[0114] Using the pooling layer of the vehicle trajectory analysis model to extract the local average of the convolutional features, to obtain pooling features;

[0115] Using the activation function preset in the activation layer of the vehicle trajectory analysis model to add a nonlinear factor to the pooling feature, so as to obtain an activation feature;

[0116] The fully connected layer of the vehicle trajectory analysis model is used to output the predicted driving trajectory of each surrounding vehicle according to the activation feature to obtain a driving trajectory set.

[0117] In the embodiment of the present invention, before predicting the driving trajectory of each surrounding vehicle according to the time series features using the pre-trained vehicle trajectory analysis model, the method further includes:

[0118] Obtain a pre-prepared vehicle trajectory data set;

[0119] Using a first preset ratio of continuous frame images in the vehicle trajectory data set as training data, and a second preset ratio of continuous frame images as preset standard results of the training data;

[0120] Extracting the time series features of the training data to obtain training time series features;

[0121] Using a neural network model to predict the vehicle's driving trajectory according to the training time series characteristics to obtain a training prediction result;

[0122] Calculating a loss value between the training prediction result and a preset standard result of the training data;

[0123] If the loss value is greater than a preset loss value threshold and the training has not reached a preset number of iterations, the weight parameters and learning rate of the neural network model are updated, and the step of using the neural network model to predict the vehicle driving trajectory according to the training time series characteristics to obtain the training prediction result is re-executed;

[0124] If the loss value is less than or equal to the loss value threshold or the training reaches the number of iterations, the model training is confirmed to be completed and the vehicle trajectory analysis model is obtained.

[0125] In an embodiment of the present invention, the convolution kernel is a matrix whose scale is much smaller than the driving timing feature, and the extraction of the local features of the driving timing feature refers to using the convolution kernel to move on the driving timing feature according to a preset step size, calculating the dot product between the convolution kernel and the data of the coverage area after each movement, and using the calculated dot product as the feature of the coverage area to obtain the local features.

[0126] In an embodiment of the present invention, extracting the local average value of the convolution feature refers to moving the convolution feature using a preset rectangular serial port, calculating the average value of the data in the covered area after each movement, and using the calculated average value as the feature of the area.

[0127] In the embodiment of the present invention, the preset activation function includes but is not limited to a Sigmoid function, a Softmax function and a ReLU function.

[0128] In an embodiment of the present invention, the fully connected layer of the vehicle trajectory analysis model is used to output the predicted driving trajectory of each surrounding vehicle according to the activation feature, which means that the output of each neuron is classified and regressed according to the weight parameters and bias items of the model to obtain the final output value of the model.

[0129] In an embodiment of the present invention, the loss value is an indicator for measuring the difference between the model prediction and the actual target value, which can be obtained through a loss function preset in the neural network model. Common loss functions include mean square error loss function, cross entropy loss function and logarithmic loss function.

[0130] In the embodiment of the present invention, the weight parameters refer to the parameters connecting neurons. These parameters are used to adjust and learn the behavior of the model so as to effectively map the input data and extract useful features. The size of the weight parameters determines the degree of influence of the input signal during the propagation process in the network, that is, it determines the contribution of the input features to the model output.

[0131] In an embodiment of the present invention, the learning rate refers to a parameter used to control the weight update amplitude during the neural network training process. It determines the step size each time the weight parameters are updated according to the gradient descent. A larger learning rate can speed up the convergence speed, so that the model approaches the optimal solution faster. However, an excessively large learning rate may cause the model to oscillate near the optimal solution, or even fail to converge, resulting in an explosion of the loss function. A smaller learning rate can ensure that the model converges more stably near the optimal solution, but may cause the convergence speed to be too slow, requiring more iterations. The initial learning rate can be set to 0.001.

[0132] In the embodiment of the present invention, the driving trajectory of surrounding vehicles is predicted according to the real-time vehicle driving data set to obtain a driving trajectory set, thereby improving the accuracy of subsequent vehicle driving simulation.

[0133] S5. Generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory.

[0134] In the embodiment of the present invention, the future driving trajectory refers to the driving trajectory predicted for the current vehicle within a preset driving duration.

[0135] In the embodiment of the present invention, the step of generating the future driving trajectory of the current vehicle according to the current driving speed of the vehicle includes:

[0136] Obtain the latest past travel route of the current vehicle with a preset travel length in the traveled path;

[0137] Using the past driving route as the future driving route of the current vehicle;

[0138] constructing checkpoints on the future driving route at preset time intervals based on the current vehicle speed;

[0139] The location information and time information corresponding to each checkpoint on the future driving route are integrated to obtain the future driving trajectory.

[0140] In detail, the past driving route of the preset driving length is a driving route with the current position of the vehicle as the end point.

[0141] In detail, the preset driving length is the same as the driving trajectory length in the driving trajectory set.

[0142] In detail, the preset time interval needs to be set according to actual requirements. A too short time interval can improve the data accuracy of the future driving trajectory, but it will increase the amount of calculation. A too long time interval can reduce the amount of calculation, but it will reduce the data accuracy of the future driving trajectory. The preset time interval can be the same as the acquisition frequency, that is, the preset time interval can be 0.1 seconds.

[0143] In an embodiment of the present invention, the vehicle driving simulation based on the driving trajectory set and the future driving trajectory in the real-time road model refers to establishing models of the current vehicle and surrounding vehicles in the real-time road model, and the model size is set according to the preset vehicle size. The model of the current vehicle is simulated to drive in the real-time road model according to the future driving trajectory, and the models of the surrounding vehicles are simulated to drive in the real-time road model according to the formal trajectory in the driving trajectory set. Subsequently, it can be judged whether the current vehicle has a collision risk based on whether the model of the current vehicle collides with the models of the surrounding vehicles during the simulation process.

[0144] In an embodiment of the present invention, by generating the future driving trajectory of the current vehicle according to the current vehicle's driving speed, the efficiency of subsequent vehicle driving modeling is improved, and by performing vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory, the accuracy of judging whether the current vehicle has a collision risk is improved.

[0145] S6. Determine whether a vehicle collision occurs during the vehicle driving simulation.

[0146] If no vehicle collision occurs in the vehicle driving simulation, the process returns to S1, the step of collecting surrounding images in real time by using cameras pre-set around the current vehicle.

[0147] In an embodiment of the present invention, when no vehicle collision occurs in the vehicle driving simulation, it indicates that there is no risk of collision between the current vehicle and surrounding vehicles in a future time period. Therefore, it is necessary to continue to collect surrounding images and continuously update the driving trajectory set to achieve continuous simulation of the vehicle driving simulation.

[0148] If a vehicle collision occurs in the vehicle driving simulation, S7, a warning message is generated according to the driving data corresponding to the collision vehicle in the real-time vehicle driving data set.

[0149] In the embodiment of the present invention, when a vehicle collision occurs in the vehicle driving simulation, it indicates that there is a risk of collision between the current vehicle and surrounding vehicles, and therefore it is necessary to issue a warning message to the user.

[0150] In the embodiment of the present invention, the generating of warning information according to the driving data corresponding to the collision vehicle in the real-time vehicle driving data set is generating warning information according to the position information of the vehicle corresponding to the model that collides with the model of the current vehicle in the vehicle driving simulation. For example, in the vehicle driving simulation, if the model of the current vehicle collides with the model of the vehicle in front, the user is prompted to slow down and keep a distance from the vehicle in front.

[0151] In the embodiment of the present invention, after the warning information is generated according to the driving data corresponding to the real-time vehicle driving data set according to the collision vehicle, the warning information is displayed to the user using a display screen preset in the current vehicle.

[0152] like Figure 4 , which is a functional module diagram of an AI vision-based vehicle warning device provided in one embodiment of the present invention.

[0153] The AI ​​vision-based vehicle warning device 100 of the present invention can be installed in an electronic device. According to the functions implemented, the AI ​​vision-based vehicle warning device 100 can include a data acquisition module 101, a data recognition module 102, a driving simulation module 103 and a collision warning module 104. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0154] In this embodiment, the functions of each module / unit are as follows:

[0155] The data acquisition module 101 is used to collect the surrounding images of the vehicle body in real time by using the cameras pre-set around the current vehicle, and if no vehicle collision occurs in the vehicle driving simulation, return to the step of collecting the surrounding images in real time by using the cameras pre-set around the current vehicle;

[0156] The data recognition module 102 is used to recognize vehicle information in the image around the vehicle body, obtain a real-time vehicle driving data set, and predict the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set;

[0157] The driving simulation module 103 is used to establish a real-time road model according to the image around the vehicle body, generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory;

[0158] The collision warning module 104 generates warning information according to the driving data of the collision vehicle corresponding to the real-time vehicle driving data set if a vehicle collision occurs in the vehicle driving simulation.

[0159] In detail, each module described in the AI ​​vision-based vehicle warning device 100 in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The technical means are the same as the AI ​​vision-based vehicle warning method described in, and can produce the same technical effects, so I will not go into details here.

[0160] like Figure 5 , which is a schematic diagram of the structure of an electronic device of an AI vision-based vehicle warning method provided in an embodiment of the present invention.

[0161] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an AI vision-based vehicle warning program.

[0162] Among them, the processor 10 can be composed of an integrated circuit in some embodiments, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and various control chips. The processor 10 is the control core (ControlUnit) of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes the programs or modules stored in the memory 11 (such as AI vision-based vehicle warning programs, etc.), and calls the data stored in the memory 11 to execute various functions of the electronic device and process data.

[0163] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a vehicle warning program based on AI vision, but also can be used to temporarily store data that has been output or is to be output.

[0164] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0165] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in this embodiment, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0166] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0167] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0168] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0169] The AI ​​vision-based vehicle warning program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0170] The camera pre-set around the current vehicle is used to collect the image around the vehicle body in real time;

[0171] Identify vehicle information in the image around the vehicle body to obtain a real-time vehicle travel data set;

[0172] Establishing a real-time road model based on the image around the vehicle body;

[0173] Predicting the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set;

[0174] Generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory;

[0175] If no vehicle collision occurs in the vehicle driving simulation, returning to the step of collecting surrounding images in real time using cameras pre-set around the current vehicle;

[0176] If a vehicle collision occurs in the vehicle driving simulation, warning information is generated according to the driving data of the collision vehicle corresponding to the real-time vehicle driving data set.

[0177] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0178] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0179] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0180] The camera pre-set around the current vehicle is used to collect the image around the vehicle body in real time;

[0181] Identify vehicle information in the image around the vehicle body to obtain a real-time vehicle travel data set;

[0182] Establishing a real-time road model based on the image around the vehicle body;

[0183] Predicting the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set;

[0184] Generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory;

[0185] If no vehicle collision occurs in the vehicle driving simulation, returning to the step of collecting surrounding images in real time using cameras pre-set around the current vehicle;

[0186] If a vehicle collision occurs in the vehicle driving simulation, warning information is generated according to the driving data of the collision vehicle corresponding to the real-time vehicle driving data set.

[0187] In the embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0188] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0190] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0191] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0192] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0193] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A vehicle warning method based on AI vision, characterized in that: The method comprises: The camera pre-set around the current vehicle is used to collect the image around the vehicle body in real time; Identify vehicle information in the image around the vehicle body to obtain a real-time vehicle travel data set; Establishing a real-time road model based on the image around the vehicle body; Predicting the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set; Generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory; If no vehicle collision occurs in the vehicle driving simulation, returning to the step of collecting surrounding images in real time using cameras pre-set around the current vehicle; If a vehicle collision occurs in the vehicle driving simulation, warning information is generated according to the driving data of the collision vehicle corresponding to the real-time vehicle driving data set.

2. The vehicle warning method based on AI vision as claimed in claim 1, characterized in that: The identifying of the vehicle information in the image around the vehicle body to obtain a real-time vehicle travel data set includes: Performing grayscale processing on each of the images around the vehicle body to obtain a set of grayscale images; Performing edge detection on each grayscale image in the grayscale image set using an edge detection algorithm to obtain an edge contour of an object in each image; Compare the object edge contour of each image with each vehicle edge contour in a pre-established vehicle contour database to obtain contour similarity; If the contour similarity is greater than a preset similarity threshold, the position of the edge contour of the object is used as the vehicle position; After confirming the vehicle positions of all images, a vehicle position image set is obtained; By comparing the changes in the vehicle position in the continuous images in the vehicle position image set, a real-time vehicle travel data set is obtained.

3. The vehicle warning method based on AI vision as claimed in claim 2, characterized in that: The object edge contour information of each image is compared with each vehicle edge contour in a pre-established vehicle contour database to obtain contour similarity, including: Enlarging the contour information of the object identified in each image to a preset scale to obtain enlarged contour information; The following formula is used to calculate the contour similarity between each enlarged contour information and each vehicle edge contour in the vehicle contour database: Among them, N is the contour similarity, I1 is the image matrix of the enlarged contour information, μ1 is the average value of the image matrix of the enlarged contour information, i, j represent the horizontal coordinate and vertical coordinate of the pixel in the preset coordinate system, I2 is the image matrix of the vehicle edge contour, and μ2 is the average value of the image matrix of the vehicle edge contour.

4. The vehicle warning method based on AI vision as claimed in claim 1, characterized in that: The step of establishing a real-time road model according to the image around the vehicle body comprises: Performing denoising processing on each of the images around the vehicle body to obtain a denoised image set; Using an edge detection algorithm to identify road edge information in each denoised image in the denoised image set; Obtaining road set data by continuously identifying road edge information of each denoised image in the denoised image set; A real-time road model is established according to the road set data.

5. The vehicle warning method based on AI vision as claimed in claim 4, characterized in that: The denoising process is performed on each of the images around the vehicle body, including: Each image is denoised using the following formula: Wherein, L is the denoised image, M is the size of the preset mean filter, x and y are the coordinate points of the original image, I(x+i, y+j) represents the pixel value of the coordinate point (x+i, y+j) in the original image, and i and j represent the offset of the mean filter.

6. The vehicle warning method based on AI vision as claimed in claim 1, characterized in that: The step of predicting the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set includes: Extracting the time series characteristics of the real-time vehicle driving data to obtain driving time series characteristics; Extracting local features of the driving time sequence features using a preset convolution kernel in a convolution layer of a pre-trained vehicle trajectory analysis model to obtain a convolution feature; Using the pooling layer of the vehicle trajectory analysis model to extract the local average of the convolutional features, to obtain pooling features; Using the activation function preset in the activation layer of the vehicle trajectory analysis model to add a nonlinear factor to the pooling feature, so as to obtain an activation feature; The fully connected layer of the vehicle trajectory analysis model is used to output the predicted driving trajectory of each surrounding vehicle according to the activation feature to obtain a driving trajectory set.

7. The vehicle warning method based on AI vision as claimed in claim 1, characterized in that: The generating the future driving trajectory of the current vehicle according to the current driving speed of the vehicle comprises: Obtain the latest past travel route of the current vehicle with a preset travel length in the traveled path; Using the past driving route as the future driving route of the current vehicle; constructing checkpoints on the future driving route at preset time intervals based on the current vehicle speed; The location information and time information corresponding to each checkpoint on the future driving route are integrated to obtain the future driving trajectory.

8. A vehicle warning device based on AI vision, characterized in that: The device comprises: A data acquisition module, used for acquiring images of the surroundings of the vehicle body in real time by using cameras pre-set around the current vehicle, and if no vehicle collision occurs in the vehicle driving simulation, returning to the step of acquiring images of the surroundings in real time by using cameras pre-set around the current vehicle; A data recognition module, used to recognize vehicle information in the image around the vehicle body, obtain a real-time vehicle driving data set, and predict the driving trajectories of surrounding vehicles according to the real-time vehicle driving data set to obtain a driving trajectory set; A driving simulation module, used to establish a real-time road model according to the image around the vehicle body, generate a future driving trajectory of the current vehicle according to the current driving speed of the vehicle, and perform vehicle driving simulation in the real-time road model based on the driving trajectory set and the future driving trajectory; The collision warning module generates warning information according to the driving data corresponding to the collision vehicle in the real-time vehicle driving data set if a vehicle collision occurs in the vehicle driving simulation.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the AI ​​vision-based vehicle warning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the AI ​​vision-based vehicle warning method as described in any one of claims 1 to 7 is implemented.

Images