Method, device and medium for rapid image processing during intelligent driving

Abstract

The present invention relates to image processing technology and discloses a method for rapid image processing during intelligent driving, comprising: performing targeted analysis on forward-view image data, surround-view image data, and side-view image data collected during the driving process of a target vehicle to obtain road marking data, surrounding vehicle data, and road pedestrian data; converting the road marking data, surrounding vehicle data, and road pedestrian data into coordinate data; performing rapid three-dimensional spatial modeling based on the coordinate data to obtain an intelligent driving spatial model; generating driving recommendations based on the intelligent driving spatial model; performing three-dimensional coarse modeling based on the collected wide-angle image data to obtain an intelligent driving coarse spatial model; performing real-time comparison between the intelligent driving spatial model and the coarse intelligent driving spatial model to obtain a model error; and adaptively broadcasting driving recommendations based on the model error. The present invention also provides a device and medium for rapid image processing during intelligent driving. The present invention can improve the efficiency of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, and medium for rapid image processing during intelligent driving. Background Art

[0002] With the continued rise in car ownership and increasing demands for safer and more convenient travel, intelligent driving technology is gradually moving from concept to reality, becoming a core driver of change in the automotive industry and the entire transportation sector. Intelligent driving systems aim to enable autonomous or assisted driving through a series of advanced sensors, algorithms, and electronic control units, significantly reducing human error, lowering the risk of traffic accidents, and improving traffic efficiency.

[0003] Image recognition and processing technology, as a key component of intelligent driving, plays a vital role. Intelligent vehicles are typically equipped with multiple cameras, located in various locations, to capture comprehensive visual information surrounding the vehicle. The massive amounts of image data collected by these cameras in real time must be efficiently processed in a very short period of time to provide timely and accurate information for vehicle decision-making and control. For example, when a vehicle is traveling at high speed, the forward-facing camera may capture dozens or even hundreds of images per second. If these images cannot be rapidly analyzed to identify key elements such as the road ahead, vehicles, pedestrians, and traffic signs, the vehicle will be unable to make reasonable driving decisions such as acceleration, deceleration, and steering, thereby endangering driving safety.

[0004] Currently, traditional image processing methods have exposed numerous limitations in intelligent driving scenarios. For one thing, existing image algorithms are mostly based on general-purpose computer vision technology and are insufficiently optimized for the specific needs of intelligent driving. When processing high-resolution images under complex road conditions, the high computational complexity leads to slow processing speeds and difficulty meeting real-time requirements. For example, on congested urban roads, where vehicles, pedestrians, and non-motorized vehicles coexist and road signs appear frequently, traditional algorithms often take more than tens of milliseconds to process forward-view images. Intelligent driving systems require processing time per frame to be controlled within a few milliseconds to ensure timely vehicle response. Summary of the Invention

[0005] The present invention provides a method, device and medium for rapid image processing during intelligent driving, the main purpose of which is to solve the problem of low response efficiency of existing image processing methods in intelligent driving scenarios.

[0006] To achieve the above objectives, the present invention provides a method for rapid image processing during intelligent driving, comprising:

[0007] Real-time collection of front view image data, surround view image data and side view image data of the target vehicle during its driving process;

[0008] Performing targeted analysis of road markings based on the forward-view image data to obtain road marking data;

[0009] Performing targeted analysis of surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data;

[0010] Performing targeted recognition of pedestrians on the road based on the side view image data to obtain pedestrian data on the road;

[0011] Converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data, and performing three-dimensional space rapid modeling based on the comprehensive coordinate data to obtain an intelligent driving space model;

[0012] generating driving recommendations based on the intelligent driving space model;

[0013] collecting wide-angle image data of the target vehicle in real time, performing three-dimensional rough modeling based on the wide-angle image data set, and obtaining a rough intelligent driving spatial model;

[0014] Comparing the intelligent driving space model with the intelligent driving rough space model in real time to obtain a model error;

[0015] Determining whether the model error is greater than or equal to a preset error threshold;

[0016] If the model error is less than the error threshold, announcing the driving advice to the user;

[0017] If the model error is greater than or equal to the error threshold, the driving advice is adjusted in real time according to the model error, and then the process returns to the step of broadcasting the driving advice to the user.

[0018] Optionally, performing targeted analysis of road markings based on the forward-view image data to obtain road marking data includes:

[0019] Segmenting the front view image using a pre-trained lightweight road segmentation model to obtain a front view road image;

[0020] Performing color enhancement on the forward road image using the HSV color space to obtain an enhanced road image;

[0021] Performing edge contour detection on the enhanced road image to obtain line marking contour data;

[0022] Detecting straight lines and curves in the enhanced road image based on Hough transform to obtain shape data;

[0023] The road markings in the forward-view road image are located based on the road marking contour data and the shape data to obtain road marking data.

[0024] Optionally, the segmenting the front view image using a pre-trained lightweight road segmentation model to obtain a front view road image includes:

[0025] performing normalization processing on the front view image to obtain a normalized front view image;

[0026] Performing dimensional conversion processing on the normalized front view image to obtain a converted image;

[0027] Inferring the transformed image by calling the forward propagation function of the lightweight road segmentation model to obtain a probability distribution tensor;

[0028] The probability distribution tensor is converted into a two-dimensional array based on a preset probability threshold, and the front view image is segmented using the two-dimensional array to obtain a front view road image.

[0029] Optionally, performing targeted analysis on surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data includes:

[0030] Performing vehicle target recognition on the surround view image data to obtain target recognition data;

[0031] Tracking the positions of vehicles in the surround view image data based on the target recognition data using a multi-target tracking algorithm to obtain vehicle trajectories;

[0032] Performing short-term trajectory prediction on the vehicle in the surround view image data based on the vehicle trajectory to obtain a predicted trajectory;

[0033] The target recognition data, the vehicle trajectory, and the predicted trajectory are aggregated to obtain the surrounding vehicle data.

[0034] Optionally, converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data includes:

[0035] Obtaining a first marking distance and a second marking distance included in the road marking data, and calculating a marking position according to the first marking distance, the second marking distance, and a preset front view camera distance;

[0036] Obtaining a first vehicle distance and a second vehicle distance included in the surrounding vehicle data, and calculating a vehicle position based on the first vehicle distance, the second vehicle distance, and a preset surround view camera distance;

[0037] Obtaining a first distance and a second distance of a pedestrian included in the road pedestrian data, and calculating a position of the pedestrian based on the first distance, the second distance, and a preset side view camera distance;

[0038] Establishing a two-dimensional rectangular coordinate system parallel to the horizontal plane with the target vehicle as the origin;

[0039] The marking position, the vehicle position, and the pedestrian position are converted into coordinate data based on the two-dimensional rectangular coordinate system to obtain comprehensive coordinate data.

[0040] Optionally, the real-time acquisition of wide-angle image data of the target vehicle and the performing of three-dimensional coarse modeling based on the wide-angle image data set to obtain a rough intelligent driving spatial model include:

[0041] performing wide-angle distortion correction processing on the wide-angle image data to obtain a corrected image;

[0042] Performing target object feature extraction on the corrected image to obtain target object features;

[0043] The target object features are used to perform fast approximate feature matching to obtain target recognition data.

[0044] Acquire the target object in the corrected image according to the target recognition data using a three-dimensional reconstruction algorithm to obtain the three-dimensional coordinates of the target object;

[0045] Three-dimensional modeling is performed based on the three-dimensional coordinates of the target object to obtain a rough spatial model of intelligent driving.

[0046] Optionally, the comparing the intelligent driving space model with the intelligent driving rough space model in real time to obtain a model error includes:

[0047] The number of surrounding vehicles and the number of surrounding pedestrians in the intelligent driving space model are obtained, and the rough number of surrounding vehicles and the rough number of surrounding pedestrians in the intelligent driving rough space model are obtained.

[0048] Establishing a three-dimensional coordinate system with the target vehicle as the origin;

[0049] Acquiring coordinate data of the intelligent driving space model and the intelligent driving rough space model based on the three-dimensional coordinate system to obtain model three-dimensional data and rough model three-dimensional data;

[0050] Determining a model volume and a model space range of the intelligent driving coarse spatial model according to the three-dimensional model data;

[0051] Determining a coarse model volume and a coarse model space range of the intelligent driving coarse spatial model according to the coarse model three-dimensional data;

[0052] calculating a non-overlapping volume between the intelligent driving space model and the intelligent driving coarse space model according to the model volume, the model space range, the coarse model volume, and the coarse model space range to obtain a non-overlapping volume;

[0053] A model error is calculated based on the model volume, the rough model volume, the non-overlapping volume, the rough number of surrounding vehicles, and the rough number of surrounding pedestrians.

[0054] Optionally, calculating the model error according to the model volume, the rough model volume, the non-overlapping volume, the rough number of surrounding vehicles, and the rough number of surrounding pedestrians includes:

[0055] The model error is calculated using the following formula: in, is the model error, is the model volume, is the rough model volume, is the non-coincident volume, is the base of natural logarithms, is the number of pedestrians around, is the rough number of pedestrians around, is the number of surrounding vehicles, is the approximate number of surrounding vehicles.

[0056] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0057] at least one processor;

[0058] and, a memory communicatively coupled to the at least one processor;

[0059] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned method for rapid image processing during intelligent driving.

[0060] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one computer program. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned method for rapid image processing during intelligent driving.

[0061] The embodiment of the present invention collects the front view image data, surround view image data and side view image data of the target vehicle in real time during its driving process, performs targeted analysis of road markings based on the front view image data to obtain road marking data, performs targeted analysis of the positions of surrounding vehicles based on the surround view image data to obtain surrounding vehicle data, and performs targeted recognition of pedestrians on the road based on the side view image data to obtain pedestrian data, thereby improving the efficiency of image data analysis, converting the road marking data, the surrounding vehicle data and the pedestrian data into coordinate data to obtain comprehensive coordinate data, and performing rapid three-dimensional space modeling based on the comprehensive coordinate data to obtain intelligent driving space. A spatial model is generated, driving suggestions are generated based on the intelligent driving spatial model, wide-angle image data of the target vehicle is collected in real time, three-dimensional coarse modeling is performed based on the wide-angle image data set to obtain an intelligent driving coarse spatial model, the intelligent driving spatial model is compared with the intelligent driving coarse spatial model in real time to obtain a model error, and it is determined whether the model error is greater than or equal to a preset error threshold. If the model error is less than the error threshold, the driving suggestions are broadcast to the user. If the model error is greater than or equal to the error threshold, the driving suggestions are adjusted in real time according to the model error, and the process returns to the step of broadcasting the driving suggestions to the user. This can improve the safety of intelligent driving. Therefore, the method, device, and medium for rapid image processing during intelligent driving proposed in the present invention can solve the problem of low response efficiency of existing image processing methods in intelligent driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic flow chart of a method for rapid image processing during intelligent driving provided by one embodiment of the present invention;

[0063] Figure 2 A schematic diagram of a process for analyzing road markings provided by one embodiment of the present invention;

[0064] Figure 3 A schematic diagram of a coordinate data conversion process provided by one embodiment of the present invention;

[0065] Figure 4 A schematic diagram of the structure of an electronic device for implementing the method for rapid image processing during intelligent driving provided by one embodiment of the present invention.

[0066] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0068] The embodiment of the present application provides a method for rapid image processing during intelligent driving. The execution subject of the method for rapid image processing during intelligent driving includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for rapid image processing during intelligent driving can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0069] Reference Figure 1 FIG. 1 is a flow chart of a method for rapidly processing images during intelligent driving according to an embodiment of the present invention. In this embodiment, the method for rapidly processing images during intelligent driving includes:

[0070] S1. Real-time collection of front view image data, surround view image data, and side view image data during the target vehicle's driving process.

[0071] In an embodiment of the present invention, the forward-looking image data is collected by a forward-looking camera preset in the target vehicle.

[0072] In an embodiment of the present invention, the surround view image data is collected by a surround view camera preset in the target vehicle.

[0073] In an embodiment of the present invention, the side view image data is collected by a side view camera preset in the target vehicle.

[0074] In detail, the forward-looking camera is usually installed behind the front windshield of the vehicle, the number of the forward-looking cameras may be two, and the forward-looking image data may be two sets of image data.

[0075] In detail, the surround view cameras are usually installed under the front bumper and the rear bumper of the vehicle. There may be two surround view cameras installed under the front bumper and the rear bumper of the vehicle, and the surround view image data includes multiple sets of images.

[0076] In detail, the side view cameras are usually mounted on the sides of the vehicle. Two side view cameras may be mounted on each of the left and right sides of the vehicle. The side view image data includes multiple sets of images.

[0077] In an embodiment of the present invention, the front view image data, the surround view image data, and the side view image data are all collected according to a preset collection frequency, and the front view image data, the surround view image data, and the side view image data all contain continuous images.

[0078] In detail, the preset collection frequency may be once per second.

[0079] S2. Analyze the road markings based on the forward-view image data to obtain road marking data.

[0080] In the embodiment of the present invention, referring to Figure 2 FIG. 1 is a flow chart of a road marking analysis process according to an embodiment of the present invention. The road marking analysis is performed based on the front view image data to obtain the road marking data, including:

[0081] S21. Segmenting the front view image using a pre-trained lightweight road segmentation model to obtain a front view road image;

[0082] S22. Performing color enhancement on the forward road image using an HSV color space to obtain an enhanced road image;

[0083] S23, performing edge contour detection on the enhanced road image to obtain line marking contour data;

[0084] S24, detecting straight lines and curves in the enhanced road image based on Hough transform to obtain shape data;

[0085] S25 . Locate road markings in the forward-view road image based on the road marking contour data and the shape data to obtain road marking data.

[0086] In detail, the HSV color space consists of three dimensions: hue, saturation, and value.

[0087] Specifically, the use of the HSV color space to perform color enhancement on the forward road image to obtain an enhanced road image refers to improving the purity of the colors in the forward road image by enhancing saturation, and then enhancing the colors corresponding to the road markings in the forward road image (usually white and yellow) by adjusting a specific hue range.

[0088] In detail, the edge contour detection of the enhanced road image is performed to obtain the marking contour data, which is detected based on a color threshold. Since the color in the enhanced road image has been enhanced, the contour can be confirmed by identifying areas with large color intensity differences in the enhanced road image.

[0089] The Hough transform is a widely used technique in computer vision and image processing, primarily for detecting geometric shapes in images, such as lines, circles, and other parametric curves. The Hough transform converts shapes (such as lines) in an image from the spatial domain to the parametric domain. In the parametric domain, an accumulator array records the "votes" or intersections of each possible shape parameter, thereby detecting the shape. For line detection, the accumulator array records the slope and intercept of the line.

[0090] In an embodiment of the present invention, locating the road markings in the forward-view road image based on the marking contour data and the shape data to obtain the road marking data refers to identifying areas in the forward-view road image that are white or yellow in color and have straight or curved shapes to obtain the road marking data.

[0091] In detail, the lightweight road segmentation model is a neural network model that can maintain high segmentation accuracy while optimizing the network structure and computational efficiency, so that it can run in real time on resource-constrained devices (such as embedded systems or mobile devices).

[0092] In an embodiment of the present invention, the step of segmenting the front view image using a pre-trained lightweight road segmentation model to obtain a front view road image includes:

[0093] performing normalization processing on the front view image to obtain a normalized front view image;

[0094] Performing dimensional conversion processing on the normalized front view image to obtain a converted image;

[0095] Inferring the transformed image by calling the forward propagation function of the lightweight road segmentation model to obtain a probability distribution tensor;

[0096] The probability distribution tensor is converted into a two-dimensional array based on a preset probability threshold, and the front view image is segmented using the two-dimensional array to obtain a front view road image.

[0097] In detail, the normalization process is to map the pixel value range of the image to [0, 1] or [-1, 1] to improve the processing efficiency of the model.

[0098] In detail, the dimension conversion processing performed on the normalized front view image is to convert the dimension of the image from (height, width, channel) to (channel, height, width).

[0099] In detail, the probability distribution tensor refers to the distribution of the probability of whether any pixel in the image is a road in the road segmentation task.

[0100] In detail, converting the probability distribution tensor into a two-dimensional array based on a preset probability threshold means converting the elements in the probability distribution tensor that are greater than the probability threshold into 1, and converting the elements that are less than the probability threshold into 0.

[0101] In detail, segmenting the front-view image by using the two-dimensional array refers to segmenting the front-view image according to the portion with a value of 1 in the two-dimensional array.

[0102] In the embodiment of the present invention, by performing targeted analysis of road markings based on the forward-view image data to obtain road marking data, the efficiency of identifying road markings can be improved.

[0103] S3. Analyze the positions of surrounding vehicles based on the surround view image data to obtain surrounding vehicle data.

[0104] In an embodiment of the present invention, performing targeted analysis of surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data refers to identifying and locating surrounding vehicles based on the surround view image data, and analyzing and predicting the vehicle's trajectory by analyzing the position changes of the vehicles in continuous images.

[0105] In an embodiment of the present invention, performing targeted analysis of surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data includes:

[0106] Performing vehicle target recognition on the surround view image data to obtain target recognition data;

[0107] Tracking the positions of vehicles in the surround view image data based on the target recognition data using a multi-target tracking algorithm to obtain vehicle trajectories;

[0108] Performing short-term trajectory prediction on the vehicle in the surround view image data based on the vehicle trajectory to obtain a predicted trajectory;

[0109] The target recognition data, the vehicle trajectory, and the predicted trajectory are aggregated to obtain the surrounding vehicle data.

[0110] Specifically, vehicle target recognition in the surround view image data can be achieved using a pre-trained lightweight target detection model, such as YOLOv8n. YOLOv8n is a lightweight version of the YOLOv8 family, designed for scenarios requiring efficient computing resources. It inherits the core advantages of YOLOv8, such as high accuracy and fast inference speed, and further reduces the number of parameters through an optimized network structure. YOLOv8n supports a variety of tasks, including target detection, image segmentation, and pose estimation, making it suitable for embedded devices and real-time applications.

[0111] In detail, the multi-target tracking algorithm is used to track the position of vehicles in the surround view image data based on the target recognition data. This is done by extracting a bounding box of the vehicle in each image based on the target recognition data, tracking the position of the same vehicle in consecutive surround view images, and extracting key features such as trajectory length, driving speed, and acceleration based on the vehicle coordinates and timestamp information obtained by tracking.

[0112] In an embodiment of the present invention, the short-term trajectory prediction of the vehicle in the surround view image data based on the vehicle trajectory is to establish a vehicle motion model based on a Kalman filter algorithm, and predict the running trajectory based on the driving speed and acceleration of the vehicle at each timestamp.

[0113] Specifically, the Kalman filter is an efficient recursive estimation algorithm for state estimation of dynamic systems. It combines a mathematical model of the system with observed data to provide an optimal estimate (minimum mean square error) of the system state in the presence of noise.

[0114] In the embodiment of the present invention, by performing targeted analysis on the positions of surrounding vehicles based on the surround view image data to obtain surrounding vehicle data, the efficiency of identifying surrounding vehicles can be improved.

[0115] S4. Targeted identification of pedestrians on the road is performed based on the side view image data to obtain pedestrian data on the road.

[0116] In an embodiment of the present invention, the targeted identification of road pedestrians based on the side view image data to obtain road pedestrian data is performed based on a neural network model to perform target recognition on the side view image data to obtain position information of road pedestrians, and then pedestrian trajectory analysis and short-term trajectory prediction are performed based on the position of road pedestrians in continuous images.

[0117] S5. Convert the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data, and perform three-dimensional space rapid modeling based on the comprehensive coordinate data to obtain an intelligent driving space model.

[0118] In an embodiment of the present invention, the road marking data, the surrounding vehicle data, and the road pedestrian data are converted into coordinate data to obtain comprehensive coordinate data, and three-dimensional space rapid modeling is performed based on the comprehensive coordinate data to obtain an intelligent driving space model.

[0119] In the embodiment of the present invention, since the road marking data, the surrounding vehicle data, and the road pedestrian data are captured and identified by multiple cameras, binocular positioning can be performed on the same target using two images at the same timestamp.

[0120] In the embodiment of the present invention, the road marking data includes distance data identified from different images of the same road marking.

[0121] In the embodiment of the present invention, referring to Figure 3 FIG. 1 is a flow chart of coordinate data conversion according to an embodiment of the present invention, wherein the road marking data, the surrounding vehicle data, and the road pedestrian data are converted into coordinate data to obtain comprehensive coordinate data, including:

[0122] S31, obtaining a first marking distance and a second marking distance included in the road marking data, and calculating a marking position according to the first marking distance, the second marking distance, and a preset front view camera distance;

[0123] S32, obtaining a first vehicle distance and a second vehicle distance included in the surrounding vehicle data, and calculating a vehicle position based on the first vehicle distance, the second vehicle distance, and a preset surround view camera distance;

[0124] S33, obtaining a first distance and a second distance of the pedestrian included in the road pedestrian data, and calculating the position of the pedestrian according to the first distance, the second distance, and a preset side view camera distance;

[0125] S34, establishing a two-dimensional rectangular coordinate system parallel to the horizontal plane with the target vehicle as the origin;

[0126] S35. Convert the line position, the vehicle position, and the pedestrian position into coordinate data based on the two-dimensional rectangular coordinate system to obtain comprehensive coordinate data.

[0127] In an embodiment of the present invention, the three-dimensional space rapid modeling based on the comprehensive coordinate data to obtain the intelligent driving space model is to select a vehicle model, a pedestrian model and a road marking model from a pre-established model library, and then place the vehicle model, the pedestrian model and the road marking model into the pre-established three-dimensional model space based on the coordinate data, and perform short-term predicted trajectory simulation of vehicles and pedestrians based on the surrounding vehicle data and the vehicle predicted trajectory data and the pedestrian predicted trajectory data contained in the road pedestrian data.

[0128] In an embodiment of the present invention, by converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data, the efficiency of model establishment can be improved. By performing rapid three-dimensional space modeling based on the comprehensive coordinate data to obtain an intelligent driving space model, the accuracy of generating driving suggestions can be improved.

[0129] S6. Generate driving suggestions based on the intelligent driving space model.

[0130] In this embodiment of the present invention, since the surrounding vehicle data contains speed data and predicted trajectory data for surrounding vehicles, and the pedestrian data contains predicted pedestrian trajectory data, the intelligent driving space model can simulate the predicted trajectories of surrounding vehicles and pedestrians, and, combined with the speed of the target vehicle, can generate more accurate and reasonable driving recommendations.

[0131] In an embodiment of the present invention, generating driving advice based on the intelligent driving space model refers to identifying information about vehicles surrounding the target vehicle (including position, speed, and future short-term trajectory) and information about pedestrians (including position and motion trajectory) based on the intelligent driving space model, and generating driving advice based on this position information. For example, when a pedestrian appears in front of the target vehicle, the driving advice generated is "Slow down and avoid the pedestrian ahead." If the vehicle ahead is driving too slowly and there is no vehicle to the left or the vehicle behind in the left lane has not accelerated, the driving advice generated is to accelerate and overtake the vehicle ahead in the left lane. If the speed of the vehicle ahead suddenly drops, the driving advice generated is "Detect vehicle ahead, avoid it."

[0132] In the embodiment of the present invention, driving safety can be improved by generating driving suggestions based on the intelligent driving space model.

[0133] S7. Collect wide-angle image data of the target vehicle in real time, perform three-dimensional coarse modeling based on the wide-angle image data set, and obtain a rough intelligent driving space model.

[0134] In an embodiment of the present invention, the real-time acquisition of wide-angle image data of the target vehicle and the performance of three-dimensional coarse modeling based on the wide-angle image data set to obtain a rough intelligent driving spatial model include:

[0135] performing wide-angle distortion correction processing on the wide-angle image data to obtain a corrected image;

[0136] Performing target object feature extraction on the corrected image to obtain target object features;

[0137] The target object features are used to perform fast approximate feature matching to obtain target recognition data.

[0138] Acquire the target object in the corrected image according to the target recognition data using a three-dimensional reconstruction algorithm to obtain the three-dimensional coordinates of the target object;

[0139] Three-dimensional modeling is performed based on the three-dimensional coordinates of the target object to obtain a rough spatial model of intelligent driving.

[0140] In detail, the use of the target object features to perform rapid approximate feature matching to obtain target recognition data refers to matching the target object features with the features in a preset feature database in sequence, and determining that the match is successful when the matching degree reaches a preset matching degree threshold.

[0141] In detail, the feature database contains a large number of road marking features, vehicle contour features and pedestrian contour features.

[0142] In detail, the size of the preset matching degree threshold can be set according to task requirements. The larger the matching degree threshold, the higher the matching efficiency, but the lower the matching accuracy.

[0143] S8. Compare the intelligent driving space model with the intelligent driving rough space model in real time to obtain a model error.

[0144] In an embodiment of the present invention, the comparing the intelligent driving space model with the intelligent driving rough space model in real time to obtain a model error includes:

[0145] The number of surrounding vehicles and the number of surrounding pedestrians in the intelligent driving space model are obtained, and the rough number of surrounding vehicles and the rough number of surrounding pedestrians in the intelligent driving rough space model are obtained.

[0146] Establishing a three-dimensional coordinate system with the target vehicle as the origin;

[0147] Acquiring coordinate data of the intelligent driving space model and the intelligent driving rough space model based on the three-dimensional coordinate system to obtain model three-dimensional data and rough model three-dimensional data;

[0148] Determining a model volume and a model space range of the intelligent driving coarse spatial model according to the three-dimensional model data;

[0149] Determining a coarse model volume and a coarse model space range of the intelligent driving coarse spatial model according to the coarse model three-dimensional data;

[0150] calculating a non-overlapping volume between the intelligent driving space model and the intelligent driving coarse space model according to the model volume, the model space range, the coarse model volume, and the coarse model space range to obtain a non-overlapping volume;

[0151] A model error is calculated based on the model volume, the rough model volume, the non-overlapping volume, the rough number of surrounding vehicles, and the rough number of surrounding pedestrians.

[0152] In detail, the calculating of the model error according to the model volume, the rough model volume, the non-overlapping volume, the rough number of surrounding vehicles, and the rough number of surrounding pedestrians includes:

[0153] The model error is calculated using the following formula: in, is the model error, is the model volume, is the rough model volume, is the non-coincident volume, is the base of natural logarithms, is the number of pedestrians around, is the rough number of pedestrians around, is the number of surrounding vehicles, is the approximate number of surrounding vehicles.

[0154] S9. Determine whether the model error is greater than or equal to a preset error threshold.

[0155] If the model error is less than the error threshold, S10 is executed to announce the driving advice to the user.

[0156] In the embodiment of the present invention, when the model error is less than the error threshold, it indicates that the intelligent driving space model has high accuracy and the driving advice generated based on the intelligent driving space model has high reliability.

[0157] In the embodiment of the present invention, the broadcasting of the driving advice to the user may be performed by converting the driving advice into voice data and then broadcasting the data using a speaker provided in the target vehicle.

[0158] If the model error is greater than or equal to the error threshold, step S11 is executed to adjust the driving suggestion in real time according to the model error.

[0159] In an embodiment of the present invention, when the model error is greater than or equal to the error threshold, it indicates that the target vehicle is in a complex road condition, the intelligent driving space model may have errors, and the driving advice generated based on the intelligent driving space model is less reliable, so the driving advice needs to be adjusted.

[0160] In this embodiment of the present invention, adjusting the driving advice in real time based on the model error refers to adjusting the driving advice to a safer and more conservative one. For example, when the preceding vehicle is traveling at a low speed, a suggestion to overtake in the left lane will only be made if there are no vehicles in the left lane. If there are vehicles behind or in front of the left lane, a suggestion to overtake in the left lane will not be generated. For example, when there are vehicles ahead or behind, lowering the threshold for triggering the reminder to maintain a safe distance will make it easier to generate the driving advice to "maintain safe distance."

[0161] In the embodiment of the present invention, after the driving advice is adjusted in real time according to the model error, the process returns to S10 and the driving advice is broadcast to the user.

[0162] In an embodiment of the present invention, when the model error is greater than or equal to the error threshold, the step of broadcasting the driving advice to the user is re-executed after adjusting the driving advice in real time according to the model error, thereby greatly improving the safety of intelligent driving under complex road conditions.

[0163] like Figure 4 , which is a schematic diagram of the structure of an electronic device for a method for rapid image processing during intelligent driving provided by an embodiment of the present invention.

[0164] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a fast image processing program for intelligent driving.

[0165] In some embodiments, the processor 10 may be comprised of an integrated circuit, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the electronic device using various interfaces and circuits. It executes programs or modules stored in the memory 11 (such as a fast image processing program for intelligent driving) and accesses data stored in the memory 11 to perform various functions and process data.

[0166] The memory 11 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory 11 may include both an internal storage unit of the electronic device and an external storage device. The memory 11 can be used not only to store application software installed in the electronic device and various types of data, such as the code of the image rapid processing program for intelligent driving, but also to temporarily store data that has been output or is about to be output.

[0167] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0168] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in this embodiment, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0169] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0170] For example, although not shown, the electronic device may further include a power source (e.g., a battery) to power various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby enabling functions such as charge management, discharge management, and power consumption management via the power management device. The power source may further include any of one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not further detailed here.

[0171] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0172] The intelligent driving image fast processing program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0173] Real-time collection of front view image data, surround view image data and side view image data of the target vehicle during its driving process;

[0174] Performing targeted analysis of road markings based on the forward-view image data to obtain road marking data;

[0175] Performing targeted analysis of surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data;

[0176] Performing targeted recognition of pedestrians on the road based on the side view image data to obtain pedestrian data on the road;

[0177] Converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data, and performing three-dimensional space rapid modeling based on the comprehensive coordinate data to obtain an intelligent driving space model;

[0178] generating driving recommendations based on the intelligent driving space model;

[0179] collecting wide-angle image data of the target vehicle in real time, performing three-dimensional rough modeling based on the wide-angle image data set, and obtaining a rough intelligent driving spatial model;

[0180] Comparing the intelligent driving space model with the intelligent driving rough space model in real time to obtain a model error;

[0181] Determining whether the model error is greater than or equal to a preset error threshold;

[0182] If the model error is less than the error threshold, announcing the driving advice to the user;

[0183] If the model error is greater than or equal to the error threshold, the driving advice is adjusted in real time according to the model error, and then the process returns to the step of broadcasting the driving advice to the user.

[0184] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0185] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. The computer-readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0186] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0187] Real-time collection of front view image data, surround view image data and side view image data of the target vehicle during its driving process;

[0188] Performing targeted analysis of road markings based on the forward-view image data to obtain road marking data;

[0189] Performing targeted analysis of surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data;

[0190] Performing targeted recognition of pedestrians on the road based on the side view image data to obtain pedestrian data on the road;

[0191] Converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data, and performing three-dimensional space rapid modeling based on the comprehensive coordinate data to obtain an intelligent driving space model;

[0192] generating driving recommendations based on the intelligent driving space model;

[0193] collecting wide-angle image data of the target vehicle in real time, performing three-dimensional rough modeling based on the wide-angle image data set, and obtaining a rough intelligent driving spatial model;

[0194] Comparing the intelligent driving space model with the intelligent driving rough space model in real time to obtain a model error;

[0195] Determining whether the model error is greater than or equal to a preset error threshold;

[0196] If the model error is less than the error threshold, announcing the driving advice to the user;

[0197] If the model error is greater than or equal to the error threshold, the driving advice is adjusted in real time according to the model error, and then the process returns to the step of broadcasting the driving advice to the user.

[0198] In the embodiments provided herein, it should be understood that the disclosed devices, apparatuses, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0199] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0200] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0201] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0202] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0203] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0204] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for rapid image processing during intelligent driving, characterized in that: The method comprises: Real-time collection of front view image data, surround view image data and side view image data of the target vehicle during its driving process; Performing targeted analysis of road markings based on the forward-view image data to obtain road marking data; Performing targeted analysis of surrounding vehicle positions based on the surround view image data to obtain surrounding vehicle data; Performing targeted recognition of pedestrians on the road based on the side view image data to obtain pedestrian data on the road; Converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data, and performing rapid three-dimensional space modeling based on the comprehensive coordinate data to obtain an intelligent driving space model; generating driving recommendations based on the intelligent driving space model; collecting wide-angle image data of the target vehicle in real time, performing three-dimensional rough modeling based on the wide-angle image data set, and obtaining a rough intelligent driving spatial model; Obtaining the number of surrounding vehicles and the number of surrounding pedestrians in the intelligent driving space model, obtaining the rough number of surrounding vehicles and the rough number of surrounding pedestrians in the intelligent driving coarse space model, establishing a three-dimensional coordinate system with the target vehicle as the origin, obtaining coordinate data of the intelligent driving space model and the intelligent driving coarse space model based on the three-dimensional coordinate system, obtaining model three-dimensional data and coarse model three-dimensional data, determining a model volume and a model space range of the intelligent driving coarse space model according to the model three-dimensional data, determining a coarse model volume and a coarse model space range of the intelligent driving coarse space model according to the coarse model three-dimensional data, calculating a non-overlapping volume between the intelligent driving space model and the intelligent driving coarse space model according to the model volume, the model space range, the coarse model volume, and the coarse model space range to obtain a non-overlapping volume, and calculating a model error according to the model volume, the coarse model volume, the non-overlapping volume, the coarse number of surrounding vehicles, and the coarse number of surrounding pedestrians; Determining whether the model error is greater than or equal to a preset error threshold; If the model error is less than the error threshold, announcing the driving advice to the user; If the model error is greater than or equal to the error threshold, the driving advice is adjusted in real time according to the model error, and then the process returns to the step of broadcasting the driving advice to the user.

2. The method for rapid image processing during intelligent driving according to claim 1, wherein: The performing targeted analysis of road markings based on the forward-view image data to obtain road marking data includes: Segmenting the front view image using a pre-trained lightweight road segmentation model to obtain a front view road image; Performing color enhancement on the forward road image using the HSV color space to obtain an enhanced road image; Performing edge contour detection on the enhanced road image to obtain line marking contour data; Detecting straight lines and curves in the enhanced road image based on Hough transform to obtain shape data; The road markings in the forward-view road image are located based on the road marking contour data and the shape data to obtain road marking data.

3. The method for rapid image processing during intelligent driving according to claim 2, wherein: The step of segmenting the front view image using a pre-trained lightweight road segmentation model to obtain a front view road image includes: performing normalization processing on the front view image to obtain a normalized front view image; Performing dimensional conversion processing on the normalized front view image to obtain a converted image; Inferring the transformed image by calling the forward propagation function of the lightweight road segmentation model to obtain a probability distribution tensor; The probability distribution tensor is converted into a two-dimensional array based on a preset probability threshold, and the front view image is segmented using the two-dimensional array to obtain a front view road image.

4. The method for rapid image processing during intelligent driving according to claim 1, wherein: The method of analyzing the positions of surrounding vehicles based on the surround view image data to obtain surrounding vehicle data includes: Performing vehicle target recognition on the surround view image data to obtain target recognition data; Tracking the positions of vehicles in the surround view image data based on the target recognition data using a multi-target tracking algorithm to obtain vehicle trajectories; Performing short-term trajectory prediction on the vehicle in the surround view image data based on the vehicle trajectory to obtain a predicted trajectory; The target recognition data, the vehicle trajectory, and the predicted trajectory are aggregated to obtain the surrounding vehicle data.

5. The method for rapid image processing during intelligent driving according to claim 1, wherein: The converting the road marking data, the surrounding vehicle data, and the road pedestrian data into coordinate data to obtain comprehensive coordinate data includes: Obtaining a first marking distance and a second marking distance included in the road marking data, and calculating a marking position according to the first marking distance, the second marking distance, and a preset front view camera distance; Obtaining a first vehicle distance and a second vehicle distance included in the surrounding vehicle data, and calculating a vehicle position based on the first vehicle distance, the second vehicle distance, and a preset surround view camera distance; Obtaining a first distance and a second distance of a pedestrian included in the road pedestrian data, and calculating a position of the pedestrian based on the first distance, the second distance, and a preset side view camera distance; Establishing a two-dimensional rectangular coordinate system parallel to the horizontal plane with the target vehicle as the origin; The marking line position, the vehicle position, and the pedestrian position are converted into coordinate data based on the two-dimensional rectangular coordinate system to obtain comprehensive coordinate data.

6. The method for rapid image processing during intelligent driving according to claim 1, wherein: The real-time acquisition of wide-angle image data of the target vehicle and the performing of three-dimensional coarse modeling based on the wide-angle image data set to obtain a rough intelligent driving spatial model include: performing wide-angle distortion correction processing on the wide-angle image data to obtain a corrected image; Performing target object feature extraction on the corrected image to obtain target object features; Using the target object features to perform rapid approximate feature matching to obtain target recognition data; Acquire the target object in the corrected image according to the target recognition data using a three-dimensional reconstruction algorithm to obtain the three-dimensional coordinates of the target object; Three-dimensional modeling is performed based on the three-dimensional coordinates of the target object to obtain a rough spatial model of intelligent driving.

7. The method for rapid image processing during intelligent driving according to claim 1, wherein: The calculating of the model error according to the model volume, the rough model volume, the non-overlapping volume, the rough number of surrounding vehicles, and the rough number of surrounding pedestrians includes: The model error is calculated using the following formula: in, is the model error, is the model volume, is the rough model volume, is the non-coincident volume, is the base of natural logarithms, is the number of pedestrians around, is the rough number of pedestrians around, is the number of surrounding vehicles, is the approximate number of surrounding vehicles.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for rapid image processing during intelligent driving as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for rapid image processing during intelligent driving as claimed in any one of claims 1 to 7 is implemented.

Images