Image processing method for automatic driving of heavy truck

Abstract

The invention discloses an image processing method for automatic driving of a heavy truck, relates to the technical field of image processing, and solves the problems that the prior art has a certain pre-judgment function and can better serve transportation of the heavy truck; when the heavy truck steers or turns, a view blind area exists, and traffic accidents are caused. A plurality of real-time images in the driving range of the heavy truck are acquired through image acquisition equipment; preprocessing the plurality of real-time images to obtain a driving image; extracting traffic features from the driving image; the adjustment measures of the heavy truck are analyzed based on the traffic characteristics; screening out an area image of a view blind area of the heavy truck from the driving image; key features of the regional image are extracted, and the adjustment time of the heavy truck is determined according to the key features and the adjustment measures; adjusting and controlling the running of the heavy truck according to the adjusting measures and the adjusting time; and the driving safety of the heavy truck can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and relates to image processing technology for autonomous driving, and specifically to an image processing method for autonomous driving of a heavy truck. Background Art

[0002] Autonomous driving technology mainly relies on the synergy of multiple sensors, artificial intelligence, visual computing, radar, monitoring devices, and global positioning systems. Unmanned intelligent car systems are realized through on-board computer systems. Heavy trucks are important tools for road transportation. Self-driving heavy trucks can avoid traffic accidents caused by drivers' fatigue, distraction, drunkenness, or operational errors. Through sensors and algorithms, autonomous driving systems can detect obstacles within milliseconds and respond quickly to avoid collisions and traffic jams. Image processing is crucial in autonomous driving technology, capturing real-time images and extracting key features and target objects, such as road lines, traffic signs, pedestrians, other vehicles, etc., to help autonomous driving systems understand the surrounding environment more accurately.

[0003] The prior art (invention patent application with publication number CN115320632A) discloses an automatic driving heavy-duty truck based on AI algorithm. During the driving process of the heavy-duty truck, the surrounding environment is photographed and data is collected. The collected data is analyzed by AI algorithm and suspected data information is judged; the suspected data information is secondary analyzed, and if it is judged to be defective data information, it is marked and uploaded to the cloud server. The cloud server immediately stops the automatic driving of the heavy-duty truck remotely to ensure the safety of the vehicle and prompts the workers to perform on-site maintenance; the prior art has a certain predictive function and can better serve the transportation of heavy-duty trucks; however, when the heavy-duty truck is turning or turning, there is a blind spot in the field of vision, which leads to traffic accidents.

[0004] The present invention provides an image processing method for automatic driving of a heavy truck to solve the above technical problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an image processing method for automatic driving of a heavy-duty truck, which is used to solve the technical problem that the prior art has a certain prediction function and can better serve the transportation of heavy-duty trucks; when a heavy-duty truck is turning or making a turn, there is a blind spot in the field of vision, which leads to traffic accidents.

[0006] To achieve the above object, a first aspect of the present invention provides an image processing method for automatic driving of a heavy truck, comprising:

[0007] Step S1: using an image acquisition device to acquire a number of real-time images within the driving range of a heavy truck; pre-processing the real-time images to obtain driving images; wherein the image acquisition device is set according to the heavy truck;

[0008] Step S2: extracting traffic features from the driving image; analyzing adjustment measures for heavy trucks based on the traffic features;

[0009] Step S3: Filtering out the regional image of the blind spot of the heavy truck from the driving image; extracting the key features of the regional image, and determining the adjustment time of the heavy truck according to the key features and the adjustment measures;

[0010] Step S4: adjusting and controlling the driving of the heavy truck according to the adjustment measures and adjustment time.

[0011] Preferably, the preprocessing of a plurality of real-time images to obtain a driving image includes:

[0012] Retrieving a number of real-time images, and determining whether the clarity of the real-time images is greater than a clarity threshold; if so, retaining the corresponding real-time images; if not, discarding the corresponding real-time images, and re-collecting the real-time images of the corresponding areas;

[0013] The retained real-time image and the re-collected real-time image are grayed, denoised, and filtered, and the processed real-time image is marked as a driving image.

[0014] It should be noted that when the real-time image is recaptured, the clarity of the real-time image needs to be re-judged; the clarity threshold is set based on experience.

[0015] The present invention detects the clarity of the collected real-time image, and when the clarity does not meet the requirement, the real-time image is collected again, so that the quality of the collected real-time image can be guaranteed; it is beneficial to improve the accuracy of subsequent analysis and reduce the occurrence of traffic accidents.

[0016] Preferably, the image acquisition device is configured according to a heavy truck, including:

[0017] Obtaining the acquisition range of the image acquisition device; obtaining the field of view of the heavy truck; dividing the field of view into several areas according to the acquisition range; and setting an image acquisition device in each area;

[0018] Obtain the blind spots of heavy trucks; set up several image acquisition devices in the blind spots according to the acquisition range.

[0019] It should be noted that the image acquisition device is set at the corresponding position of the heavy truck and has been set before the heavy truck leaves the factory.

[0020] The present invention arranges the image acquisition device at a corresponding position of the heavy truck according to the field of view of the heavy truck and the acquisition range of the image acquisition device, and also arranges the image acquisition device in the blind spot of the field of view of the heavy truck, which can ensure the integrity of the real-time image acquired and is helpful to avoid the blind spot of the field of view of the heavy truck, thereby avoiding traffic accidents caused by the heavy truck.

[0021] Preferably, the extracting of traffic features from the driving image includes:

[0022] Retrieving a driving image, dividing the driving image into a number of connected areas, collecting directional histograms of the gradients of each pixel point in the number of connected areas; integrating the number of directional histograms into a feature input sequence;

[0023] The feature extraction model is called, and the feature input sequence is input into the feature extraction model to obtain the corresponding traffic characteristics; wherein the feature extraction model is constructed based on the artificial intelligence model.

[0024] The present invention divides a driving image into several connected areas and generates a directional histogram, and uses a feature extraction model to extract traffic features; it can ensure the accuracy and completeness of feature recognition in the driving image, and provide a basis for subsequent adjustment of heavy truck driving.

[0025] Preferably, the feature extraction model is constructed based on an artificial intelligence model, including:

[0026] Acquire a standard data set; wherein the standard data set includes standard input data consistent with the content attributes of the feature input sequence, and standard output data consistent with the content attributes of the traffic feature;

[0027] The standard data set is divided into a training set, a validation set, and a test set; the training set is used to train the artificial intelligence model, and the validation set is used to adjust the coefficients within the artificial intelligence model; the test set is used to analyze the performance parameters of the artificial intelligence model to determine whether the performance parameters of the artificial intelligence model are all greater than the parameter threshold; if so, the artificial intelligence model is marked as a feature extraction model; if not, the artificial intelligence model is retrained.

[0028] It should be noted that the performance parameters of the artificial intelligence model include accuracy, F1 score, recall rate and stability; the parameter thresholds are set by expert scores.

[0029] Preferably, the adjustment measures for heavy trucks based on traffic characteristics analysis include:

[0030] Retrieve traffic characteristics and obtain a traffic adjustment library; determine whether there are obstacles in the traffic characteristics; if yes, mark the corresponding driving image; if no, match the traffic characteristics with the traffic adjustment library to obtain adjustment measures;

[0031] Retrieve the marked driving image, retrieve the historical driving image of a period of time in the corresponding area, and judge whether the heavy truck has collided with the obstacle based on the historical driving image and the driving image; if yes, adjust the driving of the heavy truck; if not, keep the heavy truck moving.

[0032] It should be noted that traffic features include traffic lights, zebra crossings, lane lines, traffic signs, pedestrians and other vehicles; pedestrians and other vehicles are marked as obstacles; adjustment measures include deceleration, lane change, parking, starting, acceleration, turning and other operations.

[0033] The present invention classifies traffic features in driving images, and when there are no obstacles, adjusts the driving of heavy trucks according to a traffic adjustment library; when there are obstacles and a collision may occur, adjusts the driving of heavy trucks; and can adjust the driving of heavy trucks according to actual conditions, which is beneficial to ensuring the safe driving of heavy trucks.

[0034] Preferably, the step of determining the adjustment time of the heavy truck according to the key features and the adjustment measures includes:

[0035] Retrieve key features and determine whether there are obstacles in the blind spot of the heavy truck based on the key features; if yes, perform collision analysis based on adjustment measures; if no, record the adjustment time as 0;

[0036] Analyze the expected driving path of the heavy truck according to the adjustment measures, and judge whether the heavy truck will collide with the obstacle according to the expected driving path; if yes, determine the adjustment time according to the driving speed of the obstacle; if no, record the adjustment time as 0;

[0037] Obtain the speed of the obstacle and the distance from the front of the heavy truck; obtain the driving time by dividing the driving distance by the driving speed; and obtain the adjustment time by adding the time threshold to the driving time.

[0038] It should be noted that the method for extracting key features in the regional image is consistent with the method for extracting traffic features in the driving image; the time threshold is set manually based on experience.

[0039] The present invention analyzes key features in the regional image of the blind spot and determines the adjustment time; it can ensure the safety of the surrounding environment when the heavy truck is adjusted, avoid affecting the driving of the heavy truck due to the presence of pedestrians or other vehicles in the blind spot, and is conducive to ensuring the safety of the driving adjustment of the heavy truck.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The present invention collects a number of real-time images within the driving range of a heavy truck through an image acquisition device, detects the clarity of the collected real-time images, and re-collects the real-time images when the clarity does not meet the requirements, thereby ensuring the quality of the collected real-time images; being beneficial to improving the accuracy of subsequent analysis and reducing the occurrence of traffic accidents; setting the image acquisition device at a position corresponding to the heavy truck according to the field of view of the heavy truck and the acquisition range of the image acquisition device, and also setting the image acquisition device in the blind spot of the field of view of the heavy truck, thereby ensuring the integrity of the collected real-time images, and being beneficial to avoiding the blind spot of the field of view of the heavy truck, thereby causing traffic accidents; dividing the driving image into a number of connected areas and generating a directional histogram, and extracting traffic features using a feature extraction model; being able to ensure the accuracy and integrity of feature recognition in the driving image.

[0042] 2. The present invention classifies traffic features in driving images, and when there are no obstacles, adjusts the driving of heavy trucks according to the traffic adjustment library; when there are obstacles and a collision may occur, adjusts the driving of heavy trucks; can adjust the driving of heavy trucks according to actual conditions, which is beneficial to ensuring the safe driving of heavy trucks; analyzes key features in regional images of blind spots and determines the adjustment time; can ensure the safety of the surrounding environment of heavy trucks when making adjustments, avoid affecting the driving of heavy trucks due to the presence of pedestrians or other vehicles in the blind spots, and is beneficial to ensuring the safety of driving adjustments of heavy trucks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a schematic diagram of the overall steps of the present invention;

[0045] Figure 2 A schematic diagram of the image acquisition and feature analysis steps of the present invention;

[0046] Figure 3 It is a schematic diagram of the adjustment measures and the adjustment time determination steps of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] See also Figure 1 The first embodiment of the present invention provides an image processing method for automatic driving of a heavy truck, comprising:

[0049] Step S1: using an image acquisition device to acquire a number of real-time images within the driving range of a heavy truck; pre-processing the real-time images to obtain driving images; wherein the image acquisition device is set according to the heavy truck;

[0050] Step S2: extracting traffic features from the driving image; analyzing adjustment measures for heavy trucks based on the traffic features;

[0051] Step S3: Filtering out the regional image of the blind spot of the heavy truck from the driving image; extracting the key features of the regional image, and determining the adjustment time of the heavy truck according to the key features and the adjustment measures;

[0052] Step S4: adjusting and controlling the driving of the heavy truck according to the adjustment measures and adjustment time.

[0053] See also Figure 2 , obtain the acquisition range of the image acquisition device; obtain the field of view of the heavy truck; divide the field of view into several areas according to the acquisition range; set an image acquisition device in each area; obtain the blind area of ​​the heavy truck; set several image acquisition devices in the blind area according to the acquisition range.

[0054] It should be noted that the image acquisition device is set at the corresponding position of the heavy truck and has been set before the heavy truck leaves the factory.

[0055] For example: suppose the field of view of a heavy truck is divided into four areas. Two image acquisition devices are set in the second area according to the acquisition range of the image acquisition device, and one image acquisition device is set in the other areas; the blind area of ​​the field of view is divided into two areas, and one image acquisition device is set in each area.

[0056] A number of real-time images within the driving range of a heavy truck are collected by an image acquisition device; it is determined whether the clarity of the real-time images is greater than a clarity threshold; if yes, the corresponding real-time images are retained; if not, the corresponding real-time images are discarded, and real-time images of the corresponding area are re-collected; the retained real-time images and the re-collected real-time images are grayed, denoised, and filtered, and the processed real-time images are marked as driving images.

[0057] It should be noted that when the real-time image is recaptured, the clarity of the real-time image needs to be re-judged; the clarity threshold is set based on experience.

[0058] For example: assuming that 7 real-time images of a heavy truck are currently collected, the clarity of the 7 real-time images is analyzed, and the clarity of the third real-time image is lower than the clarity threshold, the real-time images of the corresponding area are re-collected; the retained real-time images and the re-collected real-time images are grayed, denoised, and filtered, and the processed 7 real-time images are marked as driving images.

[0059] Retrieving a driving image, dividing the driving image into a number of connected areas, collecting directional histograms of the gradients of each pixel point in the number of connected areas; integrating the number of directional histograms into a feature input sequence;

[0060] The feature extraction model is called, and the feature input sequence is input into the feature extraction model to obtain the corresponding traffic characteristics; wherein the feature extraction model is constructed based on the artificial intelligence model.

[0061] For example: suppose that the traffic features of the third and fourth driving images are extracted; the third and fourth driving images are divided into several connected areas respectively, and the directional histograms of the gradients of each pixel in the several connected areas are collected; the corresponding directional histograms are integrated into two groups of feature input sequences; the feature extraction model is called, and the two groups of feature input sequences are input into the feature extraction model to obtain the corresponding two groups of traffic features.

[0062] It is worth noting that the feature extraction model is built based on the artificial intelligence model, including:

[0063] Acquire a standard data set; wherein the standard data set includes standard input data consistent with the content attributes of the feature input sequence, and standard output data consistent with the content attributes of the traffic feature;

[0064] The standard data set is divided into a training set, a validation set, and a test set; the training set is used to train the artificial intelligence model, and the validation set is used to adjust the coefficients within the artificial intelligence model; the test set is used to analyze the performance parameters of the artificial intelligence model to determine whether the performance parameters of the artificial intelligence model are all greater than the parameter threshold; if so, the artificial intelligence model is marked as a feature extraction model; if not, the artificial intelligence model is retrained.

[0065] It should be noted that the performance parameters of the artificial intelligence model include accuracy, F1 score, recall rate and stability; the parameter thresholds are set by expert scores.

[0066] See also Figure 3 , retrieve traffic characteristics and obtain a traffic adjustment library; determine whether there are obstacles in the traffic characteristics; if yes, mark the corresponding driving image; if no, match the traffic characteristics with the traffic adjustment library to obtain adjustment measures; retrieve the marked driving image, retrieve the historical driving images of a period of time in the corresponding area, and determine whether the heavy truck collides with the obstacle based on the historical driving images and the driving images; if yes, adjust the driving of the heavy truck; if no, keep the heavy truck driving.

[0067] It should be noted that traffic features include traffic lights, zebra crossings, lane lines, traffic signs, pedestrians and other vehicles; pedestrians and other vehicles are marked as obstacles; adjustment measures include deceleration, lane change, parking, starting, acceleration, turning and other operations.

[0068] For example, assuming that there is a driving image at time T1; there are no obstacles in the 7 driving images, and the traffic light in the traffic feature is red, the heavy truck is stopped;

[0069] If an obstacle exists in the fourth driving image in the driving image at time T2, it is found after analysis that a collision may occur, and the heavy truck is controlled to change lanes.

[0070] Filter out regional images of the blind spot of the heavy truck from the driving image, and extract key features of the regional images; determine whether there is an obstacle in the blind spot of the heavy truck based on the key features; if yes, perform collision analysis based on adjustment measures; if no, record the adjustment time as 0; analyze the expected driving path of the heavy truck based on the adjustment measures, and determine whether the heavy truck will collide with the obstacle based on the expected driving path; if yes, determine the adjustment time based on the driving speed of the obstacle; if no, record the adjustment time as 0; obtain the driving speed of the obstacle and the driving distance from the front of the heavy truck; obtain the driving time by dividing the driving distance by the driving speed; obtain the adjustment time by adding the driving time to the time threshold; and adjust and control the driving of the heavy truck based on the adjustment measures and the adjustment time.

[0071] For example: suppose that a heavy truck needs to turn at an intersection. According to the regional image analysis of the blind spot in the driving image, there is an obstacle in the regional image and a collision will occur. After analysis, it is found that the driving time of the obstacle is 30S, and the time threshold is set to 10S. The adjustment time is 40S. The heavy truck is first controlled to stop, and then controlled to turn after 40S.

[0072] It should be noted that the method for extracting key features in the regional image is consistent with the method for extracting traffic features in the driving image; the time threshold is set manually based on experience.

[0073] The working principle of the present invention is as follows: the present invention collects a number of real-time images within the driving range of a heavy-duty truck through an image acquisition device; pre-processes the several real-time images to obtain driving images; extracts traffic features from the driving images; analyzes adjustment measures of the heavy-duty truck based on the traffic features; screens out regional images of the blind spots of the heavy-duty truck's field of vision from the driving images; extracts key features of the regional images, determines the adjustment time of the heavy-duty truck according to the key features and the adjustment measures; and adjusts and controls the driving of the heavy-duty truck according to the adjustment measures and the adjustment time.

[0074] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An image processing method for automatic driving of a heavy truck, characterized in that: include: Step S1: using an image acquisition device to acquire a number of real-time images within the driving range of a heavy truck; pre-processing the real-time images to obtain driving images; wherein the image acquisition device is set according to the heavy truck; Step S2: extracting traffic features from the driving image; analyzing adjustment measures for heavy trucks based on the traffic features; Step S3: Filtering out the regional image of the blind spot of the heavy truck from the driving image; extracting the key features of the regional image, and determining the adjustment time of the heavy truck according to the key features and the adjustment measures; Step S4: adjusting and controlling the driving of the heavy truck according to the adjustment measures and adjustment time.

2. The image processing method for automatic driving of a heavy truck according to claim 1, characterized in that: The preprocessing of a plurality of real-time images to obtain a driving image includes: Retrieving a number of real-time images, and determining whether the clarity of the real-time images is greater than a clarity threshold; if so, retaining the corresponding real-time images; if not, discarding the corresponding real-time images, and re-collecting the real-time images of the corresponding areas; The retained real-time image and the re-collected real-time image are grayed, denoised, and filtered, and the processed real-time image is marked as a driving image.

3. The image processing method for automatic driving of a heavy truck according to claim 1, characterized in that: The image acquisition device is configured according to a heavy truck, and includes: Obtaining the acquisition range of the image acquisition device; obtaining the field of view of the heavy truck; dividing the field of view into several areas according to the acquisition range; and setting an image acquisition device in each area; Obtain the blind spots of heavy trucks; set up several image acquisition devices in the blind spots according to the acquisition range.

4. The image processing method for automatic driving of a heavy truck according to claim 1, characterized in that: The extracting of traffic features from the driving image includes: Retrieving a driving image, dividing the driving image into a number of connected areas, collecting directional histograms of the gradients of each pixel point in the number of connected areas; integrating the number of directional histograms into a feature input sequence; The feature extraction model is called, and the feature input sequence is input into the feature extraction model to obtain the corresponding traffic characteristics; wherein the feature extraction model is constructed based on the artificial intelligence model.

5. The image processing method for automatic driving of a heavy truck according to claim 4, characterized in that: The feature extraction model is constructed based on an artificial intelligence model, including: Acquire a standard data set; wherein the standard data set includes standard input data consistent with the content attributes of the feature input sequence, and standard output data consistent with the content attributes of the traffic feature; The standard data set is divided into a training set, a validation set, and a test set; the training set is used to train the artificial intelligence model, and the validation set is used to adjust the coefficients within the artificial intelligence model; the test set is used to analyze the performance parameters of the artificial intelligence model to determine whether the performance parameters of the artificial intelligence model are all greater than the parameter threshold; if so, the artificial intelligence model is marked as a feature extraction model; if not, the artificial intelligence model is retrained.

6. The image processing method for automatic driving of a heavy truck according to claim 1, characterized in that: The adjustment measures for heavy trucks based on traffic characteristics analysis include: Retrieve traffic characteristics and obtain a traffic adjustment library; determine whether there are obstacles in the traffic characteristics; if yes, mark the corresponding driving image; if no, match the traffic characteristics with the traffic adjustment library to obtain adjustment measures; Retrieve the marked driving image, retrieve the historical driving image of a period of time in the corresponding area, and judge whether the heavy truck has collided with the obstacle based on the historical driving image and the driving image; if yes, adjust the driving of the heavy truck; if not, keep the heavy truck moving.

7. The image processing method for automatic driving of a heavy truck according to claim 1, characterized in that: The adjustment time of heavy trucks is determined based on key features and adjustment measures, including: Retrieve key features and determine whether there are obstacles in the blind spot of the heavy truck based on the key features; if yes, perform collision analysis based on adjustment measures; if no, record the adjustment time as 0; Analyze the expected driving path of the heavy truck according to the adjustment measures, and judge whether the heavy truck will collide with the obstacle according to the expected driving path; if yes, determine the adjustment time according to the driving speed of the obstacle; if no, record the adjustment time as 0; Obtain the speed of the obstacle and the distance from the front of the heavy truck; obtain the driving time by dividing the driving distance by the driving speed; and obtain the adjustment time by adding the time threshold to the driving time.

Images