Safe driving assistance method and system in severe weather

Abstract

The invention relates to the technical field of auxiliary driving, and discloses a safe driving auxiliary method and system in severe weather, and the method comprises the steps: carrying out the frame extraction of a driving video in severe weather, so as to obtain continuous frames of driving images in severe weather; performing image restoration processing on the driving image in the severe weather according to the driving image in the excellent weather to obtain a restored driving image; and generating a restored driving video according to the restored driving image, so as to assist the driver to realize safe driving based on the restored driving video. According to the invention, image restoration is carried out on the driving image in the severe weather, and the driving video is generated based on the restored driving image, so that the influence caused by the severe weather is eliminated, and the driving video like driving in the excellent weather is presented. The driving safety problem caused by visual field interference brought by severe weather to a driver in the driving process is reduced, so that safe driving of the driver is assisted, and the traffic accident rate is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving technology, and in particular to a method and system for assisting safe driving in bad weather. Background Art

[0002] With the popularity of cars, cars have become one of the main ways for people to travel. However, in actual travel, it is inevitable to encounter bad weather (such as rain, snow, fog, darkness, etc.). Bad weather causes great interference to the driver's field of vision, and also causes great safety hazards to traffic safety, which has become one of the major causes of traffic accidents. In the prior art, there is little reference to assisting the driver's field of vision observation in bad weather, so that even if the driver drives in bad weather, he or she will have the experience of driving in good weather (such as windy and sunny, etc.), so as to significantly improve driving safety, thereby effectively reducing the incidence of traffic accidents. Therefore, how to reduce the driving safety problems caused by the interference of the driver's field of vision caused by bad weather during driving, to assist the driver's safe driving, thereby significantly improving driving safety and effectively reducing the incidence of traffic accidents, has become a problem to be solved urgently.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a method and system for assisting safe driving in severe weather, aiming to solve the technical problem that the existing technology is unable to reduce the driving safety problem caused by the interference of the driver's vision caused by severe weather during driving, so as to assist the driver in safe driving, thereby significantly improving driving safety and effectively reducing the occurrence rate of traffic accidents.

[0005] To achieve the above object, the present invention provides a method for assisting safe driving in bad weather, the method comprising the following steps:

[0006] Perform frame extraction processing on driving videos in bad weather to obtain continuous frames of driving images in bad weather;

[0007] Performing image restoration processing on the driving image under bad weather conditions according to the driving image under good weather conditions to obtain a restored driving image;

[0008] A restored driving video is generated according to the restored driving image, and the restored driving video is displayed to assist the driver in achieving safe driving based on the restored driving video.

[0009] Optionally, the step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain a restored driving image comprises:

[0010] Performing image recognition processing on the driving image under inclement weather to identify nearby lane lines in the driving image under inclement weather;

[0011] Calculating a coordinate deviation between a position coordinate of a nearby lane line in the driving image under good weather and a position coordinate of a nearby lane line in the driving image under bad weather;

[0012] The distant lane line in the driving image under inclement weather is determined according to the coordinate deviation and the position coordinates of the distant lane line in the driving image under good weather, and the RGB value of the pixel point of the distant lane line in the driving image under good weather is assigned to the RGB value of the pixel point of the distant lane line in the driving image under inclement weather to obtain a restored driving image.

[0013] Optionally, the step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain a restored driving image comprises:

[0014] The RGB values ​​of the pixels in the area outside the lane lines in the driving image under good weather conditions are assigned to the RGB values ​​of the pixels in the area outside the lane lines in the driving image under bad weather conditions to obtain a restored driving image.

[0015] Optionally, the step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain a restored driving image comprises:

[0016] Performing image recognition processing on the inner area of ​​the lane line in the driving image under the inclement weather to identify the target obstacle in the inner area of ​​the lane line in the driving image under the inclement weather;

[0017] The RGB values ​​of the pixel points in the area inside the lane line excluding the target obstacle in the driving image under good weather conditions are assigned to the RGB values ​​of the pixel points in the area inside the lane line excluding the target obstacle in the driving image under bad weather conditions to obtain a restored driving image.

[0018] Optionally, the step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain a restored driving image comprises:

[0019] The obstacle with the highest similarity to the target obstacle is matched from a preset image library, and the RGB value of the pixel points of the outer contour of the target obstacle is replaced according to the RGB value of the pixel points of the outer contour of the obstacle with the highest similarity, so as to obtain a restored driving image.

[0020] In addition, to achieve the above-mentioned purpose, the present invention also proposes a safe driving assistance system in bad weather, the safe driving assistance system in bad weather comprising:

[0021] An image acquisition module is used to extract frames from driving videos in bad weather to obtain continuous frames of driving images in bad weather;

[0022] An image restoration module, used for performing image restoration processing on the driving image under bad weather conditions according to the driving image under good weather conditions, to obtain a restored driving image;

[0023] The video generation module generates a restored driving video according to the restored driving image, and displays the restored driving video to assist the driver to achieve safe driving based on the restored driving video.

[0024] Optionally, the image restoration module is further used to perform image recognition processing on the driving image under inclement weather to identify nearby lane lines in the driving image under inclement weather;

[0025] The image restoration module is further used to calculate the coordinate deviation between the position coordinates of the nearby lane line in the driving image under good weather conditions and the position coordinates of the nearby lane line in the driving image under bad weather conditions;

[0026] The image restoration module is also used to determine the distant lane line in the driving image under bad weather conditions based on the coordinate deviation and the position coordinates of the distant lane line in the driving image under good weather conditions, and assign the RGB values ​​of the pixel points of the distant lane line in the driving image under good weather conditions to the RGB values ​​of the pixel points of the distant lane line in the driving image under bad weather conditions, so as to obtain a restored driving image.

[0027] Optionally, the image restoration module is also used to assign the RGB values ​​of the pixel points in the area outside the lane line in the driving image in good weather to the RGB values ​​of the pixel points in the area outside the lane line in the driving image in bad weather to obtain a restored driving image.

[0028] Optionally, the image restoration module is further used to perform image recognition processing on the inner area of ​​the lane line in the driving image under inclement weather, so as to identify the target obstacle in the inner area of ​​the lane line in the driving image under inclement weather;

[0029] The image restoration module is also used to assign the RGB values ​​of the pixel points in the area inside the lane line except the target obstacle in the driving image under good weather conditions to the RGB values ​​of the pixel points in the area inside the lane line except the target obstacle in the driving image under bad weather conditions to obtain a restored driving image.

[0030] Optionally, the image restoration module is further used to match the obstacle with the highest similarity to the target obstacle from a preset library, and replace the RGB values ​​of the pixel points of the outer contour of the target obstacle according to the RGB values ​​of the pixel points of the outer contour of the obstacle with the highest similarity, so as to obtain a restored driving image.

[0031] In the present invention, the driving video in bad weather is subjected to frame extraction processing to obtain continuous frames of driving images in bad weather; the driving images in bad weather are subjected to image restoration processing according to the driving images in good weather to obtain the restored driving images; the restored driving video is generated according to the restored driving images, and the restored driving video is displayed to assist the driver to achieve safe driving based on the restored driving video. By performing image restoration on the driving images in bad weather according to the driving images in good weather, and generating a driving video based on the restored driving images, the influence of interference factors such as rain, snow, fog and darkness in bad weather is eliminated, and a driving video as if driving in good weather is presented, thereby reducing the driving safety problems caused by the interference of the driver's vision caused by bad weather during driving, so as to assist the driver in safe driving, thereby significantly improving driving safety and effectively reducing the incidence of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of a first embodiment of a method for assisting safe driving in bad weather according to the present invention;

[0033] Figure 2 A schematic diagram showing the comparison of images before and after restoration according to the first embodiment of the method for assisting safe driving in bad weather according to the present invention;

[0034] Figure 3 It is a structural block diagram of the first embodiment of the safe driving assistance system in severe weather of the present invention.

[0035] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0037] The embodiment of the present invention provides a method for assisting safe driving in bad weather. Figure 1 , Figure 1 Schematic diagram of the flow chart of the first embodiment of the method for assisting safe driving in bad weather according to the present invention.

[0038] In this embodiment, the method for assisting safe driving in bad weather includes the following steps:

[0039] Step S10: performing frame extraction processing on the driving video under inclement weather to obtain continuous frames of driving images under inclement weather;

[0040] It is easy to understand that this embodiment needs to first collect driving videos of highways or urban roads in good weather, and record the position coordinates frame by frame through the vehicle-mounted GPS system, and then perform frame extraction processing on the driving videos in good weather to convert them into continuous frame driving images. Among them, the good weather described in this embodiment refers to the situation where the weather is clear, the wind is gentle, the lane lines (i.e., road traffic markings) are clearly visible, and the driver's field of vision is wide without interference or obstruction. The driving videos in good weather can be collected by the developer. In the specific implementation, the road network data of the city can be directly collected manually, such as driving videos of highways, urban expressways, urban main roads, urban secondary roads and other roads with lane lines in good weather, and it can also be checked for omissions or updated in time based on the existing road network data.

[0041] Accordingly, the bad weather described in this example refers to the situation where the lane line is not clearly visible or is covered, the driver's field of vision is blocked, and there is interference or obstruction, such as rain, snow, fog, dim vision, etc. The collection of driving video in bad weather is the video collected by the driver's driving recorder in bad weather such as rain, snow, fog, etc. Then, the driving video in bad weather can be frame-drawn to obtain continuous frames of driving images in bad weather.

[0042] In the specific implementation, when matching driving images in good weather and driving images in bad weather, that is, matching driving images in the same road section, therefore, the collection of driving videos in good weather should try to ensure that all lanes on the road are collected, so that any coordinate position of the driving video collected by the driver during the actual driving process can be matched. If there are still unmatched sections or missing sections, image similarity matching can be performed based on the existing driving images in good weather in the gallery, so as to perform subsequent image restoration operations based on the image with the highest matching degree. For example, the feature information of surrounding static obstacles, such as type, quantity, pixel distance, etc., can be detected by the target detection algorithm (such as YOLOv8), and then image similarity matching can be performed based on the extracted feature information, so as to perform subsequent image restoration operations based on the image with the highest matching degree.

[0043] Step S20: performing image restoration processing on the driving image under bad weather conditions according to the driving image under good weather conditions to obtain a restored driving image;

[0044] It should be noted that the image restoration described in this embodiment, that is, image restoration processing of driving images in bad weather, allows the driver to experience driving in good weather even in bad weather, thereby significantly improving driving safety and effectively reducing the incidence of traffic accidents. Figure 2 , Figure 2 Schematic diagram showing comparison of images before and after restoration according to the first embodiment of the method for assisting safe driving in bad weather of the present invention. Figure 2 The final effect is that the pixel points of the area other than pedestrians and cars in the lane replace the pixel points of the area corresponding to good weather conditions, so as to retain the main obstacles and not affect driving safety. The interference factors in bad weather (such as rain, snow, fog, darkness, etc.) have been eliminated to present the visual effect of driving in good weather conditions.

[0045] In this embodiment, the image recognition processing can be performed on the driving image under bad weather to identify the near lane line in the driving image under bad weather; then, the coordinate deviation between the position coordinates of the near lane line in the driving image under good weather and the position coordinates of the near lane line in the driving image under bad weather is calculated; then, the far lane line in the driving image under bad weather is determined according to the coordinate deviation and the position coordinates of the far lane line in the driving image under good weather, and the RGB value of the pixel point of the far lane line in the driving image under good weather is assigned to the RGB value of the pixel point of the far lane line in the driving image under bad weather, so as to restore the lane line in the driving image under bad weather, thereby obtaining the restored driving image. Among them, the near lane line and the far lane line can be distinguished based on the line of sight of the vehicle-mounted camera, that is, the near lane line is within the line of sight of the vehicle-mounted camera, and the far lane line is beyond the line of sight of the vehicle-mounted camera; it can also be distinguished based on whether it can be recognized, that is, the recognizable lane line is the near lane line, and the unrecognizable lane line is the far lane line.

[0046] In a specific implementation, an algorithm suitable for lane line detection (such as LaneNet) can be used to detect the position coordinates of all pixel points on the lane line in the driving image under good weather and bad weather, and the position coordinates of the lane line are saved in a txt file, wherein the lane line recognition of the driving image under good weather can be pre-recognized and stored for subsequent direct use. LaneNet regards lane line detection in an image as an instance segmentation problem and is an end-to-end model. The lane line pixels and the lane line ID corresponding to each pixel can be directly output based on the input image without artificially designed features. Further, after extracting the lane pixels and lane ID, the image can be projected to a bird's-eye view to further complete the lane line fitting (mainly cubic polynomial fitting). The traditional approach is to use a fixed transformation matrix to perform perspective transformation on the image. This method has a large deviation when the road surface is uneven. In this embodiment, a neural network named H-Net can be trained, the current image is input to H-Net, and then the neural network outputs the corresponding bird's-eye transformation matrix. After completing the bird's-eye transformation, a quadratic or cubic polynomial is fitted using the least squares method to complete the lane line detection.

[0047] Furthermore, the driving images under good weather and bad weather under the same GPS position coordinates can be found. The difference between the two images under the same position coordinates is the smallest, which is helpful for the image restoration effect under bad weather. The position coordinate information of the lane line saved in txt under two weather conditions is read respectively, and then the RGB values ​​of the pixel points of the driving images under good weather and bad weather under the same GPS position coordinates are read respectively. When restoring the lane line position information, considering that the lane line is static and the position does not change over time, the near and far lane lines can be recognized in good weather, while the far lane lines will be unclear in bad weather, and the recognition of the near lane lines is no problem. Therefore, the coordinates of the nearby lane lines under the two weather conditions can be compared, and the deviation of the lane line coordinates of the two lane lines at the same horizontal pixel position can be calculated. The coordinates of the distant lane lines under good weather conditions are then added with the deviation, and the obtained lane line coordinates are assigned to the distant lane line coordinates under bad weather conditions. At the same time, the RGB values ​​of the pixels at the distant lane lines under good weather conditions are assigned to the RGB values ​​of the pixels at the corrected distant lane lines in the driving image under bad weather conditions. As a result, the distant lane lines under bad weather conditions will be displayed on the driving image, thus realizing the restoration of the lane lines in the driving image under bad weather conditions.

[0048] In this embodiment, the RGB values ​​of the pixels in the area outside the lane lines in the driving image under good weather conditions may be assigned to the RGB values ​​of the pixels in the area outside the lane lines in the driving image under bad weather conditions to obtain a restored driving image.

[0049] In a specific implementation, when restoring the environment information outside the road, the RGB values ​​of the pixels of the environment information outside the left and right lane lines in good weather can be assigned to the RGB values ​​of the pixels of the environment information outside the left and right lane lines in bad weather. It should be noted that the left and right lane lines in bad weather are repaired after the above operation. Considering the accuracy of GPS coordinates, the replacement of these environmental pixels has no effect even if there is a small deviation compared with the real environment, because the environment outside the road has little impact on the driver's driving safety. The main threat lies in the unclear position of the lane line inside the road and the unclear identification of traffic participants in the distance.

[0050] In this embodiment, image recognition processing can also be performed on the inner area of ​​the lane line in the driving image under inclement weather to identify the target obstacle in the inner area of ​​the lane line in the driving image under inclement weather; the RGB values ​​of the pixel points in the area inside the lane line excluding the target obstacle in the driving image under good weather are assigned to the RGB values ​​of the pixel points in the area inside the lane line excluding the target obstacle in the driving image under inclement weather to obtain a restored driving image.

[0051] In the specific implementation, an open source semantic segmentation algorithm suitable for obstacle detection (such as DeepLabV3) can be used to detect the position coordinates of the target obstacle in the driving image, and the coordinates of the target obstacle in the image can be saved in a txt file. Among them, the semantic segmentation algorithm can finely identify the outer contour coordinates of the target obstacle. The target obstacles mainly include common obstacles in road scenes such as cars and pedestrians, such as vehicles of different colors, sizes, and models, and pedestrians of different forms. DeepLabV3 is a semantic segmentation network model. When using DeepLabV3 for real-time road obstacle recognition, a pre-trained DeepLabV3 model can be obtained from a deep learning framework (such as TensorFlow or PyTorch). These models are usually trained on large-scale data sets (such as COCO, PASCAL VOC). Next, collect image data containing road scenes to ensure that the image data set contains various obstacles (such as pedestrians, vehicles, buildings, trees, etc.). Then annotate the collected data to ensure that each pixel is labeled as a specific category. Tools such as LabelMe, VIA, LabelImg, etc. can be used for annotation. In the specific implementation, in order to improve the generalization ability of the model, data enhancement operations such as rotation, scaling, cropping and color adjustment can be performed on the training data. Then, the data set can be divided into training set, validation set and test set. 80% can be set for training, 10% for validation and 10% for testing. The DeepLabV3 model can be trained using the labeled data set. For example, the API provided by the deep learning framework (such as TensorFlow's tf.data.Dataset API or PyTorch's DataLoader) can be used to load data and train the model. Among them, appropriate hyperparameters such as learning rate and batch size can be set according to requirements. The model is then evaluated using the validation set and test set to ensure that it performs well on unknown data. Finally, the trained DeepLabV3 model can be integrated in real-time applications. For example, TensorFlow Serving, ONNXRuntime or other inference engines can be used to accelerate model inference. Thus, each frame of the acquired real-time video stream can be passed to the model for prediction to obtain obstacle detection results.

[0052] Furthermore, when restoring the road surface inside the lane line, the RGB values ​​of the pixels on the inside of the left and right lane lines except the detected target obstacles in good weather can be assigned to the RGB values ​​of the pixels on the inside of the left and right lane lines except the detected target obstacles in bad weather, and the unassigned pixels can directly use the RGB values ​​of the assigned pixels next to them, because in actual operation, the difference between the RGB values ​​of pixels at different positions in the area inside the lane line except the detected target obstacles can be ignored when presented on the vehicle screen.

[0053] In this embodiment, the obstacle with the highest similarity to the target obstacle can also be matched from the preset library, and the RGB value replacement processing is performed on the pixel points of the outer contour of the target obstacle according to the RGB values ​​of the pixel points of the outer contour of the obstacle with the highest similarity, so as to obtain a restored driving image.

[0054] In the specific implementation, the image similarity method can be used to match the obstacle with the highest similarity to the target obstacle detected in bad weather from the massive image library, extract the RGB value of its outer contour pixel point, and then replace the RGB value of all pixel points corresponding to the position coordinates of the outer contour of the target obstacle identified by the semantic segmentation algorithm in bad weather. It should be noted that the accuracy of identifying nearby target obstacles in bad weather is good, while the accuracy of identifying distant target obstacles is poor. The semantic segmentation algorithm will inevitably miss and misidentify distant target obstacles. However, with the development of the semantic segmentation algorithm, the contour recognition of distant target obstacles in bad weather will become more and more accurate. At present, considering that it is impossible to accurately identify 100%, for the sake of driving safety, it is better to misidentify than miss, otherwise it will cause safety hazards. Therefore, the threshold of the semantic segmentation algorithm is reduced to reduce the phenomenon of missed recognition, thereby improving driving safety.

[0055] Step S30: generating a restored driving video according to the restored driving image, and displaying the restored driving video, so as to assist the driver to achieve safe driving based on the restored driving video.

[0056] In this example, a restored driving video can be generated based on the restored driving image obtained through the above operation, and the restored driving video can be displayed on the in-vehicle screen to assist the driver in achieving safe driving based on the restored driving video. In the restored driving video, the pixel points of the area in the lane other than the target obstacle have been replaced with the pixel points of the area corresponding to good weather, and interference factors such as rain, snow, fog and darkness have been eliminated, presenting a driving video as if driving in good weather.

[0057] In this embodiment, the driving video in bad weather is subjected to frame extraction processing to obtain continuous frames of driving images in bad weather; the driving image in bad weather is subjected to image restoration processing according to the driving image in good weather to obtain the restored driving image; the restored driving video is generated according to the restored driving image, and the restored driving video is displayed to assist the driver to achieve safe driving based on the restored driving video. By performing image restoration on the driving image in bad weather according to the driving image in good weather, and generating a driving video based on the restored driving image, the influence of interference factors such as rain, snow, fog and darkness in bad weather is eliminated, and a driving video as if driving in good weather is presented, thereby reducing the driving safety problem caused by the interference of the driver's vision caused by bad weather during driving, so as to assist the driver in safe driving, thereby significantly improving driving safety and effectively reducing the incidence of traffic accidents.

[0058] Reference Figure 3 , Figure 3 It is a structural block diagram of the first embodiment of the safe driving assistance system in severe weather of the present invention.

[0059] like Figure 3 As shown, the bad weather safety driving assistance system proposed in the embodiment of the present invention includes:

[0060] The image acquisition module 10 is used to perform frame extraction processing on the driving video under bad weather conditions to obtain continuous frames of driving images under bad weather conditions;

[0061] An image restoration module 20 is used to perform image restoration processing on the driving image under bad weather conditions according to the driving image under good weather conditions to obtain a restored driving image;

[0062] The video generation module 30 generates a restored driving video according to the restored driving image, and displays the restored driving video to assist the driver in achieving safe driving based on the restored driving video.

[0063] Optionally, the image restoration module 20 is further used to perform image recognition processing on the driving image under inclement weather to identify nearby lane lines in the driving image under inclement weather;

[0064] The image restoration module 20 is further used to calculate the coordinate deviation between the position coordinates of the nearby lane line in the driving image under good weather and the position coordinates of the nearby lane line in the driving image under bad weather;

[0065] The image restoration module 20 is also used to determine the distant lane line in the driving image under bad weather conditions based on the coordinate deviation and the position coordinates of the distant lane line in the driving image under good weather conditions, and assign the RGB values ​​of the pixel points of the distant lane line in the driving image under good weather conditions to the RGB values ​​of the pixel points of the distant lane line in the driving image under bad weather conditions, so as to obtain a restored driving image.

[0066] Optionally, the image restoration module 20 is further used to assign the RGB values ​​of the pixel points in the area outside the lane line in the driving image under good weather conditions to the RGB values ​​of the pixel points in the area outside the lane line in the driving image under bad weather conditions to obtain a restored driving image.

[0067] Optionally, the image restoration module 20 is further used to perform image recognition processing on the inner area of ​​the lane line in the driving image under inclement weather, so as to identify the target obstacle in the inner area of ​​the lane line in the driving image under inclement weather;

[0068] The image restoration module 20 is also used to assign the RGB values ​​of the pixel points in the area inside the lane line except the target obstacle in the driving image under good weather conditions to the RGB values ​​of the pixel points in the area inside the lane line except the target obstacle in the driving image under bad weather conditions to obtain a restored driving image.

[0069] Optionally, the image restoration module 20 is further used to match the obstacle with the highest similarity to the target obstacle from a preset library, and replace the RGB values ​​of the pixel points of the outer contour of the target obstacle according to the RGB values ​​of the pixel points of the outer contour of the obstacle with the highest similarity, so as to obtain a restored driving image.

[0070] In this embodiment, the driving video in bad weather is subjected to frame extraction processing to obtain continuous frames of driving images in bad weather; the driving image in bad weather is subjected to image restoration processing according to the driving image in good weather to obtain the restored driving image; the restored driving video is generated according to the restored driving image, and the restored driving video is displayed to assist the driver to achieve safe driving based on the restored driving video. By performing image restoration on the driving image in bad weather according to the driving image in good weather, and generating a driving video based on the restored driving image, the influence of interference factors such as rain, snow, fog and darkness in bad weather is eliminated, and a driving video as if driving in good weather is presented, thereby reducing the driving safety problem caused by the interference of the driver's vision caused by bad weather during driving, so as to assist the driver in safe driving, thereby significantly improving driving safety and effectively reducing the incidence of traffic accidents.

[0071] Other embodiments or specific implementation methods of the safe driving assistance system in severe weather of the present invention may refer to the above-mentioned method embodiments and will not be described in detail here.

[0072] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0073] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0075] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A safe driving assistance method in bad weather, characterized in that: The method for assisting safe driving in bad weather comprises the following steps: Perform frame extraction processing on driving videos in bad weather to obtain continuous frames of driving images in bad weather; Performing image restoration processing on the driving image under bad weather conditions according to the driving image under good weather conditions to obtain a restored driving image; A restored driving video is generated according to the restored driving image, and the restored driving video is displayed to assist the driver in achieving safe driving based on the restored driving video.

2. The method for assisting safe driving in bad weather as claimed in claim 1, characterized in that: The step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain the restored driving image comprises: Performing image recognition processing on the driving image under inclement weather to identify nearby lane lines in the driving image under inclement weather; Calculating a coordinate deviation between a position coordinate of a nearby lane line in the driving image under good weather and a position coordinate of a nearby lane line in the driving image under bad weather; The distant lane line in the driving image under inclement weather is determined according to the coordinate deviation and the position coordinates of the distant lane line in the driving image under good weather, and the RGB value of the pixel point of the distant lane line in the driving image under good weather is assigned to the RGB value of the pixel point of the distant lane line in the driving image under inclement weather to obtain a restored driving image.

3. The method for assisting safe driving in bad weather as claimed in claim 2, characterized in that: The step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain the restored driving image comprises: The RGB values ​​of the pixels in the area outside the lane lines in the driving image under good weather conditions are assigned to the RGB values ​​of the pixels in the area outside the lane lines in the driving image under bad weather conditions to obtain a restored driving image.

4. The method for assisting safe driving in bad weather as claimed in claim 3, characterized in that: The step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain the restored driving image comprises: Performing image recognition processing on the inner area of ​​the lane line in the driving image under the inclement weather to identify the target obstacle in the inner area of ​​the lane line in the driving image under the inclement weather; The RGB values ​​of the pixel points in the area inside the lane line excluding the target obstacle in the driving image under good weather conditions are assigned to the RGB values ​​of the pixel points in the area inside the lane line excluding the target obstacle in the driving image under bad weather conditions to obtain a restored driving image.

5. The method for assisting safe driving in bad weather as claimed in claim 4, characterized in that: The step of performing image restoration processing on the driving image under bad weather conditions based on the driving image under good weather conditions to obtain the restored driving image comprises: The obstacle with the highest similarity to the target obstacle is matched from a preset image library, and the RGB value of the pixel points of the outer contour of the target obstacle is replaced according to the RGB value of the pixel points of the outer contour of the obstacle with the highest similarity, so as to obtain a restored driving image.

6. A safe driving assistance system in bad weather, characterized in that: The bad weather safety driving assistance system includes: An image acquisition module is used to extract frames from driving videos in bad weather to obtain continuous frames of driving images in bad weather; An image restoration module, used for performing image restoration processing on the driving image under bad weather conditions according to the driving image under good weather conditions, to obtain a restored driving image; The video generation module generates a restored driving video according to the restored driving image, and displays the restored driving video to assist the driver to achieve safe driving based on the restored driving video.

7. The bad weather driving assistance system according to claim 6, characterized in that: The image restoration module is further used to perform image recognition processing on the driving image under inclement weather to identify nearby lane lines in the driving image under inclement weather; The image restoration module is further used to calculate the coordinate deviation between the position coordinates of the nearby lane line in the driving image under good weather conditions and the position coordinates of the nearby lane line in the driving image under bad weather conditions; The image restoration module is also used to determine the distant lane line in the driving image under bad weather conditions based on the coordinate deviation and the position coordinates of the distant lane line in the driving image under good weather conditions, and assign the RGB values ​​of the pixel points of the distant lane line in the driving image under good weather conditions to the RGB values ​​of the pixel points of the distant lane line in the driving image under bad weather conditions, so as to obtain a restored driving image.

8. The bad weather driving assistance system according to claim 7, characterized in that: The image restoration module is also used to assign the RGB values ​​of the pixel points in the area outside the lane line in the driving image under good weather conditions to the RGB values ​​of the pixel points in the area outside the lane line in the driving image under bad weather conditions to obtain a restored driving image.

9. The bad weather safety driving assistance system according to claim 8, characterized in that: The image restoration module is further used to perform image recognition processing on the inner area of ​​the lane line in the driving image under inclement weather, so as to identify the target obstacle in the inner area of ​​the lane line in the driving image under inclement weather; The image restoration module is also used to assign the RGB values ​​of the pixel points in the area inside the lane line except the target obstacle in the driving image under good weather conditions to the RGB values ​​of the pixel points in the area inside the lane line except the target obstacle in the driving image under bad weather conditions to obtain a restored driving image.

10. The bad weather safety driving assistance system according to claim 9, characterized in that: The image restoration module is further used to match the obstacle with the highest similarity to the target obstacle from a preset library, and replace the RGB values ​​of the pixel points of the outer contour of the target obstacle according to the RGB values ​​of the pixel points of the outer contour of the obstacle with the highest similarity, so as to obtain a restored driving image.

Images