Assisted driving method, device and computer-readable medium based on machine learning

Abstract

The present invention relates to an assisted driving method, device and computer-readable medium based on machine learning. The method comprises: obtaining training data, the training data comprising multiple images of a target road section under different weather conditions, each of the images having a visibility; constructing a machine learning model, inputting the training data into the machine learning model, the output result of the machine learning model comprising the visibility of each of the images; obtaining a first image of the target road section, inputting the first image into the trained machine learning model, the first output result of the trained machine learning model comprising the current visibility of the first image; performing defogging processing on the first image according to the current visibility to obtain a second image; and providing visualization information on a vehicle-mounted display device according to the second image.

Description

Technical Field

[0001] The present invention mainly relates to the field of smart cars, and specifically to an assisted driving method, device and computer-readable medium based on machine learning. Background Art

[0002] With the rapid development of new energy vehicles and smart cars, in addition to traditional driving functions, cars have more and more functions such as automatic driving and computer-assisted driving, which not only provide drivers with a variety of auxiliary functions, but more importantly, improve driving safety. The weather in nature is ever-changing. In bad weather such as fog, rain, snow, etc., factors such as reduced visibility, slippery ground, and complex environment will affect the driver's observation and judgment, making traffic accidents more likely to occur. At present, there is no effective solution to help drivers eliminate the impact of the environment in bad weather. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a machine learning-based assisted driving method, device and computer-readable medium for improving driving safety in severe weather.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is an assisted driving method based on machine learning, comprising: obtaining training data, the training data including multiple images of a target road section under different weather conditions, each of the image having a visibility; constructing a machine learning model, inputting the training data into the machine learning model, the output result of the machine learning model including the visibility of each of the images; obtaining a first image of the target road section, inputting the first image into the trained machine learning model, the first output result of the trained machine learning model including the current visibility of the first image; defogging the first image according to the current visibility to obtain a second image; and providing visualization information on a vehicle-mounted display device according to the second image.

[0005] In one embodiment of the present application, it also includes: acquiring multiple sets of radar data of the vehicle while acquiring the multiple images, the radar data is used to provide the obstacle distribution around the vehicle, and the multiple sets of radar data correspond to the multiple images one by one; determining the driver's visible range according to the obstacle distribution and the visibility; the training data includes the multiple sets of radar data and the visible range; the output result of the machine learning model also includes the visible range corresponding to each of the images; acquiring the first radar data of the vehicle while acquiring the first image of the target road section, inputting the first image and the first radar data into the trained machine learning model, and the second output result of the trained machine learning model includes the current visible range; generating a third image according to the second image and the current visible range; and providing visualization information on the vehicle-mounted display device according to the third image.

[0006] In one embodiment of the present application, the step of defogging the first image according to the current visibility includes: obtaining a clear image of the target road section on a sunny day; grouping each of the images in the training data according to the visibility, the visibility of the images in the same visibility group is within a corresponding preset range, and each visibility group includes at least 2 images; respectively calculating the similarity between multiple images in each visibility group and the clear image, and obtaining a target image with the maximum similarity in each group; extracting similar feature parameters based on the target image and the clear image, each visibility group has one similar feature parameter; processing all images in the visibility group according to the similar feature parameters to obtain the second image.

[0007] In one embodiment of the present application, sources of the multiple images include: one or more of a vehicle-mounted camera, a road-mounted camera, and a mobile terminal.

[0008] In one embodiment of the present application, the vehicle-mounted display device includes a large vehicle-mounted screen, and the step of providing visual information on the vehicle-mounted display device according to the second image includes: displaying the second image on the large vehicle-mounted screen.

[0009] In one embodiment of the present application, the vehicle-mounted display device includes a vehicle-mounted large screen, and the step of providing visual information on the vehicle-mounted display device according to the third image includes: displaying the third image on the vehicle-mounted large screen.

[0010] In an embodiment of the present application, the third image includes a simulated image of the vehicle and visualized first radar data surrounding the vehicle.

[0011] In an embodiment of the present application, the different weather conditions include at least one of sunny days, rainy days, snowy days, and foggy days.

[0012] In order to solve the above technical problems, the present application also proposes a driving assistance device based on machine learning, which is characterized by comprising: an image acquisition unit, a machine learning model, a defogging processing unit and a vehicle-mounted display device.

[0013] The image acquisition unit is used to acquire a first image of the target road section; the machine learning model is used to output the current visibility of the first image based on the input first image; the defogging unit is used to perform defogging on the first image according to the current visibility to obtain a second image; and the vehicle-mounted display device is used to provide visualization information on the vehicle-mounted display device based on the second image.

[0014] In one embodiment of the present application, it also includes: a radar data acquisition unit, used to acquire first radar data of the vehicle while the image acquisition unit acquires the first image; the machine learning model is also used to output the driver's current visual range based on the first image and the first radar data.

[0015] In order to solve the above technical problems, the present application also proposes an assisted driving device based on machine learning, comprising: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method described above.

[0016] In order to solve the above technical problem, the present application further proposes a computer-readable medium storing a computer program code, and the computer program code implements the above method when executed by a processor.

[0017] The machine learning-based assisted driving method of the present application can obtain the visibility of the first image acquired by the vehicle in real time through a machine learning model, and defog the first image according to the visibility to obtain a relatively clear second image, and intuitively display it to the driver through the on-board display device, thereby providing the driver with clear and safe driving information. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein:

[0019] Figure 1 is an exemplary flow chart of an assisted driving method based on machine learning according to an embodiment of the present application;

[0020] Figure 2 is an exemplary flow chart of performing defogging processing on a first image according to current visibility in an assisted driving method according to an embodiment of the present application;

[0021] Figure 3is a block diagram of a driving assistance device based on machine learning according to an embodiment of the present application;

[0022] Figure 4 It is a system block diagram of a driving assistance device based on machine learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and therefore the present application is not limited to the specific embodiments disclosed below.

[0025] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0026] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0027] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0028] The machine learning-based assisted driving method, device, and computer-readable medium of the present application are suitable for safe driving in adverse weather conditions with reduced visibility, such as fog, rain, snow, etc.

[0029] Figure 1 is an exemplary flow chart of an assisted driving method based on machine learning according to an embodiment of the present application. Figure 1 As shown, the assisted driving method of this embodiment includes the following steps:

[0030] Step S110: acquiring training data, wherein the training data includes a plurality of images of a target road section under different weather conditions, each of the images having a visibility;

[0031] Step S120: constructing a machine learning model, inputting training data into the machine learning model, and outputting the machine learning model to include visibility of each image;

[0032] Step S130: acquiring a first image of the target road section, and inputting the first image into a trained machine learning model, wherein a first output result of the trained machine learning model includes current visibility of the first image;

[0033] Step S140: performing defogging processing on the first image according to the current visibility to obtain a second image; and

[0034] Step S150: providing visual information on the vehicle-mounted display device according to the second image.

[0035] The above steps S110 - S150 are described below with reference to the accompanying drawings.

[0036] In step S110, the training data refers to data used to train the machine learning model. The target road section is used to clarify the location of the image to be captured. This application does not limit the location, characteristics, etc. of the target road section. It can be a road section of interest or a section with high accident rates, such as an intersection, a bend, etc.

[0037] This application does not limit the method of obtaining multiple images as training data.

[0038] In some embodiments, the sources of the multiple images include: one or more of a vehicle-mounted camera, a road-mounted camera, and a mobile terminal.

[0039] In some embodiments, multiple images of the target road section under different weather conditions are obtained by a vehicle-mounted camera. In these embodiments, multiple images of the same target road section can be obtained by vehicle-mounted cameras on multiple different vehicles. It is understandable that due to differences in vehicle speed, environment, etc., there are also some differences in the multiple images of the same target road section obtained by different vehicles. The environment here refers to the environment around the vehicle, including other vehicles, pedestrians, non-motor vehicles, etc. However, since they are all images obtained on the target road section, these images have a certain correlation.

[0040] In some embodiments, multiple images of the target road section under different weather conditions are obtained by a road-mounted camera device. In these embodiments, the road-mounted camera device can be, for example, a camera fixed on the roadside, such as a camera installed on a traffic light, or a special traffic camera, etc. For these embodiments, since the position of the road-mounted camera device is usually fixed, the images obtained have a higher correlation than those of the vehicle-mounted camera. The multiple images of the target road section obtained by the road-mounted camera device will also be affected by the above-mentioned environment, so that different images will have certain differences.

[0041] In some embodiments, the multiple images are from a mobile terminal, which includes but is not limited to a mobile terminal of a driver or a vehicle occupant, or a vehicle-mounted mobile terminal.

[0042] In some embodiments, the different weather conditions in step S110 include at least one of sunny days, rainy days, snowy days, and foggy days.

[0043] In step S110, each image has a visibility. The visibility refers to the maximum distance at which a person with normal vision can identify the target object from the background. That is, during the day, with the sky close to the horizon as the background, the outline of the gray target object on the ground with a viewing angle greater than 20 degrees can be clearly seen and what object it is, and at night, the light point of the target light can be clearly seen. The unit is m (meter). The size of visibility is mainly determined by two factors: ① The brightness difference between the target object and the background that sets off it. The larger (smaller) the difference, the larger (smaller) the visible distance. But this brightness difference usually does not change much. ② Atmospheric transparency. The air layer between the observer and the target object can reduce the aforementioned brightness difference. The worse (better) the atmospheric transparency, the smaller (larger) the visible distance.

[0044] In bad weather, especially foggy days, the transparency of the atmosphere is greatly reduced compared to sunny days, resulting in low visibility, making it difficult for drivers to judge the surrounding environment, causing driving difficulties and prone to traffic accidents.

[0045] Visibility is generally measured by visual inspection, but it can also be measured using instruments, such as atmospheric projectors, laser visibility automatic measuring instruments, and other instruments.

[0046] This application does not limit the method for obtaining visibility.

[0047] In some embodiments, when acquiring training data, a visibility measuring instrument is simultaneously set up on the target road section, and the image acquisition time and the measurement time of the visibility measuring instrument are matched, so that each image can have corresponding visibility.

[0048] In some embodiments, visibility data is centrally obtained by a meteorological center, and the visibility corresponding to the image can be obtained from a database of the meteorological center according to the time and location of the obtained image.

[0049] In step S120, the present application does not limit the specific real-time method of the machine learning model. The machine learning model can be a traditional neural network model, and can also be constructed using principles such as support vector machines, logistic regression, decision trees, Bayesian, etc. A deep learning model can also be used to jointly implement the machine learning model of the present application using a variety of different algorithms and methods.

[0050] In step S120, the training data acquired in step S110 is input into the machine learning model, and the output result of the machine learning model includes the visibility of each image in the training data.

[0051] In some embodiments, step S120 also includes testing the machine learning model. During the test, a portion of the data can be selected as test data, and the test data is similar to the training data in step S110, which is a plurality of images of the target road section under different weather conditions. However, the test data is not the training data. After testing the test data, the machine learning model can be optimized according to the test results so that it can be classified according to the visibility of the image, thereby obtaining the visibility of the image.

[0052] In step S130, a first image of the target road section is acquired.

[0053] In some embodiments, the first image is obtained by a vehicle-mounted camera device rather than a road-mounted camera device.

[0054] This application does not limit the specific installation method of the vehicle-mounted camera device on the vehicle. For example, the vehicle-mounted camera device can be installed inside or outside the vehicle body, and according to its installation angle, the vehicle-mounted camera device can obtain images from different directions and angles such as the front, rear, and side of the vehicle.

[0055] The first image is input into the machine learning model trained by step S120, and the machine learning model outputs a first output result, which includes the current visibility of the first image.

[0056] According to step S130, any first image of the target road section input to the machine learning model can be processed to obtain the current visibility corresponding to the first image. It can be understood that if the first image is obtained on a sunny day, the current visibility is high; if the first image is obtained on a foggy day, the current visibility is low.

[0057] In step S140, the first image is defogged according to the obtained current visibility to obtain a second image.

[0058] This application does not limit how to perform defogging on the first image. The first image can be processed by various image processing methods in the field of image processing, such as image enhancement, segmentation, and sharpening, so that the visibility of the first image with low visibility can be improved, and the recognition of the feature image in the first image can be improved. In the embodiments of this application, the feature image refers to image content that has an impact on the driving process, including: traffic lights, pedestrians, various vehicles, roadblocks, etc.

[0059] Figure 2 is an exemplary flow chart of performing defogging on a first image according to current visibility in an assisted driving method according to an embodiment of the present application. Figure 2 As shown, in this embodiment, the step of performing defogging processing on the first image according to the current visibility in step S140 includes:

[0060] Step S141: Acquire a clear image of the target road section on a sunny day;

[0061] Step S142: grouping each image in the training data according to visibility, the visibility of images in the same visibility group is within a corresponding preset range, and each visibility group includes at least 2 images;

[0062] Step S143: calculating the similarities between the multiple images in each visibility group and the clear image respectively, and obtaining the target image with the maximum similarity in each group;

[0063] Step S144: extracting similar feature parameters according to the target image and the clear image, each visibility group has one similar feature parameter;

[0064] Step S145: Process all images in the visibility group according to similar feature parameters to obtain a second image.

[0065] The above steps S141-S145 are described below respectively.

[0066] In step S141, the clear image can be obtained from the training data. Whether an image is a clear image can be determined based on the weather conditions when the image is obtained. For example, if the weather conditions at the time of image acquisition are sunny, the image is a clear image; if the weather conditions at the time of image acquisition are rainy or snowy, the image is not a clear image.

[0067] In some embodiments, whether an image is a clear image can be determined based on the visibility of the image. For example, a visibility threshold S is set, and if the visibility of the image is greater than S, the image is a clear image, and if the visibility of the image is less than S, the image is not a clear image.

[0068] In step S142, each image in the training data is grouped, for example, images with visibility in a first preset range F1 are grouped as F1, images with visibility in a second preset range F2 are grouped as F2, and so on.

[0069] The present application does not limit the specific value of the preset range of visibility. For example, the first preset range F1 is 0-10 meters, the second preset range is 10-30 meters, and so on.

[0070] According to step S142, images with similar visibility may be placed in the same visibility group.

[0071] In step S143, the similarities between the multiple images in each visibility group and the clear image are calculated respectively. The present application does not limit the method for calculating the similarities. For example, if there are n images in a visibility group, then in step S143, the similarities between the n images and a clear image are calculated respectively, and n similarities can be obtained. When there are m clear images, the similarities between the n images and the m clear images are calculated respectively, and n*m similarities can be obtained.

[0072] In some embodiments, the similarity is calculated by calculating the mutual information or fuzzy mutual information between the multiple images and the clear image in each visibility group. The calculation of the fuzzy mutual information is as follows:

[0073] Let image E A 、E B The grayscale value range is {a1, a2, ..., a k}{b1,b2,…,b J}, the joint probability distribution is {P(a k ,b j )}, k=1,2,…,K; j=1,2,…,J, For its grayscale registration relationship, we define image E A 、E B The fuzzy mutual information is:

[0074]

[0075] For image E A 、E B Grayscale registration correlation, indicating a k and b j The degree of registration relationship. The following two conditions are met:

[0076] (1)

[0077] (2)

[0078] when hour,

[0079]

[0080] Formula (2) indicates that mutual information I(A, B) is a special case when the grayscale registration correlation function of fuzzy mutual information fI(A; B) takes the value of 1, that is, fuzzy mutual information fI(A; B) is more universal.

[0081] In the field of image registration, when the spatial position of the image to be registered is exactly the same as the target image, the information expressed between the images is the largest, and the grayscale registration correlation of the grayscale sequence reaches the maximum, and the fuzzy mutual information reaches the maximum value. Therefore, the fuzzy mutual information can be used as a registration measure for image registration, and the position with the maximum fuzzy mutual information is the registration position of the image.

[0082] According to the above characteristics of fuzzy mutual information, in step S142, the fuzzy mutual information between the multiple images and the clear image is calculated, and the obtained result is used as the similarity consideration between the multiple images and the clear image. Obtaining the target image with the maximum similarity in each group can be obtaining the target image with the maximum fuzzy mutual information in each group.

[0083] Continuing with the previous example, in a visibility group with n images, the target image is the image with the largest fuzzy mutual information among the n fuzzy mutual information. When there are m clear images, a fuzzy mutual information matrix of n rows and m columns can be obtained, where the i-th row corresponds to the i-th image, i=1:n. The average or maximum value of the m fuzzy mutual information of the i-th row is obtained as the similarity value of the i-th row. By comparing the n similarity values, the target image with the largest similarity value can be determined.

[0084] In step S144, the grayscale values ​​of the target image and the clear image may be compared to extract similar feature parameters. For example, the grayscale value relationship between the target image and the clear image may be established in the following manner:

[0085] f(x)=a0+a1*g(x) (3)

[0086] Among them, f(x) is used to represent the gray value of the clear image at position x, g(x) is used to represent the gray value of the target image at position x, and a0 and a1 are similar feature parameters. The position x can be numbered from the upper left corner of the image, first row and then column, to represent the position of each pixel.

[0087] Substituting the specific values ​​of f(x) and g(x) into the above formula (3) can obtain similar characteristic parameters a0 and a1.

[0088] It should be noted that formula (3) is only an example. In other embodiments, other linear or nonlinear formulas may be used to obtain corresponding similarity characteristic parameters.

[0089] In step S145, all images in each visibility group are processed according to similar feature parameters.

[0090] For example, continuing with the example shown in formula (3), according to the obtained similar feature parameters a0 and a1, the grayscale values ​​of all images in a group are substituted into formula (3), and the second image corresponding to each image can be obtained in reverse. Compared with the original image, the second image is clearer and the unclear factors in the original image are removed.

[0091] According to the above steps S142 - S145 , a correspondingly clear second image can be obtained for each first image.

[0092] Thus, in step S150 , visual information is provided on the vehicle-mounted display device according to the second image.

[0093] In some embodiments, step S150 of displaying the second image directly on the vehicle-mounted display device.

[0094] In some embodiments, step S150 refers to further processing the second image based on the second image before displaying it on the vehicle-mounted display device.

[0095] In some embodiments, the vehicle-mounted display device includes a large vehicle-mounted screen, and the step of providing visual information on the vehicle-mounted display device according to the second image includes: displaying the second image on the large vehicle-mounted screen.

[0096] In some embodiments, the assisted driving method of the present application also includes controlling the vehicle to automatically turn on the fog lights according to visibility.

[0097] In some embodiments, the assisted driving method of the present application also includes automatically reducing the vehicle speed based on visibility.

[0098] In some embodiments, the assisted driving method of the present application also includes providing alarms and reminders based on visibility.

[0099] According to the machine learning-based assisted driving method of the present application, the visibility of the first image acquired by the vehicle in real time can be obtained through the machine learning model, and the first image can be defogged according to the visibility to obtain a relatively clear second image, which is intuitively displayed to the driver through the on-board display device, thereby providing the driver with clear and safe driving information.

[0100] In some embodiments, the machine learning-based assisted driving method of the present application also includes: acquiring multiple sets of radar data of the vehicle while acquiring multiple images, the radar data is used to provide the obstacle distribution around the vehicle, and the multiple sets of radar data correspond to the multiple images one-to-one; determining the driver's visible range according to the obstacle distribution and visibility; the training data includes multiple sets of radar data and the visible range; the output result of the machine learning model also includes the visible range corresponding to each image; acquiring the first radar data of the vehicle while acquiring the first image of the target road section, and inputting the first image and the first radar data into the trained machine learning model, and the second output result of the trained machine learning model includes the current visible range.

[0101] The method further comprises the steps of:

[0102] Step S160: generating a third image according to the second image and the current visible range; and

[0103] Step S170: providing visual information on the vehicle-mounted display device according to the third image.

[0104] Specifically, in these embodiments, the radar data may be derived from a vehicle-mounted radar device, which is used to provide obstacle distribution information around the vehicle, including but not limited to the front, back, left, and right sides. Moreover, the radar data corresponds one-to-one to the image in step S110.

[0105] In some cases, while acquiring the image or the first image through the vehicle-mounted camera device, radar data is obtained through the vehicle-mounted radar device of the same vehicle.

[0106] The driver's visual range is determined based on the obstacle distribution and visibility. For images, visibility only considers the impact of bad weather on air visibility, but does not consider the impact of the vehicle's surroundings on the driver's visual range. In an embodiment including radar data, considering both visibility and radar data helps provide the driver with complex environmental information and assists the driver in making safety judgments.

[0107] According to the above-mentioned embodiment, the training data of the machine learning model includes multiple sets of radar data and visible ranges at the same time, and the output results of the machine learning model include the visible range of each image in addition to the visibility of each image.

[0108] By inputting the first image and first radar data of the target road section acquired by the vehicle in real time into the trained machine learning model, the visual range corresponding to the first image and the first radar data can be obtained.

[0109] In the above step S160, a third image is generated based on the second image after defogging and the visible range. It can be understood that the third image includes obstacle distribution information and has more complex environment information than the second image.

[0110] In the above step S170 , visual information is provided on the vehicle-mounted display device according to the third image obtained in step S160 .

[0111] In some embodiments, the vehicle-mounted display device includes a large vehicle-mounted screen, and the step of providing visual information on the vehicle-mounted display device according to the third image includes: displaying the third image on the large vehicle-mounted screen.

[0112] It should be noted that the present application does not limit the specific location, quantity, and display mode of the vehicle-mounted display device. In some embodiments, the vehicle-mounted display device is a liquid crystal screen.

[0113] In some embodiments, the third image includes a simulated image of the vehicle and visualized first radar data around the vehicle. According to these embodiments, the relative position relationship between the vehicle and the surrounding environment can be displayed to the driver, which helps to help the driver drive safely.

[0114] According to the assisted driving method of the above embodiment, the visibility of the first image and the driver's visual range can be obtained based on the vehicle's real-time first image and first radar information through a machine learning model, and intuitively displayed to the driver through the on-board display device, thereby providing the driver with clear and safe driving information.

[0115] Figure 3 is a block diagram of a machine learning-based driver assistance device according to an embodiment of the present application. Figure 3 As shown, the auxiliary driving device 300 includes an image acquisition unit 310, a machine learning model 320, a defogging unit 330 and an onboard display device 340. The image acquisition unit 310 is used to acquire a first image of a target road section; the machine learning model 320 is used to output the current visibility of the first image according to the input first image; the defogging unit 330 is used to perform defogging on the first image according to the current visibility to obtain a second image; and the onboard display device 340 is used to provide visual information on the onboard display device according to the second image.

[0116] Figure 3 The illustrated driving assistance device 300 can be used to execute the driving assistance method described above. Figure 1 , Figure 2 The above description can be used to illustrate the auxiliary driving device 300 of the present application.

[0117] In some embodiments, Figure 3As shown, the assisted driving device 300 also includes a radar data acquisition unit 350, which is used to acquire radar data of the vehicle while the image acquisition unit 310 acquires the first image; in this embodiment, the machine learning model 320 is also used to output the driver's current visual range based on the first image and the first radar data.

[0118] The assisted driving device 300 including the radar data acquisition unit 350 can be used to execute the assisted driving method described above including acquiring multiple sets of radar data. The relevant description content in the previous text can be used to illustrate the assisted driving device 300 and will not be expanded.

[0119] The present invention also includes an assisted driving device based on machine learning, comprising a memory and a processor. The memory is used to store instructions executable by the processor; the processor is used to execute the instructions to implement the assisted driving method based on machine learning described above.

[0120] Figure 4 1 is a system block diagram of a machine learning-based driver assistance device according to an embodiment of the present invention. Figure 4 As shown, the auxiliary driving device 400 may include an internal communication bus 401, a processor 402, a read-only memory (ROM) 403, a random access memory (RAM) 404, and a communication port 405. When applied to a personal computer, the auxiliary driving device 400 may also include a hard disk 406. The internal communication bus 401 can realize data communication between the components of the auxiliary driving device 400. The processor 402 can make judgments and issue prompts. In some embodiments, the processor 402 can be composed of one or more processors. The communication port 405 can realize data communication between the auxiliary driving device 400 and the outside. In some embodiments, the auxiliary driving device 400 can send and receive information and data from the network through the communication port 405. The auxiliary driving device 400 may also include different forms of program storage units and data storage units, such as a hard disk 406, a read-only memory (ROM) 403 and a random access memory (RAM) 404, which can store various data files used for computer processing and / or communication, and possible program instructions executed by the processor 402. The processor executes these instructions to implement the main part of the method. The processing result of the processor is transmitted to the user device through the communication port and displayed on the user interface.

[0121] The above-mentioned assisted driving method based on machine learning can be implemented as a computer program, saved in the hard disk 406, and can be loaded into the processor 402 for execution to implement the assisted driving method based on machine learning of the present application.

[0122] The present invention also includes a computer-readable medium storing a computer program code, which, when executed by a processor, implements the machine learning-based assisted driving method described above.

[0123] When the assisted driving method based on machine learning is implemented as a computer program, it can also be stored in a computer-readable storage medium as a product. For example, a computer-readable storage medium may include, but is not limited to, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic strip), an optical disk (e.g., a compact disk (CD), a digital versatile disk (DVD)), a smart card, and a flash memory device (e.g., an electrically erasable programmable read-only memory (EPROM), a card, a stick, a key drive). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain and / or carry code and / or instructions and / or data.

[0124] It should be understood that the embodiments described above are only illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or a combination thereof.

[0125] Some aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". The processor may be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. In addition, various aspects of the present application may be expressed as computer products located in one or more computer-readable media, which include computer-readable program codes. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, tapes ...), optical disks (e.g., compact disks CDs, digital versatile disks DVDs ...), smart cards, and flash memory devices (e.g., cards, sticks, key drives ...).

[0126] A computer-readable medium may include a propagated data signal containing computer program code, such as in baseband or as part of a carrier wave. The propagated signal may have a variety of manifestations, including electromagnetic, optical, etc., or a suitable combination. A computer-readable medium may be any computer-readable medium other than a computer-readable storage medium, which may be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit a program for use. The program code on the computer-readable medium may be propagated via any suitable medium, including radio, cable, fiber optic cable, radio frequency signal, or similar medium, or any combination of the above mediums.

[0127] The basic concepts have been described above. Obviously, for those skilled in the art, the above invention disclosure is only used as an example and does not constitute a limitation of the present application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements and amendments to the present application. Such modifications, improvements and amendments are suggested in the present application, so such modifications, improvements and amendments still belong to the spirit and scope of the exemplary embodiments of the present application.

[0128] At the same time, the present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0129] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of the present application are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

Claims

1. An assisted driving method based on machine learning, comprising: Acquire training data, the training data comprising a plurality of images of a target road section under different weather conditions, each of the images having a visibility; Constructing a machine learning model, inputting the training data into the machine learning model, wherein the output result of the machine learning model includes the visibility of each of the images; Acquire a first image of the target road section, input the first image into the trained machine learning model, wherein a first output result of the trained machine learning model includes current visibility of the first image; Performing defogging processing on the first image according to the current visibility to obtain a second image; The method comprises the steps of: obtaining a clear image of the target road section on a sunny day; Grouping each of the images in the training data according to the visibility, wherein the visibility of the images in the same visibility group is within a corresponding preset range, and each visibility group includes at least two images; calculating the similarity between the multiple images in each visibility group and the clear image, respectively, and obtaining a target image with the maximum similarity in each group; extracting similar feature parameters according to the target image and the clear image, and each visibility group has one similar feature parameter; Process all images in the visibility group according to the similar feature parameters to obtain the second image; as well as Visual information is provided on an in-vehicle display device according to the second image.

2. The assisted driving method according to claim 1, characterized in that: Also includes: Acquiring multiple sets of radar data of the vehicle while acquiring the multiple images, the radar data being used to provide obstacle distribution around the vehicle, the multiple sets of radar data corresponding to the multiple images one-to-one; determining the driver's visual range according to the obstacle distribution and the visibility; the training data including the multiple sets of radar data and the visual range; the output result of the machine learning model also including the visual range corresponding to each of the images; Acquire a first image of the target road section and first radar data of the vehicle at the same time, input the first image and the first radar data into the trained machine learning model, wherein a second output result of the trained machine learning model includes a current visible range; generating a third image according to the second image and the current visible range; as well as Visual information is provided on the vehicle-mounted display device according to the third image.

3. The assisted driving method according to claim 1, characterized in that: The sources of the multiple images include: one or more of a vehicle-mounted camera, a road-mounted camera, and a mobile terminal.

4. The assisted driving method according to claim 1, characterized in that: The vehicle-mounted display device includes a large vehicle-mounted screen, and the step of providing visual information on the vehicle-mounted display device according to the second image includes: displaying the second image on the large vehicle-mounted screen.

5. The assisted driving method according to claim 2, characterized in that: The vehicle-mounted display device includes a vehicle-mounted large screen, and the step of providing visual information on the vehicle-mounted display device according to the third image includes: displaying the third image on the vehicle-mounted large screen.

6. The assisted driving method according to claim 5, characterized in that: The third image includes a simulated image of the vehicle and first radar data of a visualization of the surroundings of the vehicle.

7. The assisted driving method according to claim 1, characterized in that: The different weather conditions include at least one of sunny days, rainy days, snowy days, and foggy days.

8. A driving assistance device based on machine learning, characterized in that: include: Image acquisition unit, machine learning model, defogging processing unit and vehicle display device, The image acquisition unit is used to acquire a first image of the target road section; The machine learning model is used to output the current visibility of the first image according to the input first image; The defogging processing unit is used to perform defogging processing on the first image according to the current visibility to obtain a second image; The method comprises the steps of: obtaining a clear image of the target road section on a sunny day; Grouping each image in the training data according to the visibility, wherein the visibility of the images in the same visibility group is within a corresponding preset range, and each visibility group includes at least two images; Calculating the similarities between the plurality of images in each visibility group and the clear image respectively, and obtaining a target image with the maximum similarity in each group; extracting similar feature parameters according to the target image and the clear image, each visibility group having one similar feature parameter; Process all images in the visibility group according to the similar feature parameters to obtain the second image; The in-vehicle display device is used to provide visual information on the in-vehicle display device according to the second image.

9. The driving assistance device according to claim 8, characterized in that: Also includes: A radar data acquisition unit, configured to acquire first radar data of the vehicle while the image acquisition unit acquires the first image; The machine learning model is also used to output the driver's current visual range based on the first image and the first radar data.

10. A driving assistance device based on machine learning, comprising: a memory for storing instructions executable by a processor; A processor, configured to execute the instructions to implement a method as described in any one of 1-7.

11. A computer-readable medium storing a computer program code, wherein the computer program code implements the method according to any one of 1 to 7 when executed by a processor.

Images