An image-based road environment detection method and device

Abstract

The present invention discloses an image-based road environment detection method and device. By acquiring the real-time video stream image of a camera, after preprocessing the camera image, a light intensity detection model and a weather type detection model are established to obtain the light intensity level and weather type value corresponding to the camera image. At the same time, a corresponding perception processing neural network model is established based on different light intensity levels and different weather type values. When the light intensity level and weather type value corresponding to the camera image are detected, it is directly switched to the corresponding perception processing neural network model to output the real-time road environment information under the light intensity level and weather type value. Compared with the prior art, the present invention makes the acquired environmental information richer by adding the detection of light intensity and weather type, reduces the influence of environmental factors on the vehicle perception ability, and improves the robustness and accuracy of the vehicle perception ability.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method and device for road environment detection based on images. Background Art

[0002] Currently, in the ADAS system, cameras can be used to implement various functions and gradually evolve according to the development law of autonomous driving. ADAS vehicle-mounted cameras have some advantages over other perception sensors. Because the camera resolution is higher than that of other sensors, it can obtain sufficient environmental details to help the vehicle recognize the environment; the vehicle-mounted camera can depict the appearance and shape of objects, read signs, etc., which other sensors cannot do. In addition, the automotive industry is price-sensitive, and the hardware cost of cameras is relatively low. From the perspective of cost reduction, cameras are the preferred choice for environmental perception sensors, and cameras are the best choice in the case of clear vision.

[0003] However, the disadvantage of cameras as environmental perception sensors is that they are greatly affected by environmental factors, thus affecting the accuracy of environmental perception functions such as vehicle recognition. These environmental influencing factors include: being relatively sensitive to the lighting environment. For example, in an actual scenario, if shooting against the light, the target object will appear dim; if driving in a tunnel, there may be a lack of light. It also includes: being relatively sensitive to meteorological factors. For example, if encountering rainy or foggy weather, the line of sight will be reduced and become blurred. Therefore, how to reduce the influence of environmental factors on environmental perception sensors is an urgent problem to be solved currently. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to provide a method and device for road environment detection based on images, by increasing the detection of light intensity and meteorological types, making the obtained environmental information richer, reducing the influence of environmental factors on the vehicle perception ability, and improving the robustness and accuracy of the vehicle perception ability.

[0005] To solve the above technical problem, the present invention provides a method for road environment detection based on images, including:

[0006] Obtain the real-time video stream of the camera, decode the real-time video stream to obtain a camera image;

[0007] Establish a light intensity detection model, input the camera image after the first preprocessing into the light intensity detection model, extract and output the light intensity level of the camera image according to the light intensity characteristic value of the camera image;

[0008] Build a weather type detection model, input the camera image after the second preprocessing into the weather type detection model, extract and output the weather type value of the camera image according to the weather type feature value of the camera image;

[0009] Build a perception processing neural network model corresponding to different light intensity levels and different weather type values, switch to the corresponding perception processing neural network model according to the light intensity level and the weather type value, and input the camera image after the third preprocessing into the perception processing neural network model to obtain real-time road environment information.

[0010] Further, obtain the real-time video stream of the camera, decode the real-time video stream to obtain the camera image, specifically:

[0011] Set the video stream address of the camera through the configuration file to obtain the real-time video stream of the camera;

[0012] Decode the real-time video stream to obtain the real-time camera image, and save the information of the camera image to the database; wherein, the information of the camera image includes the camera image, the time stamp corresponding to the camera image, and the identifier of the camera.

[0013] Further, build a light intensity detection model, specifically:

[0014] Select the first photos taken at different times and in different environments, mark the illuminance of the first photos with an illuminometer, and establish a first mapping table between the first photos and the illuminance according to the illuminance of each first photo;

[0015] Combine the first mapping table and the light intensity division levels to establish a second mapping table between the first photos and the light intensity levels;

[0016] Divide all the first photos according to a preset ratio to establish a light sample data set;

[0017] Train and verify the first preset neural network model through the light sample data set to generate a light intensity detection model.

[0018] Further, build a weather type detection model, specifically:

[0019] Select the second photos taken under different weather conditions, mark the weather type of the second photos, and establish a third mapping table between the second photos and the weather type according to the weather type of each second photo;

[0020] Divide all the second photos according to a preset ratio to establish a weather sample data set;

[0021] Train and validate the second preset neural network model with the meteorological sample data set to generate a meteorological type detection model.

[0022] Further, the first preprocessing includes scaling the camera image to the input size of the light intensity detection model, normalizing pixel values, and converting to a grayscale image;

[0023] The second preprocessing includes scaling the camera image to the input size of the meteorological type detection model, normalizing pixel values, and converting to a grayscale image;

[0024] The third preprocessing includes scaling the camera image to the input size of the perception processing neural network model, normalizing pixel values, and converting to a grayscale image.

[0025] Further, the present invention also provides an image-based road environment detection device, including: an image acquisition module, a light intensity detection module, a meteorological type detection module, and a road information acquisition module;

[0026] Among them, the image acquisition module is used to acquire the real-time video stream of the camera, decode the real-time video stream to obtain a camera image;

[0027] The light intensity detection module is used to establish a light intensity detection model, input the camera image after the first preprocessing into the light intensity detection model, extract and output the light intensity level of the camera image according to the light intensity characteristic value of the camera image;

[0028] The meteorological type detection module is used to establish a meteorological type detection model, input the camera image after the second preprocessing into the meteorological type detection model, extract and output the meteorological type value of the camera image according to the meteorological type characteristic value of the camera image;

[0029] The road information acquisition module is used to establish a perception processing neural network model corresponding to different light intensity levels and different meteorological type values, switch to the corresponding perception processing neural network model according to the light intensity level and the meteorological type value, and input the camera image after the third preprocessing into the perception processing neural network model to obtain real-time road environment information.

[0030] Further, the image acquisition module is used to acquire the real-time video stream of the camera, decode the real-time video stream to obtain a camera image, specifically:

[0031] Set the video stream address of the camera through a configuration file to acquire the real-time video stream of the camera;

[0032] Decode the real-time video stream to obtain a real-time camera image, and save the information of the camera image into a database; wherein, the information of the camera image includes the camera image, the time stamp corresponding to the camera image, and the identifier of the camera.

[0033] Further, the light intensity detection module is used to establish a light intensity detection model, specifically:

[0034] Select the first photos taken at different times and in different environments, mark the illuminance of the first photos with an illuminometer, and establish a first mapping table between the first photos and the illuminance according to the illuminance of each first photo;

[0035] Combine the first mapping table and the light intensity grading levels to establish a second mapping table between the first photos and the light intensity levels;

[0036] Divide all the first photos according to a preset ratio to establish a light sample data set;

[0037] Train and verify the first preset neural network model through the light sample data set to generate a light intensity detection model.

[0038] Further, the weather type detection module is used to establish a weather type detection model, specifically:

[0039] Select the second photos taken under different weather conditions, mark the weather types of the second photos, and establish a third mapping table between the second photos and the weather types according to the weather types of each second photo;

[0040] Divide all the second photos according to a preset ratio to establish a weather sample data set;

[0041] Train and verify the second preset neural network model through the weather sample data set to generate a weather type detection model.

[0042] Further, the light intensity detection module includes a first preprocessing unit, the weather type detection module includes a second preprocessing unit, and the road information acquisition module includes a third preprocessing unit;

[0043] The first preprocessing unit is used to scale the camera image to the input size of the light intensity detection model, normalize the pixel values, and convert it into a grayscale image;

[0044] The second preprocessing unit is used to scale the camera image to the input size of the weather type detection model, normalize the pixel values, and convert it into a grayscale image;

[0045] The third preprocessing unit is used to scale the camera image to the input size of the perception processing neural network model, normalize the pixel values, and convert the grayscale image.

[0046] Compared with the prior art, the method and device for road environment detection based on images according to the embodiments of the present invention have the following beneficial effects:

[0047] By acquiring the real-time video stream image of the camera, preprocessing the camera image, establishing a light intensity detection model and a weather type detection model, respectively detecting the light intensity and the weather type of the camera image to obtain the light intensity level and the weather type value corresponding to the camera image; at the same time, establishing a corresponding perception processing neural network model based on different light intensity levels and different weather type values. When the light intensity level and the weather type value corresponding to the camera image are detected, directly switch to the corresponding perception processing neural network model and output the real-time road environment information under the light intensity level and the weather type value. Compared with the prior art, the present invention makes the acquired environmental information richer by adding the detection of light intensity and weather type, reduces the influence of environmental factors on the vehicle perception ability, and improves the robustness and accuracy of the vehicle perception ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a schematic flowchart of an embodiment of a method for road environment detection based on images provided by the present invention;

[0049] Figure 2 is a schematic structural diagram of an embodiment of a device for road environment detection based on images provided by the present invention;

[0050] Figure 3 is a schematic diagram of the light intensity level division of an embodiment of a method and device for road environment detection based on images provided by the present invention;

[0051] Figure 4 is a schematic diagram of the weather type category division of an embodiment of a method and device for road environment detection based on images provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0053] Embodiment 1

[0054] SeeFigure 1 , Figure 1 is a schematic flowchart of an embodiment of a road environment detection method based on images provided by the present invention. As shown in Figure 1 , this method includes steps 101 - 104, which are as follows:

[0055] Step 101: Obtain the real-time video stream of the camera, decode the real-time video stream to obtain the camera image.

[0056] In this embodiment, the video stream address of the ADAS camera is set through a configuration file, and the video stream of the in-vehicle ADAS camera is pulled to obtain the real-time video stream of the camera; the real-time video stream is decoded to obtain the real-time camera image, and the information of the camera image is saved to a cache list or a database; wherein, the information of the camera image includes the camera image, the time stamp corresponding to the camera image, and the identifier of the camera.

[0057] Step 102: Establish a light intensity detection model, input the camera image after the first preprocessing into the light intensity detection model, extract and output the light intensity level of the camera image according to the light intensity feature value of the camera image.

[0058] In this embodiment, first photos are taken at different times and in different environments, the illuminance of the first photos is marked by an illuminometer, and according to the illuminance of each first photo, a first mapping table between the first photo and the illuminance is established; combining the first mapping table and the light intensity division levels, a second mapping table between the first photo and the light intensity level is established. As an example in this embodiment, the light intensity is divided into seven light intensity levels according to a specific range, as shown in Figure 3 .

[0059] In this embodiment, all the first photos are divided according to a preset ratio to establish a light sample data set; as an example in this embodiment, all the first photos are divided into three data sets, namely a training set, a test set, and a validation set, according to the ratio of 8:1:1 to generate a light sample data set.

[0060] In this embodiment, by designing a first preset neural network model, the first preset neural network model is trained according to the light sample data set, and the first preset neural network model is adjusted. The prediction effect of the first preset neural network model is compared through the validation set to determine the final first preset neural network model as the light intensity detection model.

[0061] In this embodiment, a first preprocessing is performed on the camera image. The first preprocessing includes scaling the camera image to the input size of the light intensity detection model, normalizing the pixel values of the camera image, and converting the camera image into a grayscale image. The camera image after the first preprocessing is input into the light intensity detection model, the light intensity eigenvalue of the camera image is extracted and based on it, after feature standardization of the light intensity eigenvalue, a light intensity level is mapped and the light intensity level of the camera image is output; wherein, the feature standardization includes, but is not limited to, matrix normalization of the light intensity eigenvalue.

[0062] Step 103: Establish a weather type detection model, input the camera image after the second preprocessing into the weather type detection model, extract the weather type eigenvalue of the camera image and based on it, output the weather type value of the camera image.

[0063] In this embodiment, second photos are taken under different weather conditions, the weather types are labeled for the second photos, and based on the weather type of each second photo, a third mapping table between the second photos and the weather types is established. As an example in this embodiment, the weather types are divided into ten types: sunny, cloudy, overcast, light rain, heavy rain, fog, haze, blowing sand, light snow, and heavy snow, and the ten weather types correspond to ten weather type categories, as Figure 4 shown.

[0064] In this embodiment, all the second photos are divided according to a preset ratio to establish a weather sample data set; as an example in this embodiment, all the second photos are divided into three data sets: a training set, a test set, and a validation set according to a ratio of 8:1:1 to generate a weather sample data set.

[0065] In this embodiment, by designing a second preset neural network model, the second preset neural network model is trained according to the weather sample data set, and the second preset neural network model is adjusted. The prediction effect of the second preset neural network model is compared through the validation set to determine the final second preset neural network model as the weather intensity detection model.

[0066] In this embodiment, a second preprocessing is performed on the camera image. The second preprocessing includes scaling the camera image to the input size of the weather type detection model, normalizing the pixel values of the camera image, and converting the camera image into a grayscale image. The camera image after the second preprocessing is input into the weather type detection model, the weather type eigenvalue of the camera image is extracted and based on it, after feature standardization of the weather type eigenvalue, a weather type value is mapped and the weather type value of the camera image is output; wherein, the feature standardization includes, but is not limited to, matrix normalization of the weather type eigenvalue.

[0067] Step 104: Establish a perception processing neural network model corresponding to different light intensity levels and different meteorological type values. Switch to the corresponding perception processing neural network model according to the light intensity level and the meteorological type value, and input the camera image after the third preprocessing into the perception processing neural network model to obtain real-time road environment information.

[0068] In this embodiment, perception processing neural network models are trained respectively for different light intensity levels and different meteorological types, so as to establish a perception processing neural network model corresponding to different light intensity levels and different meteorological type values. Among them, the perception processing neural network model is used for vehicle recognition, pedestrian recognition, obstacle recognition, road marking detection and distance detection.

[0069] In this embodiment, when establishing the light intensity detection model and the meteorological type detection model, by obtaining pictures and annotating training samples in environments with different light intensities and different meteorological types, the adaptability of the algorithm to image data in different scenarios is enhanced. At the same time, by training the perception processing neural network model for different light intensities and different meteorological types, the problem that the perception processing neural network model trained relatively singly has poor generalization ability in different light and different meteorological environments can be solved, the inference accuracy of the perception processing neural network model can be improved, and at the same time, the environmental information obtained by the perception processing neural network model can be made more abundant.

[0070] In this embodiment, the perception processing neural network model can switch to the corresponding perception processing neural network model according to the light intensity level and the meteorological type value obtained in Steps 102 and 103, making the subsequent perception processing more accurate. In this embodiment, the camera image is subjected to a third preprocessing, where the third preprocessing includes scaling the camera image to the input size of the perception processing neural network model, normalizing the pixel values of the camera image, and converting the camera image into a grayscale image. The camera image after the third preprocessing is input into the perception processing neural network model to obtain real-time road environment information, where the real-time road environment information includes vehicle recognition results, pedestrian recognition results, obstacle recognition results, road marking detection results and distance detection results.

[0071] In this embodiment, after obtaining the real-time road environment information, the real-time road environment information is also uploaded to relevant warning units or control units, so that motor vehicles can make timely processing, which is beneficial to optimizing the performance of visual perception functions such as forward collision warning and lane departure warning, and improving the robustness and driving safety of the ADAS perception function.

[0072] In this embodiment, taking the camera image as the input does not involve the redesign of the hardware of the perception processing system, and can effectively reduce the implementation cost and development cycle.

[0073] See Figure 2 , Figure 2 , which is a schematic structural diagram of an embodiment of a road environment detection device based on images provided by the present invention. As shown in Figure 2 , the device includes: an image acquisition module 201, a light intensity detection module 202, a weather type detection module 203, and a road information acquisition module 204, specifically as follows:

[0074] The image acquisition module 201 is used to acquire the real-time video stream of the camera, decode the real-time video stream, and obtain the camera image.

[0075] In this embodiment, the video stream address of the ADAS camera is set through a configuration file, and the video stream of the in-vehicle ADAS camera is pulled to acquire the real-time video stream of the camera; the real-time video stream is decoded to obtain the real-time camera image, and the information of the camera image is saved to the cache list or database; wherein, the information of the camera image includes the camera image, the time stamp corresponding to the camera image, and the identifier of the camera.

[0076] The light intensity detection module 202 is used to establish a light intensity detection model, input the camera image after the first preprocessing into the light intensity detection model, extract and output the light intensity level of the camera image according to the light intensity feature value of the camera image.

[0077] In this embodiment, the first photos are taken at different times and in different environments, the illuminance of the first photos is marked by an illuminometer, and according to the illuminance of each first photo, a first mapping table between the first photos and the illuminance is established; combining the first mapping table and the light intensity division levels, a second mapping table between the first photos and the light intensity levels is established. As an example in this embodiment, the light intensity is divided into seven light intensity levels according to a specific range, as shown in Figure 3 .

[0078] In this embodiment, all the first photos are divided according to a preset ratio to establish a light sample data set; as an example in this embodiment, all the first photos are divided into three data sets, namely a training set, a test set, and a validation set, according to a ratio of 8:1:1 to generate a light sample data set.

[0079] In this embodiment, by designing a first preset neural network model, training the first preset neural network model according to the light sample data set, and adjusting the first preset neural network model, comparing the prediction effect of the first preset neural network model through the validation set to determine the final first preset neural network model as the light intensity detection model.

[0080] In this embodiment, a first preprocessing is performed on the camera image. The first preprocessing includes scaling the camera image to the input size of the light intensity detection model, normalizing the pixel values of the camera image, and converting the camera image into a grayscale image. The camera image after the first preprocessing is input into the light intensity detection model, the light intensity eigenvalue of the camera image is extracted and based on it, after feature standardization of the light intensity eigenvalue, a light intensity level is mapped and the light intensity level of the camera image is output. Among them, the feature standardization includes but is not limited to matrix normalization of the light intensity eigenvalue.

[0081] The meteorological type detection module 203 is used to establish a meteorological type detection model, input the camera image after the second preprocessing into the meteorological type detection model, extract and based on the meteorological type eigenvalue of the camera image, output the meteorological type value of the camera image.

[0082] In this embodiment, second photos are taken under different meteorological conditions, the meteorological types are labeled for the second photos, and based on the meteorological type of each second photo, a third mapping table between the second photos and the meteorological types is established. As an example in this embodiment, the meteorological types are divided into ten types: sunny, cloudy, overcast, light rain, heavy rain, fog, haze, blowing sand, light snow, and heavy snow, and the ten meteorological types correspond to ten meteorological type categories, such as Figure 4 shown.

[0083] In this embodiment, all the second photos are divided according to a preset ratio to establish a meteorological sample data set; as an example in this embodiment, all the second photos are divided into three data sets: a training set, a test set, and a validation set according to a ratio of 8:1:1 to generate a meteorological sample data set.

[0084] In this embodiment, by designing a second preset neural network model, the second preset neural network model is trained according to the meteorological sample data set, and the second preset neural network model is adjusted. The prediction effect of the neural network model is compared through the validation set to determine the final second preset neural network model as the meteorological intensity detection model.

[0085] In this embodiment, a second preprocessing is performed on the camera image. The second preprocessing includes scaling the camera image to the input size of the meteorological type detection model, normalizing the pixel values of the camera image, and converting the camera image into a grayscale image. The camera image after the second preprocessing is input into the meteorological type detection model, the meteorological type eigenvalue of the camera image is extracted and based on it, after feature standardization of the meteorological type eigenvalue, a meteorological type value is mapped and the meteorological type value of the camera image is output; among them, the feature standardization includes but is not limited to matrix normalization of the meteorological type eigenvalue.

[0086] The road information acquisition module 204 is used to establish a perception processing neural network model corresponding to different light intensity levels and different meteorological type values, switch to the corresponding perception processing neural network model according to the light intensity level and the meteorological type value, and input the camera image after the third preprocessing into the perception processing neural network model to obtain real-time road environment information.

[0087] In this embodiment, the perception processing neural network model is trained for different light intensity levels and different meteorological types respectively, so as to establish a perception processing neural network model corresponding to different light intensity levels and different meteorological type values. Among them, the perception processing neural network model is used for vehicle recognition, pedestrian recognition, obstacle recognition, road marking detection and distance detection.

[0088] In this embodiment, when establishing the light intensity detection model and the meteorological type detection model, by obtaining pictures and annotating training samples in environments with different light intensities and different meteorological types, the adaptability of the algorithm to image data in different scenarios is enhanced. At the same time, by training the perception processing neural network model for different lights and different meteorological types, the problem that the perception processing neural network model trained relatively singly has poor generalization ability in different light and different meteorological environments can be solved, the inference accuracy of the perception processing neural network model can be improved, and at the same time, the environmental information obtained by the perception processing neural network model can be made more abundant.

[0089] In this embodiment, the perception processing neural network model can switch to the corresponding perception processing neural network model according to the light intensity level and the meteorological type value obtained from the image acquisition module 201 and the light intensity detection module 202, making the subsequent perception processing more accurate. In this embodiment, the camera image is subjected to the third preprocessing. Among them, the third preprocessing includes scaling the camera image to the input size of the perception processing neural network model, normalizing the pixel values of the camera image, and converting the camera image into a grayscale image. The camera image after the third preprocessing is input into the perception processing neural network model to obtain real-time road environment information. Among them, the real-time road environment information includes vehicle recognition results, pedestrian recognition results, obstacle recognition results, road marking detection results and distance detection results.

[0090] In this embodiment, after obtaining the real-time road environment information, the real-time road environment information is also uploaded to the relevant warning unit or control unit, so that the motor vehicle can make timely processing, which is beneficial to optimizing the performance of visual perception functions such as forward collision warning and lane departure warning, and improving the robustness and driving safety of the ADAS perception function.

[0091] In this embodiment, the camera image is used as the input, without involving the re-design of the hardware of the perception processing system, which can effectively reduce the implementation cost and development cycle.

[0092] In summary, for a method and device for road environment detection based on images according to the present invention, a real-time video stream of a camera is acquired, the real-time video stream is decoded to obtain a camera image; a light intensity detection model is established, the camera image after the first preprocessing is input into the light intensity detection model, the light intensity characteristic value of the camera image is extracted and based on it, the light intensity level of the camera image is output; a weather type detection model is established, the camera image after the second preprocessing is input into the weather type detection model, the weather type characteristic value of the camera image is extracted and based on it, the weather type value of the camera image is output; a perception processing neural network model corresponding to different light intensity levels and different weather type values is established, switched to the corresponding perception processing neural network model according to the light intensity level and the weather type value, and the camera image after the third preprocessing is input into the perception processing neural network model to obtain real-time road environment information. Compared with the prior art, by adding the detection of light intensity and weather type, the present invention makes the acquired environmental information richer, reduces the influence of environmental factors on the vehicle perception ability, and improves the robustness and accuracy of the vehicle perception ability.

[0093] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and replacements can be made, and these improvements and replacements should also be regarded as the protection scope of the present invention.

Claims

1. An image-based road environment detection method, characterized in that, it includes: Obtain the real-time video stream of the camera, decode the real-time video stream to obtain a camera image; Establish a light intensity detection model, input the camera image after the first preprocessing into the light intensity detection model, extract and based on the light intensity feature value of the camera image, output the light intensity level of the camera image; Among them, the light intensity detection model is based on selecting the first photos taken at different times and different environments, annotating the illuminance of the first photos with an illuminometer, and based on the illuminance of each first photo, establishing a first mapping table between the first photo and the illuminance; Combining the first mapping table and the light intensity division levels, establish a second mapping table between the first photo and the light intensity level; Divide all the first photos according to a preset ratio to establish a light sample data set; Obtained by training and validating the first preset neural network model through the light sample data set; Establish a weather type detection model, input the camera image after the second preprocessing into the weather type detection model, extract and based on the weather type feature value of the camera image, output the weather type value of the camera image; Among them, the weather type detection model is based on selecting the second photos taken under different weather conditions, annotating the weather type of the second photos, and based on the weather type of each second photo, establishing a third mapping table between the second photo and the weather type; Divide all the second photos according to a preset ratio to establish a weather sample data set; Obtained by training and validating the second preset neural network model through the weather sample data set; Establish a perception processing neural network model corresponding to different light intensity levels and different weather type values, switch to the corresponding perception processing neural network model according to the light intensity level and the weather type value, and input the camera image after the third preprocessing into the perception processing neural network model to obtain real-time road environment information.

2. The image-based road environment detection method according to claim 1, characterized in that, Obtain the real-time video stream of the camera, decode the real-time video stream to obtain a camera image, specifically: Set the video stream address of the camera through a configuration file to obtain the real-time video stream of the camera; Decode the real-time video stream to obtain a real-time camera image, and save the information of the camera image into a database; among them, the information of the camera image includes the camera image, the time stamp corresponding to the camera image, and the identifier of the camera.

3. The image-based road environment detection method according to claim 1, characterized in that, The first preprocessing includes scaling the camera image to the input size of the light intensity detection model, normalizing the pixel values, and converting to a grayscale image; The second preprocessing includes scaling the camera image to the input size of the weather type detection model, normalizing the pixel values, and converting to a grayscale image; The third preprocessing includes scaling the camera image to the input size of the perception processing neural network model, normalizing pixel values, and converting the grayscale image.

4. An image-based road environment detection device, characterized in that, it includes: an image acquisition module, a light intensity detection module, a weather type detection module, and a road information acquisition module; wherein, the image acquisition module is used to acquire the real-time video stream of the camera, decode the real-time video stream, and obtain the camera image; the light intensity detection module is used to establish a light intensity detection model, input the camera image after the first preprocessing into the light intensity detection model, extract and output the light intensity level of the camera image according to the light intensity characteristic value of the camera image; wherein, the light intensity detection model is established by selecting the first photos taken at different times and in different environments, annotating the illuminance of the first photos with an illuminometer, and establishing a first mapping table between the first photos and the illuminance according to the illuminance of each first photo; combining the first mapping table and the light intensity division levels to establish a second mapping table between the first photos and the light intensity levels; dividing all the first photos according to a preset ratio to establish a light sample data set; obtained by training and validating a first preset neural network model through the light sample data set; the weather type detection module is used to establish a weather type detection model, input the camera image after the second preprocessing into the weather type detection model, extract and output the weather type value of the camera image according to the weather type characteristic value of the camera image; wherein, the weather type detection model is established by selecting the second photos taken under different weather conditions, annotating the weather types of the second photos, and establishing a third mapping table between the second photos and the weather types according to the weather types of each second photo; dividing all the second photos according to a preset ratio to establish a weather sample data set; obtained by training and validating a second preset neural network model through the weather sample data set; the road information acquisition module is used to establish a perception processing neural network model corresponding to different light intensity levels and different weather type values, switch to the corresponding perception processing neural network model according to the light intensity level and the weather type value, and input the camera image after the third preprocessing into the perception processing neural network model to obtain real-time road environment information.

5. The image-based road environment detection device according to claim 4, characterized in that, the image acquisition module is used to acquire the real-time video stream of the camera, decode the real-time video stream, and obtain the camera image, specifically: setting the video stream address of the camera through a configuration file to acquire the real-time video stream of the camera; Decode the real-time video stream to obtain a real-time camera image, and save the information of the camera image to a database; wherein, the information of the camera image includes the camera image, the timestamp corresponding to the camera image, and the identifier of the camera.

6. An image-based road environment detection device according to claim 4, characterized in that the light intensity detection module includes a first preprocessing unit, the weather type detection module includes a second preprocessing unit, and the road information acquisition module includes a third preprocessing unit; the first preprocessing unit is configured to scale the camera image to the input size of the light intensity detection model, normalize the pixel values, and convert the grayscale image; the second preprocessing unit is configured to scale the camera image to the input size of the weather type detection model, normalize the pixel values, and convert the grayscale image; the third preprocessing unit is configured to scale the camera image to the input size of the perception processing neural network model, normalize the pixel values, and convert the grayscale image.

Images