Driving environment detection system and method and electronic equipment

Abstract

Embodiments of the invention provide a driving environment detection system and method and an electronic device, the method can obtain a long exposure image and a short exposure image corresponding to a driving environment, and the short exposure image is used for supplementing information of a highlight area in the long exposure image. Detecting a target classification according to the long exposure image and the short exposure image, if the target classification is a first classification, executing first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection; and if the target classification is a second classification, performing second obstacle detection according to the long exposure image and the short exposure image, and outputting a second result of the second obstacle detection. Wherein the illumination intensity of the driving environment represented by the first classification is higher than that of the driving environment represented by the second classification. According to the detection method, the long exposure image and the short exposure image are fused to carry out obstacle detection in a scene with relatively low illumination intensity, so that the accuracy of obstacle detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a driving environment detection system, method and electronic equipment. Background Art

[0002] Environmental perception technology is a core component of the autonomous driving system. It simulates the sensory ability of the driver and aims to obtain comprehensive information about the vehicle's surroundings in real time and accurately. Environmental perception technology uses a variety of algorithms and sensors, such as cameras, lidar, millimeter-wave radar, ultrasonic sensors, and inertial navigation systems, to identify and analyze multi-dimensional information such as road conditions, traffic participants, traffic signs, weather conditions, and road infrastructure.

[0003] Environmental perception technology can first collect real-time images of the vehicle's surroundings through visual sensors such as on-board cameras, and then pre-process the images to improve quality and extract key feature information. Then, machine learning such as deep learning algorithms are used to identify and classify the processed images, accurately identifying obstacles such as road signs, lane lines, pedestrians, and vehicles to ensure that the vehicle can move forward steadily in complex and changing environments. At the same time, for driving environments with poor lighting conditions at night, the vehicle's headlights and low beams can also be used to actively fill in the road ahead to enhance the image acquisition effect of the visual sensor.

[0004] However, in low-light driving environments such as at night, visual sensors may not be able to capture enough details, resulting in the loss of the outline and geometric information of obstacles; and, due to the interference of high beam light, the images captured by the image collector may be overexposed, making it difficult to accurately identify and locate obstacles, resulting in a decrease in the accuracy of obstacle detection during environmental perception, affecting the user experience. Summary of the invention

[0005] The present application provides a driving environment detection system, method and electronic equipment to solve the problem of low obstacle detection accuracy in environmental perception.

[0006] In a first aspect, some embodiments of the present application provide a driving environment detection system, comprising an image acquisition module and a control module. The image acquisition module is configured to acquire images of the driving environment, wherein the images include long-exposure images and short-exposure images, wherein the exposure time of the long-exposure image is greater than the exposure time of the short-exposure image, and the short-exposure image is used to supplement the information of the highlight area in the long-exposure image; and the control module is configured to perform the following steps:

[0007] Acquire a long exposure image and a short exposure image acquired by the image acquisition module, wherein an acquisition frequency of the long exposure image is higher than an acquisition frequency of the short exposure image;

[0008] Detecting target classification according to the long exposure image and the short exposure image, the target classification is used to characterize the light intensity of the driving environment, the target classification includes a first classification and a second classification, the light intensity characterized by the first classification is higher than the light intensity characterized by the second classification;

[0009] If the target is classified as the first classification, performing a first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection;

[0010] If the target is classified as the second classification, a second obstacle detection is performed based on the long exposure image and the short exposure image, and a second result of the second obstacle detection is output.

[0011] In some embodiments of the present application, the detection system also includes a light source module, which is turned on according to time information, position information or the long exposure image; the control module executes acquisition of the long exposure image and the short exposure image acquired by the image acquisition module, and is specifically configured to: control the image acquisition module to acquire the long exposure image in real time; monitor the startup state of the light source module, and when the light source module is in the turned-on state, record the turn-on time of the light source module; when the turn-on time of the light source module is greater than or equal to a preset time threshold, control the image acquisition module to acquire the short exposure image; and read the long exposure image and the short exposure image.

[0012] In some embodiments of the present application, the control module executes acquisition of the long exposure image and the short exposure image captured by the image acquisition module, and is specifically configured to: control the image acquisition module to capture the long exposure image in real time; read the time information and the position information; if the time information is in a preset time period, or the position information is in a preset position, control the image acquisition module to capture the short exposure image; read the long exposure image and the short exposure image.

[0013] In some embodiments of the present application, the control module performs target classification detection based on the long exposure image and the short exposure image, and is specifically configured to: calculate a first grayscale histogram of the long exposure image and a second grayscale histogram of the short exposure image; detect a first pixel point in the first grayscale histogram whose grayscale value is less than a first grayscale threshold, and detect a second pixel point in the second grayscale histogram whose grayscale value is greater than a second grayscale threshold; if the proportion of the first pixel point in the first grayscale histogram is greater than a first proportion threshold, and the proportion of the second pixel point in the second grayscale histogram is greater than a second proportion threshold, mark the target classification as the second classification; if the proportion of the first pixel point is less than or equal to the first proportion threshold, or the proportion of the second pixel point is less than or equal to the second proportion threshold, mark the target classification as the first classification.

[0014] In some embodiments of the present application, the control module performs target classification based on the long exposure image and the short exposure image, and is specifically configured to: input the long exposure image and the short exposure image into a classification network, the classification network includes a dual input layer and a classification decision layer, the dual input layer is used to input the long exposure image and the short exposure image, and the classification decision layer is used to generate classification labels based on the features of the long exposure image and the short exposure image; obtain the classification label output by the classification network; and mark the target classification according to the classification label.

[0015] In some embodiments of the present application, the control module performs the second obstacle detection according to the long exposure image and the short exposure image, and is specifically configured as follows:

[0016] Encoding the long exposure image and the short exposure image to obtain a first feature of the long exposure image and a second feature of the short exposure image; fusing the first feature and the second feature to generate a fused feature; inputting the fused feature into an obstacle perception network, wherein the obstacle perception network is a perspective view-based network or a bird's-eye view-based network, and the obstacle perception network is used to identify obstacle information in the image, wherein the obstacle information includes a position, scale, and category of the obstacle; and obtaining the obstacle information output by the obstacle perception network to obtain the second result.

[0017] In some embodiments of the present application, after the control module executes the step of performing a second obstacle detection based on the long exposure image and the short exposure image, the control module is further configured to: track the obstacle across frames based on the second result to obtain motion information of the obstacle; the second result includes the position, scale and category of the obstacle, and the motion information includes at least one of the motion state, motion direction and acceleration; perform filtering processing on the motion information to generate a filtering result of the motion information.

[0018] In some embodiments of the present application, the control module is further configured to: if the target is classified as the second classification, obtain a first feature of the long exposure image, the first feature being obtained by encoding the long exposure image; input the first feature into a sign feature perception network, the sign feature perception network being used to identify sign information in the image, the sign information including at least one of traffic signs, lane lines and road surface signs; and obtain the sign information output by the sign feature perception network.

[0019] In a second aspect, some embodiments of the present application further provide a method for detecting a driving environment, comprising the following steps:

[0020] Acquire an image of a driving environment acquired by an image acquisition module, the image comprising a long-exposure image and a short-exposure image, the exposure time of the long-exposure image being greater than the exposure time of the short-exposure image, the acquisition frequency of the long-exposure image being higher than the acquisition frequency of the short-exposure image, and the short-exposure image being used to supplement information of a highlight area in the long-exposure image;

[0021] Detecting target classification according to the long exposure image and the short exposure image, the target classification is used to characterize the light intensity of the driving environment, the target classification includes a first classification and a second classification, the light intensity characterized by the first classification is higher than the light intensity characterized by the second classification;

[0022] If the target is classified as the first classification, performing a first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection;

[0023] If the target is classified as the second classification, a second obstacle detection is performed based on the long exposure image and the short exposure image, and a second result of the second obstacle detection is output.

[0024] In a third aspect, some embodiments of the present application further provide an electronic device, comprising a processor, a memory and a bus. The processor and the memory are communicatively connected via the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to execute the following program steps:

[0025] Acquire an image of a driving environment acquired by an image acquisition module, the image comprising a long-exposure image and a short-exposure image, the exposure time of the long-exposure image being greater than the exposure time of the short-exposure image, the acquisition frequency of the long-exposure image being higher than the acquisition frequency of the short-exposure image, and the short-exposure image being used to supplement information of a highlight area in the long-exposure image;

[0026] Detecting target classification according to the long exposure image and the short exposure image, the target classification is used to characterize the light intensity of the driving environment, the target classification includes a first classification and a second classification, the light intensity characterized by the first classification is higher than the light intensity characterized by the second classification;

[0027] If the target is classified as the first classification, performing a first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection;

[0028] If the target is classified as the second classification, a second obstacle detection is performed based on the long exposure image and the short exposure image, and a second result of the second obstacle detection is output.

[0029] It can be seen from the above technical solutions that some embodiments of the present application provide a driving environment detection system, method and electronic device, and the method can obtain an image of the driving environment captured by the image acquisition module. Among them, the image includes a long exposure image and a short exposure image, and the short exposure image is used to supplement the information of the highlight area in the long exposure image. Then, the target classification is detected according to the long exposure image and the short exposure image. If the target classification is the first classification, the first obstacle detection is performed according to the long exposure image, and the first result of the first obstacle detection is output; and if the target classification is the second classification, the second obstacle detection is performed according to the long exposure image and the short exposure image, and the second result of the second obstacle detection is output. Among them, the light intensity of the driving environment represented by the first classification is higher than the light intensity of the driving environment represented by the second classification. In the scene with darker light intensity, the detection method combines the long exposure image and the short exposure image to perform obstacle detection, which can improve the accuracy of obstacle detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 An architecture diagram of a driving environment system provided for some embodiments of the present application;

[0032] Figure 2 A schematic flow chart of a driving environment method provided in some embodiments of the present application;

[0033] Figure 3 A schematic diagram of a process for acquiring long exposure images and short exposure images provided in some embodiments of the present application;

[0034] Figure 4 A schematic diagram of a process for detecting target classification provided in some embodiments of the present application;

[0035] Figure 5 A schematic diagram of another process of detecting target classification provided in some embodiments of the present application;

[0036] Figure 6 A schematic diagram of an obstacle perception deep learning network provided for some embodiments of the present application;

[0037] Figure 7 A schematic diagram of a deep learning network for obstacle perception based on a landmark detection process provided for some embodiments of the present application;

[0038] Figure 8 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0040] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0041] It should be understood that the terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0042] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0043] Under poor lighting conditions at night, the implementation of environmental perception technology faces many challenges. On the one hand, the perception input of traffic signs, lane lines, and road markings requires fill light from vehicle lights. On the other hand, night vehicle detection cannot be based entirely on target contours and geometric shapes. For example, the target contour and geometric shape information is incomplete under poor lighting conditions at night, or the vehicle's high beam light group obscures the target contour.

[0044] In some embodiments, in order to improve the above-mentioned problem of insufficient light, the vehicle can actively fill in the light by turning on the headlights, low beams and auxiliary light sources to enhance the image acquisition capabilities of image acquisition modules such as cameras. At the same time, the visual perception effect in a dark environment can be further improved by optimizing image processing algorithms, such as enhancing contrast, denoising and edge detection.

[0045] However, in low-light driving environments such as at night, the visual sensor of the image acquisition module may not be able to capture enough details, resulting in the loss of the outline and geometric information of the obstacle; and, due to the interference of the high beam light, the picture captured by the image collector may be overexposed, making it difficult to accurately identify and locate obstacles, resulting in a decrease in the accuracy of image recognition during environmental perception, affecting the user experience.

[0046] To this end, some embodiments of the present application provide a driving environment detection system that can be applied to technical fields such as autonomous driving. The detection system combines long-exposure and short-exposure images. Short-exposure images can be used to obtain visual information such as vehicle high beams and low beams, and long-exposure images are mainly used to obtain contours and geometric information such as obstacles, ground signs, traffic lights, and traffic signs, so as to achieve refined perception of the driving environment and improve the detection accuracy of the driving environment.

[0047] like Figure 1 As shown, in some embodiments, the detection system includes an image acquisition module 101 and a control module 102. The image acquisition module 101 is configured to acquire images, the images including long-exposure images and short-exposure images, the exposure time of the long-exposure image is greater than the exposure time of the short-exposure image, and the short-exposure image is used to supplement the information of the highlight area in the long-exposure image. For example, the image acquisition module 101 can be a vehicle-mounted camera, etc.

[0048] In some embodiments, the exposure time of the long exposure image is greater than or equal to the target value, and the exposure time of the short exposure image is less than the target value. The detection system can control the image acquisition module to acquire the long exposure image and the short exposure image according to the range divided by the target value. Alternatively, in some embodiments, the exposure time of the long exposure image is a first target value, and the exposure time of the short exposure image is a second target value, both of which are fixed values, and the first target value is greater than the second target value. The detection system can control the image acquisition module to capture the corresponding images according to the first target value and the second target value to form a long exposure image and a short exposure image.

[0049] like Figure 2 As shown, the control module 102 is configured to perform the following program steps:

[0050] S1: Acquire a long exposure image and a short exposure image acquired by an image acquisition module.

[0051] In the process of sensing the driving environment, the detection system will continuously capture corresponding images through the image acquisition module 101 to perceive the real-time driving environment through images. In some embodiments, the detection system can control the image acquisition module to capture images with different exposure times to generate long-exposure images and short-exposure images. Among them, the detection system can continuously capture long-exposure images, and then determine whether to capture short-exposure images through other parameters.

[0052] In some embodiments, in order to adapt to the driving environment of the vehicle, the acquisition frequency of the long exposure image is higher than the acquisition frequency of the short exposure image. That is, the detection system can control the image acquisition module to acquire long exposure images by default, and only acquire short exposure images when it is determined that the scene is dark.

[0053] In some embodiments, the detection system further includes a light source module 103, which is turned on according to time information, location information or the long exposure image, such as a specific time point, a specific location and a combination of the above two information, or judging from the long exposure image that the current driving environment is dark. Figure 3 As shown, when the detection system acquires the long exposure image and the short exposure image acquired by the image acquisition module 101, the image acquisition module 101 can be controlled to acquire the long exposure image in real time, and the startup state of the light source module 103 can be monitored, and when the light source module 103 is in the on state, the on time of the light source module 103 can be recorded. When the on time of the light source module 103 is greater than or equal to the preset time threshold, the image acquisition module 101 is controlled to acquire the short exposure image. Then the long exposure image and the short exposure image are read to obtain the long exposure image and the short exposure image.

[0054] For example, the preset time threshold is T1, and the detection system continuously controls the image acquisition module to acquire the long exposure image ImgLongEx p and monitors the light source module 103 of the vehicle. The light source module 103 includes a headlight and a high beam. When the detection system detects that the headlight or high beam of the vehicle is continuously turned on for more than the threshold T1, the image acquisition module is controlled to generate a short exposure image ImgShortExp.

[0055] In some embodiments, when the detection system acquires the long exposure image and the short exposure image acquired by the image acquisition module 101, the image acquisition module 101 is controlled to acquire the long exposure image in real time and read the time information and the position information. If the time information is in a preset time period, or the position information is in a preset position, the image acquisition module is controlled to acquire the short exposure image, and then the long exposure image and the short exposure image are read to obtain the long exposure image and the short exposure image.

[0056] Exemplarily, the detection system can be preset with a specific time period to automatically start the collection of short exposure images at night; alternatively, the detection system can read the vehicle's location information in a map in real time and automatically start the collection of short exposure images at specific locations such as under a bridge or on a road without street lights.

[0057] In some embodiments, the detection system can detect the startup state of the light source module 103 or the detection system can read the time information and position information according to a preset period to reduce the consumption of system resources. For example, the detection of the light source module 103 or the reading of event information and position information is performed once every 3 seconds. For the convenience of description, some embodiments of this application represent the acquisition and judgment process of the short exposure image as a preliminary judgment.

[0058] S2: Detecting target classification according to the long exposure image and the short exposure image.

[0059] After the detection system performs preliminary judgment and collects the short exposure image, the target classification corresponding to the current driving environment can be detected based on the long exposure image and the short exposure image. For the sake of distinction and description, some embodiments of the present application represent the process of detecting target classification based on the long exposure image and the short exposure image as formal judgment.

[0060] Among them, the target classification is used to characterize the light intensity of the environment captured by the image acquisition module 101. The target classification includes a first classification and a second classification, and the light intensity represented by the first classification is higher than the light intensity represented by the second classification, that is, the first classification can be used to characterize a driving environment with brighter lighting conditions, and the second classification can be used to characterize a driving environment with darker lighting conditions.

[0061] like Figure 4As shown, in some embodiments, when the detection system detects the target classification, it can calculate the first grayscale histogram of the long exposure image and the second grayscale histogram of the short exposure image. Then detect the first pixel point whose grayscale value is less than the first grayscale threshold in the first grayscale histogram, and detect the second pixel point whose grayscale value is greater than the second grayscale threshold in the second grayscale histogram. If the proportion of the first pixel point in the first grayscale histogram is greater than the first proportion threshold, and the proportion of the second pixel point in the second grayscale histogram is greater than the second proportion threshold, the detection system marks the target classification as the second classification; if the proportion of the first pixel point is less than or equal to the first proportion threshold, or the proportion of the second pixel point is less than or equal to the second proportion threshold, the detection system marks the target classification as the first classification.

[0062] For example, the first grayscale threshold is T2, the first ratio threshold is T3, the second grayscale threshold is T4, and the second ratio threshold is T5. The detection system can calculate the grayscale histogram for the long exposure image and the short exposure image respectively, where the grayscale histogram represents the number of pixels at each grayscale level (0 to 255) in the image. T2 and T3 are used for processing long exposure images, and T4 and T5 are used for processing short exposure images. The proportion of pixels with grayscale values ​​lower than T2 in the long exposure image is calculated. If the proportion is higher than T3, the first condition is met. Similarly, the proportion of pixels with grayscale values ​​higher than T4 in the short exposure image is calculated. If the proportion is higher than T5, the second condition is met. When both the first condition and the second condition are met, the target classification is marked as the second classification; otherwise, it is marked as the first classification.

[0063] In some embodiments, when the detection system detects target classification, it can also calculate the average grayscale or non-random spot area and number of long exposure images and short exposure images, and then mark the target classification by the calculated average grayscale or non-random spot area and number.

[0064] like Figure 5 As shown, in some embodiments, when the detection system detects the classification of the target, the long exposure image and the short exposure image can also be input into the classification network. The classification network includes a dual input layer and a classification decision layer, the dual input layer is used to input the long exposure image and the short exposure image, and the classification decision layer is used to generate a classification label according to the features of the long exposure image and the short exposure image. Then, the classification label output by the classification network is obtained, and the target classification is marked according to the classification label. That is, a classification network can be trained by long exposure images and short exposure images, and the classification label output by the classification network is used as the result of the target classification.

[0065] For example, the classification decision layer is a binary classification, the classification labels are a first label and a second label, the first label is used to represent the first classification, and the second label is used to represent the second classification. The detection system can calculate the probabilities of the first label and the second label through the classification network to determine the classification labels corresponding to the current long exposure image and the short exposure image.

[0066] In some embodiments, the classification network may include a dual-input design, feature extraction capabilities (such as structures such as convolutional layers and pooling layers), a feature fusion or merging mechanism, and a classification decision layer (such as a fully connected layer). The classification network can determine whether the target is classified as the first category or the second category based on the input long exposure image and short exposure image.

[0067] S3: If the target is classified as the first classification, perform first obstacle detection according to the long exposure image, and output a first result of the first obstacle detection.

[0068] After the detection system detects the target classification of the current driving environment, obstacle detection can be performed according to the target classification. For the sake of distinction, the embodiment of the present application represents obstacle detection as first obstacle detection and second obstacle detection, where the first obstacle detection is obstacle detection corresponding to the first classification, and the second obstacle detection is obstacle detection corresponding to the second classification.

[0069] Accordingly, when the detection system detects that the target classification is the first classification, it indicates that the lighting conditions of the current driving environment are bright, and obstacle detection can be performed according to the continuously collected long exposure images. For example, obstacle detection based on deep learning, such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector), can be performed according to the long exposure images; or obstacle detection based on visual semantic segmentation can be performed according to the long exposure images.

[0070] It is understandable that the above-mentioned first obstacle detection method is only some exemplary explanations and is not a limitation thereto, and the first obstacle detection described in the embodiment of the present application may be any obstacle detection method performed based on the long exposure image. This application does not impose any limitation on this.

[0071] S4: If the target is classified as the second classification, perform second obstacle detection according to the long exposure image and the short exposure image, and output a second result of the second obstacle detection.

[0072] When the detection system detects that the target is classified as the first category, it indicates that the lighting conditions of the current driving environment are dark, and there may be problems such as incomplete target outline and lighting effects. At this time, the detection system can perform obstacle detection by combining the long exposure image and the short exposure image, that is, the second obstacle detection.

[0073] In some embodiments, when the detection system performs the second obstacle detection, the long exposure image and the short exposure image may be encoded to obtain a first feature of the long exposure image and a second feature of the short exposure image. The first feature and the second feature are then fused to generate a fused feature. Then, the fused feature is input into the obstacle perception network, and the obstacle information output by the obstacle perception network is obtained to obtain a second result. The obstacle perception network is a perspective-based network or a bird's-eye-view-based network, and the obstacle perception network is used to identify obstacle information in the image, and the obstacle information includes the position, scale, and category of the obstacle.

[0074] That is to say, the long exposure image and the short exposure image can be encoded by the corresponding encoding module first, and then the encoded long exposure image and the short exposure image can be fused by the fusion module. Then, the fused long exposure image and the short exposure image are input into the pre-trained obstacle perception network to output the position, scale, posture, category and other information of the obstacle. Among them. The obstacle perception deep learning network can be a perspective-based network, a bird's-eye view (BEV)-based network, or a network of other mechanisms. For example, Figure 6 As shown, Figure 6 A schematic diagram of an obstacle perception deep learning network in one implementation mode.

[0075] The detection system can obtain corresponding obstacle detection results, namely, first results and second results, according to the first obstacle detection and the second obstacle detection. The first results and the second results can be used to execute subsequent driving decisions to assist in realizing the vehicle's automatic driving technology.

[0076] In some embodiments, after the detection system performs the second obstacle detection, it can also track the obstacle across frames according to the second result to obtain the motion information of the obstacle. The second result includes the position, scale and category of the obstacle, and the motion information includes at least one of the motion state, motion direction and acceleration. The motion information is then filtered to generate a filtered result of the motion information, i.e., a filtered obstacle detection result. In this way, the detection system can track obstacles that are in relative motion due to the movement of the vehicle, thereby improving the accuracy and speed of obstacle detection.

[0077] Furthermore, in order to improve the detection accuracy of signs such as traffic signs, lane lines, and road surface signs, in some embodiments, if the target is classified as the second classification, the detection system also obtains the first feature of the long exposure image. The first feature is obtained by encoding the long exposure image. The first feature is then input into the sign feature perception network, and the sign information output by the sign feature perception network is obtained. The sign feature perception network is used to identify the sign information in the image, and the sign information includes at least one of traffic signs, lane lines, and road surface signs.

[0078] In this way, the detection system can further enhance the recognition and detection of traffic signs, lane lines, and road surface markings through long exposure images. Combining the above-mentioned filtered obstacle detection results and sign information, the detection system can more accurately detect the actual conditions of the driving environment under a variety of different lighting conditions, thereby improving the detection effect of the driving environment.

[0079] For example, Figure 7 As shown, Figure 7 The process includes the second obstacle detection and the detection of traffic signs, lane lines, road signs, etc., and the sign detection is performed by the ground sign / traffic sign perception output module. Among them, the obstacle perception network is a network based on the bird's eye view (BEV), including a BEV encoding module and a BEV conversion module; and when detecting signs through long exposure images, the network also enhances the image features in the long exposure images through the feature enhancement module.

[0080] In some embodiments, the feature enhancement module can be used to perform at least one of histogram equalization, sharpening filtering, noise suppression, color enhancement, and deep learning methods to improve the feature expression capability of the long exposure image.

[0081] It should be noted that in the embodiments of the present application, there may be multiple ways to detect traffic signs, lane lines, road surface markings and other signs, such as color recognition, shape recognition, machine learning and deep learning corresponding to TSR (Traffic Sign Recognition); edge detection, Hough transform and deep learning corresponding to lane line perception; image processing and deep learning corresponding to road surface marking perception, etc. This application does not impose any limitation on this.

[0082] Based on the above-mentioned driving environment detection system, some embodiments of the present application also provide a driving environment detection method, such as Figure 2 As shown, the detection method comprises the following steps:

[0083] S1: Acquire a long-exposure image and a short-exposure image acquired by an image acquisition module, wherein an acquisition frequency of the long-exposure image is higher than an acquisition frequency of the short-exposure image;

[0084] S2: Detecting target classification according to the long exposure image and the short exposure image, wherein the target classification is used to characterize the light intensity of the environment captured by the image acquisition module, and the target classification includes a first classification and a second classification, and the light intensity represented by the first classification is higher than the light intensity represented by the second classification;

[0085] S3: If the target is classified as the first classification, perform first obstacle detection according to the long exposure image, and output a first result of the first obstacle detection;

[0086] S4: If the target is classified as the second classification, perform second obstacle detection according to the long exposure image and the short exposure image, and output a second result of the second obstacle detection.

[0087] Based on the above-mentioned driving environment detection system and method, some embodiments of the present application also provide an electronic device 500, including the driving environment detection system described in the above-mentioned embodiments.

[0088] In some embodiments, Figure 8 As shown, the electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. Among them, the bus 504 is used to realize the direct connection and communication of these components, the communication interface 502 is used to communicate signaling or data with other node devices, and the memory 503 stores computer program instructions that can be executed by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 through the bus 504, the processor 501 can call the computer program stored in the memory 503, and the execution of the computer program can realize the driving environment detection method provided in the embodiment of the present application.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0090] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A driving environment detection system, characterized in that: include: An image acquisition module is configured to acquire images of a driving environment, wherein the images include a long-exposure image and a short-exposure image, wherein the exposure time of the long-exposure image is greater than the exposure time of the short-exposure image, and the short-exposure image is used to supplement information of a highlight area in the long-exposure image; The control module is configured as follows: Acquire a long exposure image and a short exposure image acquired by the image acquisition module, wherein an acquisition frequency of the long exposure image is higher than an acquisition frequency of the short exposure image; Detecting target classification according to the long exposure image and the short exposure image, the target classification is used to characterize the light intensity of the driving environment, the target classification includes a first classification and a second classification, the light intensity characterized by the first classification is higher than the light intensity characterized by the second classification; If the target is classified as the first classification, performing a first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection; If the target is classified as the second classification, a second obstacle detection is performed based on the long exposure image and the short exposure image, and a second result of the second obstacle detection is output.

2. The driving environment detection system according to claim 1, characterized in that: It also includes a light source module, which is turned on according to time information, position information or the long exposure image; the control module executes the acquisition of the long exposure image and the short exposure image acquired by the image acquisition module, and is specifically configured as follows: Controlling the image acquisition module to acquire the long exposure image in real time; monitoring the startup state of the light source module, and when the light source module is in the startup state, recording the startup time of the light source module; When the on time of the light source module is greater than or equal to a preset time threshold, controlling the image acquisition module to acquire the short exposure image; The long exposure image and the short exposure image are read.

3. The driving environment detection system according to claim 1, characterized in that: The control module executes the acquisition of the long exposure image and the short exposure image acquired by the image acquisition module, and is specifically configured as follows: Controlling the image acquisition module to acquire the long exposure image in real time; Read time information and location information; If the time information is within a preset time period, or the position information is within a preset position, controlling the image acquisition module to acquire the short exposure image; The long exposure image and the short exposure image are read.

4. The driving environment detection system according to claim 1, characterized in that: The control module performs target classification according to the long exposure image and the short exposure image, and is specifically configured as follows: Calculating a first grayscale histogram of the long-exposure image and a second grayscale histogram of the short-exposure image; Detecting a first pixel point in the first grayscale histogram whose grayscale value is less than a first grayscale threshold, and detecting a second pixel point in the second grayscale histogram whose grayscale value is greater than a second grayscale threshold; If the proportion of the first pixel in the first grayscale histogram is greater than a first proportion threshold, and the proportion of the second pixel in the second grayscale histogram is greater than a second proportion threshold, marking the target classification as the second classification; If the ratio of the first pixel points is less than or equal to a first ratio threshold, or the ratio of the second pixel points is less than or equal to a second ratio threshold, the target classification is marked as the first classification.

5. The driving environment detection system according to claim 1, characterized in that: The control module performs target classification according to the long exposure image and the short exposure image, and is specifically configured as follows: Inputting the long exposure image and the short exposure image into a classification network, wherein the classification network includes a dual input layer and a classification decision layer, wherein the dual input layer is used to input the long exposure image and the short exposure image, and the classification decision layer is used to generate classification labels according to features of the long exposure image and the short exposure image; Obtaining a classification label output by the classification network; The target classification is labeled according to the classification label.

6. The driving environment detection system according to claim 1, characterized in that: The control module performs second obstacle detection according to the long exposure image and the short exposure image, and is specifically configured as follows: performing encoding on the long exposure image and the short exposure image to obtain a first feature of the long exposure image and a second feature of the short exposure image; fusing the first feature and the second feature to generate a fused feature; Inputting the fused features into an obstacle perception network, wherein the obstacle perception network is a perspective view-based network or a bird's-eye view-based network, and the obstacle perception network is used to identify obstacle information in the image, wherein the obstacle information includes the position, scale and category of the obstacle; Obtain obstacle information output by the obstacle perception network to obtain the second result.

7. The driving environment detection system according to claim 1, characterized in that: After the control module executes the step of performing second obstacle detection according to the long exposure image and the short exposure image, the control module is further configured to: Tracking the obstacle across frames according to the second result to obtain motion information of the obstacle; the second result includes the position, scale and category of the obstacle, and the motion information includes at least one of a motion state, a motion direction and an acceleration; A filtering process is performed on the motion information to generate a filtering result of the motion information.

8. The driving environment detection system according to claim 1, characterized in that: The control module is further configured to: If the target is classified as the second classification, obtaining a first feature of the long exposure image, where the first feature is obtained by encoding the long exposure image; Inputting the first feature into a sign feature perception network, the sign feature perception network is used to identify sign information in the image, the sign information including at least one of a traffic sign, a lane line, and a road surface sign; Obtaining the landmark information output by the landmark feature perception network.

9. A method for detecting a driving environment, characterized in that: include: Acquire an image of a driving environment acquired by an image acquisition module, the image comprising a long-exposure image and a short-exposure image, the exposure time of the long-exposure image being greater than the exposure time of the short-exposure image, the acquisition frequency of the long-exposure image being higher than the acquisition frequency of the short-exposure image, and the short-exposure image being used to supplement information of a highlight area in the long-exposure image; Detecting target classification according to the long exposure image and the short exposure image, the target classification is used to characterize the light intensity of the driving environment, the target classification includes a first classification and a second classification, the light intensity characterized by the first classification is higher than the light intensity characterized by the second classification; If the target is classified as the first classification, performing a first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection; If the target is classified as the second classification, a second obstacle detection is performed based on the long exposure image and the short exposure image, and a second result of the second obstacle detection is output.

10. An electronic device, characterized in that: Includes processor, memory and bus; The processor and the memory are communicatively connected via the bus; The memory stores computer program instructions executable by the processor, the processor being configured to: Acquire an image of a driving environment acquired by an image acquisition module, the image comprising a long-exposure image and a short-exposure image, the exposure time of the long-exposure image being greater than the exposure time of the short-exposure image, the acquisition frequency of the long-exposure image being higher than the acquisition frequency of the short-exposure image, and the short-exposure image being used to supplement information of a highlight area in the long-exposure image; Detecting target classification according to the long exposure image and the short exposure image, the target classification is used to characterize the light intensity of the driving environment, the target classification includes a first classification and a second classification, the light intensity characterized by the first classification is higher than the light intensity characterized by the second classification; If the target is classified as the first classification, performing a first obstacle detection according to the long exposure image, and outputting a first result of the first obstacle detection; If the target is classified as the second classification, a second obstacle detection is performed based on the long exposure image and the short exposure image, and a second result of the second obstacle detection is output.

Images