Vehicle front-view camera system, vehicle control method and vehicle

Abstract

The invention discloses a vehicle foresight camera system, a vehicle control method and a vehicle, and relates to the technical field of vehicle control. The vehicle foresight camera system comprises an image acquisition assembly, an environment sensing assembly, an image processing assembly and a control assembly, the image acquisition component is set to acquire an image of a preset area; the environment sensing assembly is set to collect temperature information and humidity information of the vehicle; the image processing assembly is used for carrying out raindrop detection on the image to obtain a raindrop detection result and carrying out fog detection on the image to obtain a fog detection result; and the control assembly is arranged to control operation of a windscreen wiper of the vehicle based on the raindrop detection result and control operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result. The technical problem that driving safety is affected due to the fact that raindrop detection and fog detection of the vehicle windshield are independent and low in efficiency in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, in particular to a vehicle front-view camera system, a vehicle control method and a vehicle. BACKGROUND

[0002] With the rapid development of automobile intelligence, ensuring the clear vision of the driver has become a key to improving driving safety and driving experience.

[0003] On traditional vehicles, rain and fog detection relies on separate sensors, such as infrared rain sensors and temperature and humidity sensors. These sensors not only occupy space and increase system cost, but also have limited accuracy and response speed in the face of complex environmental conditions. Therefore, how to achieve efficient detection of rain and fog and ensure driving safety is one of the important technical problems in the related technical field.

[0004] At present, there is no effective solution to the above problems. SUMMARY

[0005] The embodiments of the present application provide a vehicle front-view camera system, a vehicle control method and a vehicle to at least solve the technical problem that the raindrop detection and fog detection of the windshield of the vehicle in the related art are independent and low in efficiency, thereby affecting driving safety.

[0006] According to an aspect of an embodiment of the present application, a vehicle front-view camera system is provided, comprising: an image acquisition component, an environment perception component, an image processing component and a control component; the image acquisition component is configured to acquire images of a preset area; the environment perception component is configured to acquire temperature information and humidity information of the vehicle; the image processing component is configured to perform raindrop detection on the images to obtain a raindrop detection result, and perform fog detection on the images to obtain a fog detection result; and the control component is configured to control the operation of a rain wiper of the vehicle based on the raindrop detection result, and control the operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result.

[0007] Optionally, the raindrop detection on the images to obtain the raindrop detection result comprises: preprocessing the images to obtain preprocessed images; performing pixel-level classification on the preprocessed images using a pre-trained classification model to obtain candidate raindrop regions; performing raindrop feature extraction on the candidate raindrop regions to obtain raindrop feature extraction results; and analyzing the raindrop feature extraction results using a pre-trained raindrop recognition model to obtain raindrop judgment results, wherein the raindrop judgment results are used to represent the probability that the candidate raindrop regions are real raindrops; in response to the raindrop judgment results being greater than a preset probability threshold, performing raindrop density calculation on the candidate raindrop regions to obtain calculation results; and determining the raindrop detection result according to the calculation results.

[0008] Optionally, the image is subjected to fog detection to obtain a fog detection result, including: subjecting the image to contrast analysis and edge blur analysis to obtain the fog detection result.

[0009] Optionally, the raindrop detection result includes: a first rain intensity level, a second rain intensity level and a third rain intensity level, and the operation of the vehicle wiper is controlled based on the raindrop detection result, including: in response to the raindrop detection result being the first rain intensity level, determining the wiper control strategy as stopping the wiper operation; in response to the raindrop detection result being the second rain intensity level, determining the wiper control strategy as controlling the wiper to operate intermittently at a low speed; and in response to the raindrop detection result being the third rain intensity level, determining the wiper control strategy as controlling the wiper to operate continuously at a high speed, wherein the rainfall corresponding to the first rain intensity level is less than the rainfall corresponding to the second rain intensity level, and the rainfall corresponding to the second rain intensity level is less than the rainfall corresponding to the third rain intensity level; and the operation of the wiper is controlled based on the wiper control strategy.

[0010] Optionally, the operation of the vehicle air conditioner is controlled based on the temperature information, the humidity information and the fog detection result, including: in response to the fog detection result indicating that the image has fog image, controlling the air conditioner to start a defogging mode; and in response to the fog detection result indicating that the image has no fog image, and based on the temperature information and the humidity information, determining that the vehicle glass has a risk of fogging, controlling the air conditioner to start the defogging mode.

[0011] Optionally, the image acquisition component includes: an optical lens group, an infrared filter, an image sensor and an image signal processor; the optical lens group is configured to collect an optical image and project the optical image to the image sensor; the infrared filter is connected to the optical lens group and is configured to filter out infrared light from the optical image to obtain a processed image; the image sensor is connected to the infrared filter and is configured to convert the processed image into an electrical signal; and the image signal processor is connected to the image sensor and is configured to process the electrical signal to obtain the image of the preset area.

[0012] Optionally, the vehicle front-view camera system further includes a thermal management component, an upgrade component and a communication and power supply component; the thermal management component is configured to thermally isolate the vehicle front-view camera system; the upgrade component is configured to upgrade the vehicle front-view camera system; and the communication and power supply component is configured to supply power to the vehicle front-view camera system and establish a communication connection between the vehicle front-view camera system and the vehicle control unit.

[0013] According to another aspect of the embodiments of the present application, a vehicle control method is also provided, which is applied to the vehicle front-view camera system in any of the above embodiments, and the vehicle control method comprises: collecting images of a preset area; collecting temperature information and humidity information of the vehicle; performing raindrop detection on the images to obtain a raindrop detection result, and performing fog detection on the images to obtain a fog detection result; controlling operation of a rain wiper of the vehicle based on the raindrop detection result, and controlling operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result.

[0014] According to another aspect of the embodiments of the present application, a vehicle is also provided, which comprises the vehicle front-view camera system in any of the above embodiments.

[0015] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which comprises a stored executable program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the vehicle control method when the executable program is executed.

[0016] In the embodiments of the present application, a vehicle front-view camera system is provided, which comprises: an image collection component, an environment perception component, an image processing component and a control component; the image collection component is configured to collect images of a preset area; the environment perception component is configured to collect temperature information and humidity information of the vehicle; the image processing component is configured to perform raindrop detection on the images to obtain a raindrop detection result, and perform fog detection on the images to obtain a fog detection result; and the control component is configured to control operation of a rain wiper of the vehicle based on the raindrop detection result, and control operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result. In the present application, the image collection component can comprehensively capture real-time images of the front of the vehicle and the windshield, which provides rich visual information for subsequent raindrop and fog detection, and ensures the accuracy and timeliness of the detection; the environment perception component is responsible for collecting temperature and humidity data inside and outside the vehicle, which is crucial for predicting and monitoring the formation of fog, and can provide early warning to prevent problems; the image processing component analyzes image data using advanced algorithms, which can not only accurately identify raindrops, but also effectively detect the state of fog, greatly improving the intelligent level of environment perception through the identification of raindrop characteristics and the analysis of the visual effect of fog; finally, the control component intelligently controls the operation of the rain wiper based on the results of the image processing component, ensuring that raindrops on the windshield can be quickly removed on rainy days, and automatically adjusts the air conditioning system based on the data of the environment perception component and the fog detection result, effectively preventing the windshield from fogging, and ensuring the clarity of the driving field of view on rainy and foggy days. In summary, the present application realizes efficient detection of rain and fog to ensure driving safety, thereby solving the technical problems of independent and low-efficiency raindrop detection and fog detection on the windshield of the vehicle in related technologies, which further affects driving safety. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a structural block diagram of a vehicle forward-looking camera system according to one embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the lens structure of a vehicle forward-view camera according to one embodiment of the present invention;

[0020] Figure 3 This is a system architecture diagram of a vehicle forward-looking camera system according to one embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the structure of a vehicle forward-view camera according to one embodiment of the present invention;

[0022] Figure 5 This is a flowchart of a vehicle control method according to one embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] This invention provides a vehicle forward-looking camera system. Figure 1This is a structural block diagram of a vehicle forward-looking camera system according to one embodiment of the present invention, such as... Figure 1 As shown, the vehicle forward-facing camera system 100 includes: an image acquisition component 101, an environmental perception component 102, an image processing component 103, and a control component 104; the image acquisition component 101 is configured to acquire images of a preset area; the environmental perception component 102 is configured to acquire the vehicle's temperature and humidity information; the image processing component 103 is configured to perform raindrop detection on the images to obtain raindrop detection results, and to perform fog detection on the images to obtain fog detection results; the control component 104 is configured to control the operation of the vehicle's windshield wipers based on the raindrop detection results, and to control the operation of the vehicle's air conditioning based on the temperature information, humidity information, and fog detection results.

[0026] The aforementioned preset area refers to the area in front of the vehicle and the area of ​​the vehicle's windshield.

[0027] Optionally, the preset area typically includes the road, traffic signs, pedestrians, obstacles, and windshield surface in front of the vehicle to facilitate environmental perception, raindrop detection, and glass fogging recognition.

[0028] The aforementioned image acquisition component refers to the component integrated into the forward-facing camera system, used to capture image data of the area in front of the vehicle and the windshield.

[0029] Optionally, the image acquisition components include an optical lens, an image sensor, an image signal processor, and other auxiliary hardware such as filters.

[0030] The environmental sensing component is used to collect temperature and humidity information of the vehicle.

[0031] Optionally, the environmental sensing components include temperature and humidity sensors. Real-time monitoring of the temperature inside and outside the vehicle, including glass surface temperature, is used for anti-fogging prediction. Measuring the relative humidity of the air inside the vehicle helps determine the risk of fogging.

[0032] Alternatively, the temperature sensor may use a thermistor, thermocouple, or infrared temperature sensing element, which can respond quickly to temperature changes. A thermistor is a semiconductor resistor whose resistance changes with temperature, suitable for measuring the temperature of glass surfaces. Infrared temperature sensing elements measure temperature by detecting the infrared radiation emitted by an object; this non-contact method is suitable for monitoring the temperature of glass surfaces.

[0033] Optionally, the humidity sensor may be a capacitive or resistive humidity sensor, capable of sensing the water vapor content in the air. A capacitive humidity sensor measures relative humidity by utilizing the change in dielectric constant caused by humidity changes; a resistive humidity sensor monitors the change in resistance caused by humidity.

[0034] The image processing component is used to detect raindrops in the image of the preset area, and a raindrop detection result is obtained. In an optional embodiment, the image of the preset area is pre-processed, features are extracted, and feature recognition is performed to obtain the raindrop detection result. The raindrop detection result is used to represent the current rainfall level.

[0035] The image processing component is used to detect fog in the image of the preset area, and a fog detection result is obtained. In an optional embodiment, the image of the preset area is analyzed for contrast and edge blur, and the fog detection result is obtained. The fog detection result is used to determine whether there is fog in the image.

[0036] In an optional embodiment, the control component evaluates the rainfall intensity by analyzing the raindrop detection result. According to different levels of rainfall intensity, the operation mode of the wiper is automatically adjusted from stopping, intermittent, low speed, high speed, to extreme speed, ensuring that the wiper can effectively remove the rainwater on the windshield under different rainfall conditions, and ensuring the driver's clear vision.

[0037] In an optional embodiment, the control component controls the operation of the air conditioner of the vehicle based on the temperature information, the humidity information, and the fog detection result. When it is determined based on the temperature information, the humidity information, and the fog detection result that there is fog on the vehicle glass or there is a risk of fogging, the air conditioner defogging function is started.

[0038] The vehicle front-view camera system provided in the embodiment of the present application comprises an image acquisition component, an environment perception component, an image processing component and a control component; the image acquisition component is configured to acquire images of a preset area; the environment perception component is configured to acquire temperature information and humidity information of the vehicle; the image processing component is configured to perform raindrop detection on the images to obtain a raindrop detection result and perform fog detection on the images to obtain a fog detection result; and the control component is configured to control operation of a windshield wiper of the vehicle based on the raindrop detection result and control operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result. In the present application, the image acquisition component can comprehensively capture real-time images of the front of the vehicle and the windshield, which provides rich visual information for subsequent raindrop and fog detection, ensuring the accuracy and timeliness of the detection; the environment perception component is responsible for collecting temperature and humidity data inside and outside the vehicle, which is crucial for predicting and monitoring fog formation and can provide early warning to prevent problems; the image processing component analyzes image data using advanced algorithms, which can not only accurately identify raindrops but also effectively detect fog conditions, greatly improving the intelligent level of environment perception through the identification of raindrop characteristics and the analysis of fog visual effects; finally, the control component intelligently controls the operation of the windshield wiper based on the results of the image processing component to ensure that raindrops on the windshield can be quickly removed on rainy days, and automatically adjusts the air conditioning system based on the data of the environment perception component and the fog detection result to effectively prevent the windshield from fogging, ensuring clear visibility during driving in rainy and foggy weather. In summary, the present application realizes efficient detection of rainfall and fog to ensure driving safety, thereby solving the technical problems of independent and low-efficiency raindrop detection and fog detection on the windshield of the vehicle in related technologies, which affects driving safety.

[0039] Optionally, the raindrop detection on the images to obtain the raindrop detection result comprises the following steps:

[0040] Step S1031, pre-processing the images to obtain pre-processed images;

[0041] Step S1032, performing pixel-level classification on the pre-processed images using a pre-trained classification model to obtain candidate raindrop regions;

[0042] Step S1033, extracting raindrop features from the candidate raindrop regions to obtain raindrop feature extraction results;

[0043] Step S1034, analyzing the raindrop feature extraction results using a pre-trained raindrop recognition model to obtain a raindrop judgment result, wherein the raindrop judgment result represents the probability that the candidate raindrop region is a real raindrop;

[0044] Step S1035, in response to the raindrop judgment result being greater than the preset probability threshold, raindrop density calculation is performed on the candidate raindrop region to obtain a calculation result;

[0045] Step S1036, determining a raindrop detection result according to the calculation result.

[0046] Preprocessing is an important step before image analysis, aiming to improve image quality and enhance key features, so that subsequent algorithms can more accurately identify and analyze. Preprocessing includes but is not limited to white balance adjustment, denoising, contrast enhancement, etc.

[0047] White balance is to eliminate the influence of light source color temperature difference on image color, so that white objects in the image remain white under different light sources. In rainy environment, due to the change of lighting conditions, white balance adjustment is particularly important to restore the true color of the image.

[0048] In addition, during image acquisition, due to sensor noise, uneven lighting and other factors, random pixel points may appear on the image, i.e. "noise". Double filtering and other algorithms are used to remove these noises and maintain the clarity of important features such as raindrops.

[0049] Optionally, through contrast enhancement, the visual difference between different regions in the image is improved, making the details such as raindrops more prominent, facilitating subsequent feature extraction.

[0050] The above pre-trained model refers to a deep learning model trained by a large number of labeled image data, such as a learning model obtained by combining MobileNetV3 with U-Net architecture. This model can classify each pixel in the image and segment the image into different categories, such as raindrops, glass and background. The method uses pixel-level classification, meaning the model can be fine-tuned to identify the specific category of each pixel. The pre-trained classification model is used to perform pixel-level classification on the preprocessed image to obtain the candidate raindrop region.

[0051] Based on the candidate raindrop region, a series of features that help distinguish raindrops from non-raindrop objects are extracted, including but not limited to: geometric features, lighting features, and motion features. Optionally, geometric features include: circularity (reflecting the degree of raindrop circularity), eccentricity (reflecting the degree of raindrop shape asymmetry), area (reflecting the size of raindrop). Light features include: brightness gradient direction, top / bottom brightness difference (reflecting the lighting characteristics of raindrops, helping to determine the position and shape of raindrops in the image). Motion features include the downward speed of raindrops, which helps to distinguish raindrops from other static or dynamic objects such as dust or leaves.

[0052] Further, based on the extracted raindrop features, a model such as a support vector machine or a lightweight convolutional neural network is used to identify real raindrops. The model analyzes the raindrop feature extraction results and outputs the probability that the candidate raindrop region is a real raindrop.

[0053] Once the candidate raindrop region is determined to be a real raindrop, the number of raindrops per unit area is calculated. This process can be achieved by counting the number of raindrops per unit area (e.g., per square decimeter). The density calculation result is used to evaluate the rainfall intensity and determine the operation mode of the wiper.

[0054] In an optional embodiment, the raindrop detection result, including the rainfall intensity level, is determined in combination with the raindrop density calculation result. This result directly guides the automatic control of the vehicle wiper system, ensuring a clear view of the front windshield in rainy conditions.

[0055] Optionally, the image is subjected to fog detection to obtain a fog detection result, including: performing contrast analysis and edge blur analysis on the image to obtain a fog detection result.

[0056] Contrast analysis is a technique used in image processing to evaluate the brightness difference between different regions in an image. In this application, this analysis is used to detect fog images in the image, as fog will reduce the contrast of the image, making the boundaries of objects originally clear become blurred.

[0057] In an optional embodiment, the image of the preset region is subjected to grayscale processing to obtain a grayscale image. By calculating the average brightness and variance of the brightness of each small region (such as a sliding window) in the grayscale image, the local contrast is evaluated. High contrast means that the details and boundaries of objects in the image are clear, and low contrast may indicate the presence of fog. The results of the local contrast calculation are summarized to evaluate the contrast of the entire image. If the contrast is significantly lower than the normal value, it can be determined that there may be fog.

[0058] Edge blur analysis is a technique for detecting the sharpness of object edges in an image. Fog will make the boundaries of objects unclear, so edge blur can be used to indirectly determine the presence of fog.

[0059] In an optional embodiment, an edge detection algorithm (such as Canny edge detection or Sobel operator) is used to identify edges in the image. Edge detection is a process of identifying object boundaries by calculating the gradient changes of grayscale values in the image. At the same time, the sharpness of the edges is measured, such as the width, intensity, or gradient change rate of the edges. Fog will cause the edges to widen and the gradient change rate to decrease. If the blur of the edges exceeds a set threshold, it indicates that there is fog in the image, which may affect the driver's vision.

[0060] The fog detection result is obtained by combining the contrast analysis result and the edge blur analysis result. For example, if the contrast of the image is lower than a set contrast threshold and the edge blur is higher than a set blur threshold, it is determined that fog exists and a corresponding alarm or control operation is triggered.

[0061] Optionally, the raindrop detection result includes a first rain intensity level, a second rain intensity level, and a third rain intensity level. Based on the raindrop detection result, the operation of the windshield wiper of the vehicle is controlled, including the following steps:

[0062] In step S1041, in response to the raindrop detection result being the first rain intensity level, the windshield wiper control strategy is determined to be stopping the operation of the windshield wiper; in response to the raindrop detection result being the second rain intensity level, the windshield wiper control strategy is determined to be controlling the windshield wiper to operate intermittently at a low speed; and in response to the raindrop detection result being the third rain intensity level, the windshield wiper control strategy is determined to be controlling the windshield wiper to operate continuously at a high speed, wherein the rainfall corresponding to the first rain intensity level is less than the rainfall corresponding to the second rain intensity level, and the rainfall corresponding to the second rain intensity level is less than the rainfall corresponding to the third rain intensity level.

[0063] In step S1042, the operation of the windshield wiper is controlled based on the windshield wiper control strategy.

[0064] When the raindrop detection result is the first rain intensity level, it usually indicates no rain, which does not affect the driver's vision, and at this time the windshield wiper stops working to save energy and reduce unnecessary mechanical wear.

[0065] When the raindrop detection result is the second rain intensity level, it usually indicates moderate rainfall but not enough to continuously interfere with vision. The windshield wiper will operate at a lower speed and with a certain interval time to maintain the clarity of the windshield, while avoiding excessive use.

[0066] When the raindrop detection result is the third rain intensity level, it usually indicates that the rainfall reaches a strength sufficient to seriously affect the vision, and the windshield wiper is controlled to operate continuously at the maximum speed to ensure that the driver can still maintain good vision in heavy rain and other adverse weather conditions.

[0067] Based on the above determined windshield wiper control strategy, the operation of the windshield wiper is controlled. The windshield wiper operation control includes the control of the speed, direction and start / stop signal of the windshield wiper motor.

[0068] In an optional embodiment, according to the correspondence between the rain intensity level and the windshield wiper operation mode, a corresponding control signal is output to the windshield wiper motor to adjust its operation state. After receiving the control signal, the windshield wiper motor will adjust its operation state, including starting, stopping, changing speed and adjusting the intermittent time.

[0069] Optionally, based on the temperature information, the humidity information and the fog detection result, the operation of the air conditioner of the vehicle is controlled, including the following steps:

[0070] In step S1043, in response to the fog detection result indicating that the image has fog image, the air conditioner is controlled to start the defogging mode.

[0071] In step S1044, in response to the fog detection result indicating that the image has no fog image, and based on the temperature information and the humidity information, it is determined that the vehicle glass has a risk of fogging, the air conditioner is controlled to start the defogging mode.

[0072] In an optional embodiment, when the fog detection result indicates that the image has fog image, the air conditioning system is automatically controlled to enter the defogging mode. In the defogging mode, the air conditioning system adjusts the air direction to maximize the warm air or cold air directly blowing to the windshield area to quickly eliminate the fog. In addition, the temperature setting, air speed and air intake mode of the air conditioner are adjusted (such as switching to internal circulation) to optimize the defogging effect.

[0073] In an optional embodiment, when the fog detection result indicates that the image has no fog image, it is further determined whether the vehicle glass has a risk of fogging based on the temperature information and the humidity information.

[0074] Optionally, based on the temperature information and the humidity information, it is determined whether there is a risk of fogging by calculating the difference between the dew point temperature of the glass surface (the dew point temperature refers to the temperature at which the air cools to start condensing water droplets under a certain humidity. In the vehicle environment, when the temperature of the windshield surface is below the dew point temperature, the air with high humidity in the vehicle will condense into fog or water droplets on the surface) and the actual temperature, and the comparison between the current environmental humidity and the critical humidity for fogging.

[0075] When it is determined that the vehicle glass has a risk of fogging, the air conditioning system is controlled in advance to enter the defogging mode. This preventive control strategy can avoid the formation of fog on the windshield, ensuring that the driver can maintain a good view in all weather conditions.

[0076] Optionally, the image acquisition assembly comprises an optical lens group, an infrared filter, an image sensor and an image signal processor; the optical lens group is configured to collect an optical image and project the optical image to the image sensor; the infrared filter is connected with the optical lens group and is configured to filter out infrared light from the optical image to obtain a processed image; the image sensor is connected with the infrared filter and is configured to convert the processed image into an electrical signal; and the image signal processor is connected with the image sensor and is configured to process the electrical signal to obtain the image of the preset area.

[0077] The optical lens group is composed of multiple lens elements, used to focus and capture the optical image of the outside world. In this technical solution, it is designed to collect and focus the optical image in front of the vehicle and the windshield area, providing clear raw visual information for subsequent processing. Through the refraction of the lens, the light from the distant or front environment is converged into an image and projected onto the image sensor.

[0078] Optionally, the optical lens group includes a wide-angle lens and / or a zoom lens to adapt to different field of view requirements, ensuring that environmental changes such as raindrops, fog, etc. can be accurately captured whether at a distance or close up.

[0079] The infrared filter is an optical element that blocks infrared light waves and only allows visible light waves to pass through. During image acquisition, infrared light can penetrate fog and raindrops, affecting the accuracy of raindrop identification. The role of the infrared filter is to ensure that the signals received by the image sensor only contain visible light information, improving the accuracy of environmental factor identification such as raindrops and fog.

[0080] The image sensor is a photoelectric conversion element that converts optical signals into electrical signals. The image sensor receives light from the infrared filter and converts it into electrical signals that can be processed by electronic devices. Optionally, the image sensor converts the received optical image into a digital signal for processing by the image signal processor.

[0081] The image signal processor is a processor dedicated to processing the output signals of the image sensor, which can perform image preprocessing, noise reduction, etc., to output clear image data.

[0082] Optionally, the vehicle front-view camera system also includes a thermal management component, an upgrade component, and a communication and power supply component; the thermal management component is configured to thermally isolate the vehicle front-view camera system; the upgrade component is configured to upgrade the vehicle front-view camera system; the communication and power supply component is configured to power the vehicle front-view camera system and establish a communication connection between the vehicle front-view camera system and the vehicle control unit.

[0083] The main purpose of the thermal management component is to ensure that the vehicle front-view camera system can operate stably under various environmental conditions. It achieves this goal through thermal isolation mechanisms.

[0084] Thermal isolation refers to the establishment of physical barriers in the vehicle front-view camera system to reduce or prevent heat transfer from one area to another. For example, the glass surface of the housing is provided with a heat-conducting silicone pad and an aluminum alloy heat dissipation fin to ensure that the camera and other sensors are thermally isolated. The heat-conducting silicone pad is a soft material with high thermal conductivity, used to fill the small gaps between two contact surfaces, improving heat conduction efficiency. The aluminum alloy heat dissipation fin is used to quickly dissipate the heat generated during the operation of the camera.

[0085] The main task of the upgrade component is to periodically update and maintain the software of the vehicle front-view camera system to introduce new functions, fix bugs, or improve the performance of existing algorithms. It supports remote OTA (Over-the-Air) updates, allowing software upgrades to be completed without physical contact while the vehicle is in motion, enhancing the flexibility and maintainability of the system. OTA remote upgrade is a wireless update technology that allows devices to automatically download and install software updates over the network without user intervention.

[0086] The communication and power supply component is responsible for providing power supply for the vehicle front-view camera system and establishing communication connection with the vehicle control unit to transmit data in real time and accept instructions.

[0087] Optionally, Figure 2 is a schematic diagram of the lens structure of a vehicle front-view camera according to an embodiment of the present application, as Figure 2 shown, the lens structure of the vehicle front-view camera includes an optical lens, an image sensor, an image function processor, a serializer, and a connector.

[0088] Figure 3 is a system architecture diagram of a vehicle front-view camera system according to an embodiment of the present application, as Figure 3 shown, the vehicle front-view camera system includes a camera for anti-fog, visual recognition, and rain assessment. The system includes physical integration and algorithm integration. The physical integration includes chip integration and structural integration, and the algorithm integration includes rain visual recognition algorithm (including bilateral filtering, adaptive histogram equalization technology with limited contrast, and deep learning model) and fog temperature and humidity recognition algorithm (including fusion of temperature and humidity sensor and visual sensor).

[0089] In an alternative embodiment, a front-view camera system integrating rain recognition, anti-fog monitoring, and front-view visual perception functions is provided, which replaces the traditional rain sensor with an image recognition algorithm, realizes multifunctional integrated design, and solves the problems of large number of sensors, large space occupation, high cost, and serious visual obstruction in the prior art.

[0090] Optionally, the rain recognition algorithm, temperature and humidity sensing module, and front-view camera hardware are integrated in the same housing, sharing image processing chips, power management modules, and communication interfaces. In another alternative embodiment, the infrared emission end and receiving end of the rain sensor are installed on the front-view camera A surface (the camera faces the windshield surface), closely attached to the windshield, to detect the raindrop conditions in real time. In addition, the temperature sensor of the anti-fog sensor is also installed on the A surface to detect the glass wall temperature in real time; the B surface (facing the interior of the vehicle) is arranged with a temperature and humidity sensor for anti-fog prediction.

[0091] Optionally, the camera generates heat during operation, which affects the accuracy of other sensors. Heat dissipation fins or heat-conducting and heat-insulating materials are designed in the camera shell to ensure stable operation of the entire system.

[0092] Optionally, the image recognition replaces the rain sensor. The front-view camera collects windshield images, and the raindrop characteristics, including circularity, brightness distribution, and stripe morphology, are identified based on image processing algorithms. The rain intensity is determined according to the raindrop density and motion state, and the wiper is automatically controlled.

[0093] Optionally, the temperature and humidity sensor monitors the environment inside and outside the vehicle in real time, and the difference between the glass temperature and the dew point temperature is used to determine whether to start the air conditioner defogging mode. The camera image can be used as an auxiliary judgment to determine whether the glass has fogged, improving the response accuracy of the system.

[0094] Optionally, the embodiment of the application provides an integrated front-view camera system structure design, which is arranged in the central area of the top inside of the front windshield of the vehicle. The shell is composed of engineering plastic and metal heat dissipation fins, which have electromagnetic shielding, anti-vibration, and heat dissipation functions. The system integrates the following modules: image acquisition module: including three or single optical lens group, infrared filter, CMOS image sensor, ISP image signal processor, supporting visible light and near-infrared band imaging; image processing and recognition module: based on embedded SoC, running Linux or QNX system, built-in raindrop recognition, glass fog detection, front target recognition algorithm; environmental sensing module: glass surface temperature sensor (thermistor or infrared temperature measurement); indoor temperature and humidity sensor (capacitive humidity sensor + thermistor); communication and power supply module: supporting CAN-FD, 100BASE-T1 vehicle Ethernet, power supply for 12V vehicle power supply, with reverse protection and overvoltage protection functions; heat dissipation and thermal management structure: the shell is provided with heat-conducting silicone rubber pad and aluminum alloy heat dissipation fins on the glass surface, ensuring that the camera and other sensors are thermally isolated, with temperature drift controlled within ±2℃; firmware and OTA upgrade module: supporting encrypted firmware remote upgrade, with A / B partition backup mechanism to ensure system update security.

[0095] Optionally, in the embodiment of the present application, a deep learning-based image recognition algorithm is used to replace the traditional infrared rain sensor, and the specific process is as follows: image input: RAW format image or YUV422 format image collected by the front-view camera; image preprocessing: including white balance, denoising (bilateral filtering), and contrast enhancement; raindrop candidate region extraction: using U-NetMobileNetV3 to perform pixel-level classification on the image to distinguish raindrops, glass, and background; outputting a binary mask image and extracting a connected region as a candidate raindrop; raindrop feature extraction and classification: extracting the following features for each candidate region: geometric features: roundness, eccentricity, and area; photometric features: brightness gradient direction and top / bottom brightness difference; using an SVM or a lightweight CNN classifier to determine whether it is a real raindrop; rain intensity level determination: counting the number of raindrops per unit area (such as the number of raindrops per square decimeter); combining the average area and the movement speed of the raindrops to map to a 0-5 level rain intensity; outputting a wiper control signal: stop, intermittent, low speed, high speed, and extreme speed; post-processing and filtering: performing time series filtering on the recognition result to avoid false triggering.

[0096] Optionally, the camera image is used to identify whether the glass has fogged, but since image recognition has a lag (it needs to wait for fog to form), the temperature and humidity sensor is retained, and a prediction + verification dual mechanism is used. The dew point temperature is calculated in real time based on the temperature and humidity sensor data, and if the glass temperature is lower than the dew point temperature + 2℃, the air conditioner defogging is started in advance; the camera judges whether the glass has fogged through image recognition, and uses edge blur, contrast reduction, and light spot diffusion as feedback signals; if no fog image is detected within 10 seconds after the prediction mechanism is started, the air conditioner defogging intensity is reduced to avoid energy waste.

[0097] Optionally, the raindrop information recognized by the camera, temperature and humidity data, vehicle speed, light intensity, air conditioner state, and other multi-dimensional information are fused to construct an environment perception small model. The wiper sensitivity is dynamically adjusted in combination with the vehicle speed and light intensity; the external circulation is automatically turned off in rainy days to prevent moisture from entering; the glass is preheated in advance in a rainy and humid environment.

[0098] Optionally, the vehicle front-view camera system has a fault self-diagnosis function, and can issue a calibration abnormality prompt when the image recognition and the sensor data are inconsistent.

[0099] Figure 4 is a structural schematic view of a vehicle front-view camera according to an embodiment of the present application, as Figure 4 shown, a camera is installed on the glass of the vehicle for image acquisition.

[0100] According to an embodiment of the present application, an embodiment of a vehicle control method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0101] According to another aspect of an embodiment of the present application, a vehicle control method is also provided, which is applied to the vehicle front-view camera system in any of the above. Figure 5 is a flowchart of a vehicle control method according to an embodiment of the present application, as shown in Figure 5 The vehicle control method comprises the following steps:

[0102] Step S501, collecting images of a preset area;

[0103] Step S502, collecting temperature information and humidity information of the vehicle;

[0104] Step S503, performing raindrop detection on the images to obtain a raindrop detection result, and performing fog detection on the images to obtain a fog detection result;

[0105] Step S504, controlling operation of a rain wiper of the vehicle based on the raindrop detection result, and controlling operation of an air conditioner of the vehicle based on the temperature information, the humidity information, and the fog detection result.

[0106] Optionally, the images of the preset area are collected by an image collection component, to provide a visual information basis for subsequent image processing and function control.

[0107] Optionally, the temperature information and the humidity information of the vehicle are collected by an environment perception component, to provide necessary environment data for anti-fog control, which helps to predict and prevent the windshield from fogging.

[0108] Optionally, the image processing component performs raindrop detection on the images to obtain a raindrop detection result, and performs fog detection on the images to obtain a fog detection result. The raindrop detection includes image preprocessing, raindrop candidate region extraction, raindrop feature extraction, and classification confirmation, etc. The fog detection includes image contrast analysis, edge blur calculation, etc.

[0109] Optionally, the control component controls operation of a rain wiper of the vehicle based on the raindrop detection result, and controls operation of an air conditioner of the vehicle based on the temperature information, the humidity information, and the fog detection result. Based on the raindrop detection result, the operation mode of the rain wiper (such as stop, intermittent, low speed, high speed, and extreme speed, etc.) is dynamically adjusted to adapt to different rainfall intensities. Combined with the temperature, humidity, and fog detection results in the vehicle, the air conditioner mode (such as dehumidification and defogging) is automatically switched to prevent and eliminate the fog on the surface of the glass.

[0110] The vehicle control method is applied to the vehicle front-view camera system in any one of the above embodiments, and the vehicle control method comprises: collecting images of a preset area; collecting temperature information and humidity information of the vehicle; performing raindrop detection on the images to obtain a raindrop detection result, and performing fog detection on the images to obtain a fog detection result; controlling operation of a rain wiper of the vehicle based on the raindrop detection result, and controlling operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result. In the present application, real-time images of the front of the vehicle and the windshield can be comprehensively captured, which provides rich visual information for subsequent raindrop and fog detection, ensuring the accuracy and timeliness of detection. Further, temperature and humidity data inside and outside the vehicle are collected, which is crucial for predicting and monitoring fog formation, and can provide early warning to prevent problems. In addition, advanced algorithms are used to analyze image data, which can not only accurately identify raindrops but also effectively detect fog conditions. Through the identification of raindrop characteristics and the analysis of fog visual effects, the intelligent level of environmental perception is greatly improved. Finally, the operation of the rain wiper is intelligently regulated based on the image processing result, ensuring that raindrops on the windshield can be quickly removed on rainy days. At the same time, the air conditioning system is automatically adjusted based on the environmental perception data and the fog detection result, effectively preventing the windshield from fogging, and ensuring the clarity of the driving field of view in rainy and foggy weather. In summary, the present application realizes efficient detection of rain and fog to ensure the technical effect of driving safety, thereby solving the technical problems of independent and low-efficiency vehicle windshield raindrop detection and fog detection in related technologies, which further affects driving safety.

[0111] According to another aspect of the embodiments of the present application, a vehicle is also provided, comprising: the vehicle front-view camera system in any one of the above embodiments.

[0112] Optionally, the vehicle front-view camera system comprises: an image collection component, an environmental perception component, an image processing component and a control component; the image collection component is configured to collect images of a preset area; the environmental perception component is configured to collect temperature information and humidity information of the vehicle; the image processing component is configured to perform raindrop detection on the images to obtain a raindrop detection result, and perform fog detection on the images to obtain a fog detection result; and the control component is configured to control operation of a rain wiper of the vehicle based on the raindrop detection result, and control operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result.

[0113] Optionally, the image is subjected to raindrop detection to obtain a raindrop detection result, including: preprocessing the image to obtain a preprocessed image; performing pixel-level classification on the preprocessed image using a pre-trained classification model to obtain a candidate raindrop region; performing raindrop feature extraction on the candidate raindrop region to obtain a raindrop feature extraction result; and performing analysis on the raindrop feature extraction result using a pre-trained raindrop recognition model to obtain a raindrop judgment result, wherein the raindrop judgment result is used to represent the probability that the candidate raindrop region is a real raindrop; in response to the raindrop judgment result being greater than a preset probability threshold, performing raindrop density calculation on the candidate raindrop region to obtain a calculation result; and determining the raindrop detection result based on the calculation result.

[0114] Optionally, the image is subjected to fog detection to obtain a fog detection result, including: performing contrast analysis and edge blurring analysis on the image to obtain the fog detection result.

[0115] Optionally, the raindrop detection result includes a first rain intensity level, a second rain intensity level, and a third rain intensity level, and the operation of the vehicle's wiper is controlled based on the raindrop detection result, including: in response to the raindrop detection result being the first rain intensity level, determining a wiper control strategy to stop the wiper operation; in response to the raindrop detection result being the second rain intensity level, determining the wiper control strategy to control the wiper to operate intermittently at a low speed; and in response to the raindrop detection result being the third rain intensity level, determining the wiper control strategy to control the wiper to operate continuously at a high speed, wherein the rainfall corresponding to the first rain intensity level is less than the rainfall corresponding to the second rain intensity level, and the rainfall corresponding to the second rain intensity level is less than the rainfall corresponding to the third rain intensity level; and the operation of the wiper is controlled based on the wiper control strategy.

[0116] Optionally, the operation of the vehicle's air conditioner is controlled based on the temperature information, the humidity information, and the fog detection result, including: in response to the fog detection result indicating that the image contains fog images, controlling the air conditioner to start a defogging mode; and in response to the fog detection result indicating that the image does not contain fog images and the vehicle glass is determined to have a risk of fogging based on the temperature information and the humidity information, controlling the air conditioner to start the defogging mode.

[0117] Optionally, the image acquisition component includes: an optical lens group, an infrared filter, an image sensor, and an image signal processor; the optical lens group is configured to acquire an optical image and project the optical image to the image sensor; the infrared filter is connected to the optical lens group and is configured to filter out infrared light from the optical image to obtain a processed image; the image sensor is connected to the infrared filter and is configured to convert the processed image into an electrical signal; and the image signal processor is connected to the image sensor and is configured to perform signal processing on the electrical signal to obtain an image of a preset region.

[0118] Optionally, the vehicle front-view camera system further comprises a thermal management component, an upgrade component and a communication and power supply component; the thermal management component is configured to thermally isolate the vehicle front-view camera system; the upgrade component is configured to upgrade the vehicle front-view camera system; and the communication and power supply component is configured to supply power to the vehicle front-view camera system and establish a communication connection between the vehicle front-view camera system and the vehicle control unit.

[0119] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which comprises a stored executable program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform the vehicle control method when the executable program is executed.

[0120] Optionally, the computer readable storage medium controls the device where the computer readable storage medium is located to perform the following steps when the executable program is executed:

[0121] Step S501: collect an image of a preset area;

[0122] Step S502: collect temperature information and humidity information of the vehicle;

[0123] Step S503: perform raindrop detection on the image to obtain a raindrop detection result, and perform fog detection on the image to obtain a fog detection result;

[0124] Step S504: control operation of a rain wiper of the vehicle based on the raindrop detection result, and control operation of an air conditioner of the vehicle based on the temperature information, the humidity information and the fog detection result.

[0125] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0126] In some embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The system embodiments described above are only schematic, for example, the division of units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0127] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0128] In addition, each functional unit in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0129] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, the essential part or the whole or part of the contribution to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and various program code storage media.

[0130] The above is only the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A vehicle forward looking camera system, characterized by, The method comprises the following steps: An image acquisition component, an environment perception component, an image processing component, and a control component are included; The image acquisition component is configured to acquire images of a preset area; The environment perception component is configured to acquire temperature information and humidity information of the vehicle; The image processing component is configured to perform raindrop detection on the images to obtain a raindrop detection result, and perform fog detection on the images to obtain a fog detection result; The control component is configured to control operation of a rain wiper of the vehicle based on the raindrop detection result, and control operation of an air conditioner of the vehicle based on the temperature information, the humidity information, and the fog detection result.

2. The vehicle forward looking camera system of claim 1, wherein, The raindrop detection on the images to obtain the raindrop detection result comprises the following steps: Preprocessing is performed on the images to obtain preprocessed images; A pre-trained classification model is used to perform pixel-level classification on the preprocessed images to obtain candidate raindrop regions; Raindrop feature extraction is performed on the candidate raindrop regions to obtain raindrop feature extraction results; A pre-trained raindrop recognition model is used to analyze the raindrop feature extraction results to obtain raindrop judgment results, wherein the raindrop judgment results represent probabilities that the candidate raindrop regions are real raindrops; In response to the raindrop judgment results being greater than a preset probability threshold, raindrop density calculation is performed on the candidate raindrop regions to obtain calculation results; The raindrop detection result is determined based on the calculation results.

3. The vehicle forward looking camera system of claim 1, wherein, The fog detection on the images to obtain the fog detection result comprises the following steps: Contrast analysis and edge blurring analysis are performed on the images to obtain the fog detection result.

4. The vehicle forward looking camera system of claim 2, wherein, The raindrop detection result comprises a first rain intensity level, a second rain intensity level, and a third rain intensity level. Based on the raindrop detection result, the operation of the rain wiper of the vehicle is controlled as follows: In response to the raindrop detection result being the first rain intensity level, a rain wiper control strategy is determined to be stopping the operation of the rain wiper; in response to the raindrop detection result being the second rain intensity level, the rain wiper control strategy is determined to be controlling the rain wiper to operate intermittently at a low speed; and in response to the raindrop detection result being the third rain intensity level, the rain wiper control strategy is determined to be controlling the rain wiper to operate continuously at a high speed, wherein the rain amount corresponding to the first rain intensity level is less than the rain amount corresponding to the second rain intensity level, and the rain amount corresponding to the second rain intensity level is less than the rain amount corresponding to the third rain intensity level; The operation of the rain wiper is controlled based on the rain wiper control strategy.

5. The vehicle forward looking camera system of claim 3, wherein, Based on the temperature information, the humidity information, and the fog detection result, the operation of the air conditioner of the vehicle is controlled as follows: In response to the fog detection result indicating that the images contain fog images, the air conditioner is controlled to start a defogging mode; In response to the fog detection result indicating that the images do not contain fog images, and based on the temperature information and the humidity information, it is determined that the vehicle glass is at risk of fogging, the air conditioner is controlled to start the defogging mode.

6. The vehicle forward looking camera system of claim 1, wherein, The image acquisition component comprises an optical lens group, an infrared filter, an image sensor, and an image signal processor. The optical lens group is configured to collect an optical image and project the optical image to the image sensor. The infrared filter is connected with the optical lens group and configured to filter out infrared light from the optical image to obtain a processed image. The image sensor is connected with the infrared filter and configured to convert the processed image into an electrical signal. The image signal processor is connected with the image sensor and configured to perform signal processing on the electrical signal to obtain the image of the preset area.

7. The vehicle forward looking camera system of claim 1, wherein, The vehicle front-view camera system further comprises a thermal management component, an upgrade component, and a communication and power supply component. The thermal management component is configured to thermally isolate the vehicle front-view camera system. The upgrade component is configured to upgrade the vehicle front-view camera system. The communication and power supply component is configured to supply power to the vehicle front-view camera system and establish a communication connection between the vehicle front-view camera system and a vehicle control unit.

8. A vehicle control method characterized by, The vehicle control method is applied to the vehicle front-view camera system of any one of claims 1 to 7, and the vehicle control method comprises: collecting an image of a preset area; collecting temperature information and humidity information of the vehicle; performing raindrop detection on the image to obtain a raindrop detection result, and performing fog detection on the image to obtain a fog detection result; controlling the operation of a rain wiper of the vehicle based on the raindrop detection result, and controlling the operation of an air conditioner of the vehicle based on the temperature information, the humidity information, and the fog detection result.

9. A vehicle characterized by comprising: comprise: The vehicle front-view camera system of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to perform the vehicle control method of claim 8.

Images