Multifunctional and multi-sensing intelligent automobile lamp and using method thereof

Abstract

The invention relates to a multifunctional and multi-sensing intelligent automobile lamp and a using method thereof. The multifunctional and multi-sensing intelligent automobile lamp comprises an automobile lighting module, a wide-angle camera module, a long-focus / zoom camera module, a thermal infrared camera module, a laser radar module, an ultrasonic / millimeter wave radar module, a red, blue and green laser projection module and the like. After complex environment information sensed and acquired by wide-angle and long-focus / zoom binocular vision, thermal infrared binocular vision, laser radar, ultrasonic / millimeter wave radar and the like is fused and reconstructed, reliable data support is provided for an automobile intelligent / auxiliary driving system. And the red, green and blue laser projection module is utilized to project intelligent driving information such as vehicle states, navigation maps, safety early warning and danger warning to visible road surfaces of traffic participants in different colors according to importance degrees, so that driving fatigue is relieved, and driving safety and driving experience are improved.

Description

Technical Field

[0001] The present invention relates to a multifunctional and multi-perceptive intelligent automobile and a use method thereof, and belongs to the field of automobile intelligent lighting and automatic driving environment perception. Technical Background

[0002] In 2014, the Society of Automotive Engineers (SAE) of the United States divided autonomous driving into six levels, L0~L5, according to the degree of intelligence of automobiles. At present, the global intelligent automobiles are generally at the L2 level, that is, the advanced assisted driving stage. The key and difficulty that restricts the realization of more advanced autonomous driving lies in: how to use sensors to perform high-precision, high-efficiency and high-reliability scene perception of the 360° driving environment, including the detection and recognition of key information such as obstacle types, road markings, pedestrians, vehicles and traffic signs. At present, the environmental perception of autonomous driving mainly adopts the "weak perception + super intelligence" technology route represented by Tesla and Xiaopeng Motors, and the "strong perception + strong intelligence" technology route represented by most companies such as Google Waymo and Baidu Apollo. Among them, the "weak perception + super intelligence" route mainly relies on cameras (monocular / binocular) and deep learning algorithms to achieve environmental perception, while the "strong perception + strong intelligence" route is to add sensors such as lidar on the basis of cameras to improve environmental perception capabilities, but both typical technical routes rely on the image perception technology of monocular / binocular cameras. Compared with visual recognition technology, although laser radar has obvious advantages in detection accuracy, detection distance and reliability, it also has many disadvantages such as high cost, technical difficulty, great influence by weather, and easy signal crosstalk between different vehicle radars. It is also unable to recognize traffic police gestures, pedestrian movements, traffic signs and signal lights, etc. Therefore, for a considerable period of time in the future, the mainstream technical route of "pure vision" or "vision as the main and radar as the auxiliary" will be generally adopted for autonomous driving of cars.

[0003] Regardless of what kind of intelligent driving solution is adopted in the future development, image perception and recognition technology based on monocular vision, binocular vision, multi-eye vision, panoramic vision and mixed vision will surely become a breakthrough direction and development trend to improve the degree of automobile intelligence. Among them, monocular vision first recognizes the target through image matching, and then estimates the distance by the size change of the target in the image. It not only requires accurate identification of the target before estimating the distance, but also requires the establishment and maintenance of a huge sample database containing all the features of various types of targets to be identified. The recognition efficiency and accuracy are also difficult to guarantee. Binocular vision is similar to the human eye. It mainly calculates the distance of the object based on the parallax between the left and right cameras. It can not only obtain more accurate obstacle depth information than monocular vision, but also provide denser point cloud data than lidar. Given that binocular vision technology can well meet the increasingly high performance requirements of the accuracy, robustness and density of the automobile perception system in the complex environment of the real world and high-level intelligence, it will surely become the only way for the iterative development and breakthrough of automobile assisted driving to intelligent driving in the future. However, binocular stereo vision technology still has many technical difficulties or bottlenecks that restrict its development and application in high-level intelligent driving. Therefore, the present invention proposes a multifunctional and multi-sensory automobile intelligent headlight and a method of using the same to further solve the current technical difficulties that restrict the development and application of binocular vision technology. Summary of the invention

[0004] The present invention discloses a multifunctional and multi-perceptual automotive intelligent lamp and a method for using the same, so as to effectively solve many technical difficulties and bottlenecks that restrict the development and application of existing binocular vision technology, such as short depth perception distance and low precision, difficulty in automatically calibrating multi-degree-of-freedom deviations of vision modules under no true value conditions, etc.

[0005] A multifunctional and multi-sensing intelligent automobile lamp, the intelligent automobile headlight comprises a lighting module, an environmental sensing module and a projection display module, wherein the lighting module comprises turn signal and fog light bulbs and reflectors for the turn signal and fog light bulbs, reflectors for high beam and low beam and bulbs for the high beam and low beam, reflectors for daytime running lights and width indicator lights and bulbs for the daytime running lights and width indicator lights, which are installed and fixed at different positions in the housing of the intelligent automobile lamp.

[0006] Furthermore, the four wide-angle camera modules in the front and rear lamps of the automobile are respectively fixed on the inner side of the lamp housing by four wide-angle camera fastening screws, and the two groups of wide-angle cameras of the left and right front headlights and the left and right rear taillights respectively form a front binocular vision system and a rear binocular vision system; at the same time, the image information collected by the two wide-angle camera modules of the front / rear binocular vision system is respectively processed by image stitching such as registration transformation and fusion reconstruction to generate two groups of front and rear images with a large field of view, which effectively reduces the visual blind spot and is directly used for driving assistance systems such as vehicle driving records, 360-degree imaging, sentry monitoring, parking / reversing assistance, deviation / collision warning, etc.

[0007] Furthermore, in order to increase the lateral distance between the left and right telephoto / zoom cameras, the two telephoto / zoom camera modules of the left and right headlights of the car are respectively installed and fixed on the outside of the headlight housing by four telephoto / zoom camera fastening screws to construct a telephoto binocular vision system for the car's front view, thereby solving the problem of insufficient long-distance target detection and recognition capabilities of the wide-angle binocular vision system; at the same time, the two telephoto cameras can collect and obtain image information of the area farther ahead of the car, and after completing the image stitching processing, they can be super-resolution restored and reconstructed with the image information collected by the two wide-angle cameras to further enhance the functions and performance of the car's driving assistance system.

[0008] Furthermore, the wide-angle camera and telephoto / zoom camera of the car lamp can quickly and accurately obtain the direction and distance of the corresponding car by identifying and extracting the car license plate with standard length and width dimensions (length × width: 440mm × 140mm or 440mm × 220mm) in the surrounding environment, and calculating the position, shape and number of pixels occupied by the standard license plate in different wide-angle and telephoto / zoom images after transformation and correction, thereby realizing efficient and accurate "monocular" visual recognition of car targets in different spatial ranges; at the same time, by replacing the traditional black and white checkerboard calibration plate with a standard-sized car license plate, each camera module of the wide-angle and telephoto / zoom binocular vision recognition system can be automatically calibrated and quickly calibrated with multiple degrees of freedom deviations anytime and anywhere under any driving state, effectively solving the problem of reduced target detection accuracy and consistency of the two cameras of the binocular vision system due to driving bumps (causing changes in the position and angle between the two cameras) and changes in the focal length of the lens.

[0009] Furthermore, the thermal infrared camera modules of the left and right headlights of the car are respectively fixed to the middle of the headlight housing by four thermal infrared camera fastening screws. The temperature difference that inevitably exists between vehicles and pedestrians and the surrounding environment is used to collect and obtain thermal infrared image information within a certain angle range in the driving direction. At the same time, the pixel enhancement and fusion reconstruction of thermal infrared and visible light binocular vision image information are used to solve the low pixel resolution problem commonly existing in thermal infrared camera modules. Pedestrians, animals and other targets that are difficult to directly observe with the naked eye due to obstacles such as trees, grass and fences in the green belts on both sides of the road are effectively detected and warned. The high-definition images fused by visible light and thermal infrared can also greatly enhance the driving safety of the car in various low visibility conditions such as rain and fog, at night and against light.

[0010] Furthermore, the headlights can be optionally equipped with laser radar modules and millimeter wave / ultrasonic radar modules according to the configuration requirements of the vehicle, and are installed and fixed on the middle upper edge of the headlight housing by four laser radar fastening screws and four ultrasonic / millimeter wave radar fastening screws respectively; the laser radar module and ultrasonic / millimeter wave radar module are used to collect and acquire the spatial position information of the vehicle's surrounding environment, and multi-dimensional data fusion is performed with the spatial image information collected and acquired by the visible light and thermal infrared binocular vision system, ultimately realizing the large-scale and high-precision perception and recognition of the natural environment and traffic participants around the vehicle during the automatic driving process, including the perception and recognition of pedestrians and vehicles, light weather, lane markings, traffic signs, road obstacles, slope curves and special groups within and beyond the line of sight.

[0011] Furthermore, integrated red, green and blue laser projection modules are installed in the left and right headlights and left and right rear taillights of the vehicle, and fixed to the outer lower edge of the lamp housing by five three-color laser projection fastening screws; wide-angle and telephoto / zoom binocular vision systems, thermal infrared binocular vision systems, lidar and ultrasonic / millimeter wave radars are used to perform multi-feature, multi-target and multi-dimensional perception and recognition of the driving environment, and the relevant environmental perception information is subjected to data fusion, reconstruction, splicing and super-resolution processing, and then the red, green and blue laser projection module is used to project the vehicle status, environmental perception, safety warning and other information in different colors according to the importance and safety level onto the road surface within the significant visible range of the corresponding traffic participants, effectively reducing driving fatigue, improving driving safety and enhancing the driving experience.

[0012] Furthermore, the two cameras of the visible light binocular vision system are respectively installed in the left and right headlights and the left and right rear taillights of the car, which effectively increases the lateral distance between the two cameras. It can not only improve the detection and recognition distance and accuracy of the binocular vision system, but also increase its field of view and reduce the visual blind spot through image stitching of the left and right cameras; by constructing a wide-angle binocular vision system and a telephoto / zoom binocular vision system, the perception and recognition of the driving environment within different viewing distances can be realized. After fusion and reconstruction with the thermal infrared binocular vision system, special targets such as pedestrians and animals can be effectively detected under visible light or invisible to the human eye, enhancing driving safety under low visibility conditions such as rain, fog and night; a car license plate with a standard size is used instead of a black and white chessboard as a known recognition target, and the spatial position and distance of the corresponding car are determined by extracting and calculating the position, shape and number of pixels of the standard license plate in a single camera image, and at the same time, each camera of the binocular vision system is quickly calibrated and automatically calibrated under various driving conditions.

[0013] Furthermore, the wide-angle camera module, telephoto / zoom camera module, thermal infrared camera module, red, blue, and green laser projection module, laser radar module and ultrasonic / millimeter wave radar module that have been selected according to the needs are all integrated and installed inside the automobile lamps. For the design and manufacture of new cars and the upgrading and transformation of old cars, it will not cause damage to the mechanical structure and aerodynamic shape of the car itself, and it greatly simplifies the debugging, installation, repair and maintenance process of the intelligent car lights. Different from the traditional automobile binocular vision system, the present invention arranges the thermal infrared-visible light binocular vision system outside the car cab, which can effectively avoid the visual occlusion of the car A-pillar and rearview mirror and other components, and greatly reduce the visual blind spot of the car visual perception system. Beneficial Effects

[0014] Firstly, two sets of wide-angle binocular vision systems with large baseline lengths are constructed using four wide-angle camera modules in the left and right headlights and the left and right taillights of the car to realize perception and recognition of the environment around the car. The image information collected by the four wide-angle cameras is fused and spliced ​​to generate a panoramic image with a larger field of view, thereby reducing the visual blind spots of car driving. At the same time, two telephoto / zoom camera modules in the left and right headlights of the car are used to construct a telephoto / zoom binocular vision system with a larger baseline length to realize perception and recognition of the medium and long-distance environment in front of the car. The image information obtained by the telephoto / zoom camera can be processed with the wide-angle image information through data fusion, image splicing and super-resolution reconstruction, thereby improving the perception range and recognition accuracy of the visible light wide-angle-telephoto / zoom binocular vision system. Then, the thermal infrared camera modules in the left and right lamps of the car are used to perceive and identify the environment around the car, and the visible light-thermal infrared image is fused with the wide-angle and telephoto / zoom images to detect special targets such as pedestrians and animals that are difficult to identify with visible light or human eyes, thereby enhancing driving safety in low visibility conditions such as rain, fog, night and backlight. At the same time, the laser radar and ultrasonic / millimeter wave radar in the lamp are used to perceive and identify the information of the environment around the car, and the information is spliced, fused and super-reconstructed with the perception information of the visible light and thermal infrared binocular vision system to achieve multi-target, multi-level, multi-feature and multi-dimensional environmental perception and target detection. In addition, a car license plate with a standard size is used instead of a black and white checkerboard calibration plate as a known target. By extracting and solving the position, shape and number of pixels occupied by the standard license plate in a single wide-angle or telephoto / zoom image, the "monocular" visual perception and recognition of the corresponding vehicle parameters such as direction, distance and speed can be completed efficiently and accurately, and the automatic calibration and deviation calibration of the wide-angle and telephoto / zoom binocular vision system can be quickly completed under any driving conditions. Finally, after the complex environmental information obtained by wide-angle and telephoto / zoom binocular vision, thermal infrared binocular vision, lidar and ultrasonic / millimeter wave radar is integrated and reconstructed, reliable data support is provided for the vehicle's intelligent / assisted driving system. In addition, red, green and blue laser projection modules are used to project intelligent driving information such as vehicle status, navigation maps, safety warnings and hazard warnings in different colors according to their importance onto the visible road surface of traffic participants, so as to reduce driving fatigue, improve driving safety and enhance the driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the overall structure of a multifunctional and multi-sensing intelligent automobile lamp; Figure 2 The schematic diagram of the image recognition and automatic calibration of a "monocular" camera based on a standard-sized car license plate; Figure 3 A schematic diagram of a method for using three-color laser projection of a multifunctional and multi-sensing intelligent automobile lamp; Figure 4A schematic diagram of pedestrian-vehicle blind spot detection and hazard warning for a multifunctional and multi-sensing intelligent automotive lamp; Description and abstract: Smart car lamp housing 1, turn signal / fog lamp bulb 2, turn signal / fog lamp bulb reflector 3, telephoto / zoom camera module 4, telephoto / zoom camera fastening screws 5 (4 pcs), three-color laser projection module fastening screws 6 (5 pcs), red, blue and green three-color laser projection module 7, high beam / low beam reflector 8, high beam / low beam bulb 9, laser radar module 10, daytime running light / width light reflector 11, daytime running light / width light bulb 12, wide-angle camera module 13, wide-angle camera fastening screws 14 (4 pcs), thermal infrared camera fastening screws 15 (4 pcs), thermal infrared camera module 16, laser radar fastening screws 17 (4 pcs), ultrasonic / millimeter wave radar module 18, ultrasonic / millimeter wave radar fastening screws 18 (4 pcs). DETAILED DESCRIPTION

[0016] In order to make the purpose, scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are embodiments of a part of the present invention, not all of the embodiments. The following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] A multifunctional and multi-sensing automotive intelligent lamp and its use method are used to effectively solve the technical difficulties and bottlenecks that restrict the development and application of existing binocular vision technology, such as short depth perception distance and low accuracy, and difficulty in automatically calibrating the multi-degree-of-freedom deviation of the visual module under the condition of no true value. The automotive intelligent lamp mainly includes high / low beam lights, daytime running lights, turn / fog lights, wide-angle camera modules, telephoto / zoom camera modules, thermal infrared camera modules, laser projection modules, laser radars, millimeter-wave radars, and ultrasonic radars. First, the baseline length of the existing on-board binocular vision module (i.e., the distance between the central optical axes of the left and right cameras) is generally only between 0.2 and 0.6 meters. When the target distance is far, the parallax of the left and right images will be less than one pixel and the depth cannot be accurately estimated (i.e., baseline constraint). Therefore, the present invention integrates the left and right cameras of the binocular vision into the headlights and taillights on the left and right sides of the car respectively, and increases the binocular vision baseline length to between 1.5 meters and 2.0 meters, which greatly improves the maximum distance and evaluation accuracy of target detection. Secondly, there is a contradictory relationship between the target recognition range and detection distance of the existing binocular vision module. For example, the wide-angle camera module can obtain as much environmental information as possible around the vehicle, but it is difficult to accurately detect target objects at a long distance in front and behind the vehicle. Therefore, the present invention will respectively construct a binocular stereo vision system for close distance and long distance through a wide-angle camera module and a telephoto / zoom camera module to achieve an effective balance between ultra-long detection distance and environmental perception within an ultra-wide field of view. Thirdly, the existing automobile binocular vision system is very likely to cause the position and angle between the two cameras to change due to factors such as driving bumps and focal length changes. It is difficult to complete the automatic calibration of the multi-degree-of-freedom deviation of the binocular camera module without a true value reference, which reduces the accuracy and consistency of target detection. Therefore, the present invention proposes an automatic calibration and rapid calibration method for the multi-degree-of-freedom deviation of the binocular camera module based on standard size vehicle license plate image recognition to effectively ensure that its point cloud density and detection accuracy meet the performance requirements of driving environment perception. Then, in order to enhance the adaptability of the binocular stereo vision system of the intelligent car light to complex driving environments such as night and rain and fog, the present invention combines infrared thermal imaging technology to build a binocular stereo vision system that fusions thermal infrared and visible light. The infrared and visible light image fusion technology is used to solve the problems of low resolution and little color information of thermal infrared images, and poor imaging quality of visible light images under conditions such as backlight, night, and rainy and foggy weather.Finally, the thermal infrared and visible light images acquired by the smart headlights involved in the present invention can also provide the necessary image information and data for functions such as the car's 360-degree panoramic image, transparent chassis, reversing image, sentry mode and driving recorder, and then combine image processing technologies such as data fusion, splicing synthesis and super-resolution reconstruction to improve the complex environment perception accuracy, range and dimension of the smart headlight visual system, reduce the visual blind spots of car driving and operation, and at the same time, the red, green and blue laser projection lamp group of the smart headlights can project and display relevant information such as positioning navigation, driving assistance, safety warning and danger alarm in the road area that is significantly visible to drivers and traffic participants such as pedestrians / vehicles, greatly improving the convenience, controllability, safety and intelligence of car driving. Combined with the accompanying drawings, the specific contents of the present invention include the following aspects: (a) A multifunctional and multi-sensing intelligent automotive lamp and a method of using the same, such as Figure 1 As shown, the intelligent automobile headlight mainly includes a lighting module, an environmental perception module, and a projection display module. Among them, the lighting module includes a reflector 8 and a bulb 9 for a high beam or a low beam, a reflector 3 and a bulb 2 for a turn signal or a fog lamp, and a reflector 11 and a bulb 12 for a daytime running light or a width lamp. Its main structure and use method are basically the same as those of conventional automobile lamps, but it can automatically adjust and control the illumination intensity, height direction, and near-far switching of the automobile light according to the multi-feature and multi-dimensional environmental perception information collected and acquired by the wide-angle camera module 13, the telephoto / zoom camera module 4, the thermal infrared camera module 16, the laser radar module 10, and the ultrasonic / millimeter wave radar module 18.

[0018] (b) If Figure 1 As shown, the environment perception module mainly includes a wide-angle camera module 13, a telephoto / zoom camera module 4, a thermal infrared camera module 16, a laser radar module 10 and a millimeter wave radar module 18. Among them, the wide-angle camera module 13 is fixed on the inner edge of the lamp body of the car lamp by four fastening screws 14 to reduce the visual perception blind spot between the left and right headlight cameras of the car; the telephoto / zoom camera module 4 is fixed on the outer lamp body of the car lamp by four fastening screws to increase the lateral distance between the left and right headlight cameras of the car, thereby greatly improving the binocular visual perception distance and perception accuracy of the telephoto / zoom camera module of the car lamp; as optional environmental auxiliary perception means, the thermal infrared camera module 16, the laser radar module 10 and the millimeter wave radar module 18 are also fixed in the independent space of the middle lamp body of the car lamp by three sets of fastening screws 15, 17 and 19 respectively, so as to further increase the perception range and accuracy of the front and rear headlights of the smart car to the surrounding driving environment.

[0019] (c) If Figure 1 , Figure 3 and Figure 4As shown, the red, blue and green laser projection module 7 is fixed inside the outer lamp body of the headlight by five fastening screws 6, and the multi-feature and multi-dimensional environmental perception information collected by the wide-angle camera module 13, the telephoto / zoom camera module 4, the thermal infrared camera module 16, the laser radar module 10 and the ultrasonic / millimeter wave radar module 18 generate driving parameters, navigation information, safety warnings, danger alarms and other key information through data fusion and reconstruction splicing, and then the above information is displayed on the road surface within the significant visible range of the traffic participants, thereby effectively reducing various traffic accidents caused by complex factors such as vehicle driving failures, sudden road accidents, human and vehicle visual blind spots and improper human control.

[0020] (d) If Figure 2 As shown, each wide-angle camera module 13 and telephoto / zoom camera module 4 in the front and rear headlights of the car first collects images of the surrounding environment of the car, and then identifies, extracts and corrects the measurement of the car license plate with standard length and width in the surrounding environment (length × width: 440mm × 140mm or 440mm × 220mm), and finally accurately and quickly realizes "monocular" visual recognition of the distance and position of the surrounding environment according to the number of points and position shape of the pixel area occupied by the standard license plate in the wide-angle image and the telephoto image; at the same time, using the above-mentioned "monocular" visual recognition based on the standard size license plate, the multi-degree-of-freedom deviation of each optical camera module in the wide-angle and telephoto / zoom binocular vision recognition system can be automatically calibrated and quickly calibrated anytime and anywhere in the driving state, so as to effectively solve the problem of reduced accuracy and consistency of environmental perception and target detection of each camera in the wide-angle and telephoto / zoom binocular vision system due to factors such as driving bumps (which may cause changes in the relative position and angle between the two cameras) and changes in the focal length of the lens.

[0021] (e) If Figure 1As shown, the image information of the driving environment is obtained by the wide-angle camera module 13 and the telephoto camera module 4 of the left and right groups respectively, and then the left and right camera images with partially overlapping areas are spliced ​​through image preprocessing, registration transformation and fusion reconstruction, so as to increase the horizontal field of view of the binocular vision camera module and reduce the blind spot of the vehicle's viewing angle; then, the large field angle synthetic panoramic image after image fusion and splicing processing can be directly used for the car's driving recorder, 360 imaging, sentry monitoring, automatic parking, driving / reversing assistance, blind spot monitoring and collision / bias It provides more reliable data support for intelligent auxiliary systems or functions such as clutch pretensioning; finally, the left and right thermal infrared camera modules 16 in the car's headlights collect thermal infrared image information of the area ahead of the car, and solve the low pixel problem commonly existing in existing thermal infrared camera modules through pixel enhancement and image fusion of thermal infrared binocular vision and visible light binocular vision, thereby enhancing the ability to perceive and predict pedestrians and animals hidden in the trees and grass in the green belts on both sides of the road, greatly improving the safety of the car in low visibility and human eye interference conditions such as rain and fog, night and backlight.

Claims

1. A multifunctional and multi-sensing intelligent automobile lamp, characterized in that: The smart car headlight includes a lighting module, an environment perception module and a projection display module, wherein the lighting module includes turn signal and fog light bulbs and reflectors for the turn signal and fog light bulbs, reflectors for high beam and low beam and bulbs for the high beam and low beam, reflectors for daytime running lights and width lights and bulbs for the daytime running lights and width lights, which are installed and fixed at different positions in the housing of the smart car lamp; the environment perception module includes a wide-angle camera module, a telephoto / zoom camera module, a thermal infrared camera module, a lidar module and a millimeter-wave radar module; the projection display module includes a red, green and blue three-color laser projection module.

2. A method for using a multifunctional and multi-sensing intelligent automobile lamp according to claim 1, characterized in that: The four wide-angle camera modules in the front and rear lamps of the car are respectively fixed on the inner side of the lamp housing by four wide-angle camera fastening screws, and the two groups of wide-angle cameras of the left and right front headlights and the left and right rear taillights respectively form a front binocular vision system and a rear binocular vision system; at the same time, the image information collected by the two wide-angle camera modules of the front and rear binocular vision systems are respectively processed by image stitching such as registration transformation and fusion reconstruction to generate two groups of front and rear images with a large field of view, which are directly used for car driving records, 360-degree imaging, sentry monitoring, parking / reversing assistance, and deviation / collision warning driving assistance systems.

3. The method for using a multifunctional and multi-sensing intelligent automobile lamp according to claim 2, characterized in that: The two telephoto / zoom camera modules of the left and right headlights of the car are respectively installed and fixed on the outside of the lamp housing through four telephoto / zoom camera fastening screws; at the same time, the two telephoto cameras can collect image information of the area far ahead of the car. After completing the image stitching processing, they can be used for super-resolution restoration and reconstruction with the image information collected by the two wide-angle cameras.

4. The method for using a multifunctional and multi-sensing intelligent automobile lamp according to claim 1, characterized in that: The wide-angle camera and telephoto / zoom camera of the car lamp can identify and extract the car license plates with standard length and width in the surrounding environment. The length and width of the car license plates are 440mm×140mm or 440mm×220mm. After transformation and correction, the position, shape and number of pixels occupied by the standard license plates in different wide-angle and telephoto / zoom images are calculated respectively, so as to quickly and accurately obtain the direction and distance of the corresponding car, thereby realizing efficient and accurate "monocular" visual recognition of car targets in different spatial ranges. At the same time, the standard-sized car license plates are used instead of the traditional black and white checkerboard calibration plates, and the multi-degree-of-freedom deviation automatic calibration and rapid calibration can be performed on each camera module of the wide-angle and telephoto / zoom binocular vision recognition system anytime and anywhere under any driving state, effectively solving the problem of reduced target detection accuracy and consistency of the two cameras of the binocular vision system due to driving bumps and changes in lens focal length.

5. The method for using a multifunctional and multi-sensing intelligent automobile lamp according to claim 1, characterized in that: The thermal infrared camera modules of the left and right headlights of the car are respectively installed and fixed on the middle of the headlight housing by four thermal infrared camera fastening screws. The inevitable temperature difference between vehicles and pedestrians and the surrounding environment is used to collect and obtain thermal infrared image information within a certain angle range in the driving direction. At the same time, the pixel enhancement and fusion reconstruction of thermal infrared and visible light binocular vision image information are used to solve the low pixel resolution problem commonly existing in thermal infrared camera modules. It can effectively detect and warn pedestrians, animals and other targets that are difficult to observe directly with the naked eye due to obstacles such as trees, grass and fences in the green belts on both sides of the road. The high-definition images fused by visible light and thermal infrared can also greatly enhance the driving safety of the car in various low-visibility conditions such as rain and fog, at night and against light.

6. A method for using a multifunctional and multi-sensing intelligent automobile lamp according to claims 2 to 5, characterized in that: The headlights can be equipped with optional laser radar modules and millimeter wave / ultrasonic radar modules according to the vehicle configuration requirements, and are installed and fixed on the middle upper edge of the headlight housing by four laser radar fastening screws and four ultrasonic / millimeter wave radar fastening screws respectively; the laser radar module and ultrasonic / millimeter wave radar module are used to collect and acquire the spatial position information of the vehicle's surrounding environment, and multi-dimensional data fusion is performed with the spatial image information acquired by the visible light and thermal infrared binocular vision system, ultimately achieving large-scale and high-precision perception and recognition of the natural environment and traffic participants around the vehicle during the autonomous driving process, including the perception and recognition of pedestrians and vehicles, light weather, lane markings, traffic signs, road obstacles, slope curves and special groups within and beyond the line of sight.

7. The method for using a multifunctional and multi-sensing intelligent automobile lamp according to claim 1, characterized in that: Integrated red, green and blue laser projection modules are installed in the left and right headlights and left and right rear taillights of the car, and fixed to the outer lower edge of the lamp housing by five three-color laser projection fastening screws; wide-angle and telephoto / zoom binocular vision systems, thermal infrared binocular vision systems, lidar and ultrasonic / millimeter wave radars are used to perceive and identify the driving environment in a multi-feature, multi-target and multi-dimensional manner. After data fusion, reconstruction, splicing and super-resolution processing of relevant environmental perception information, the red, green and blue laser projection module is used to project information such as vehicle status, environmental perception and safety warning in different colors according to the importance and safety level onto the road surface within the significant visible range of the corresponding traffic participants, effectively reducing driving fatigue, improving driving safety and enhancing the driving experience.

8. A method for using a multifunctional and multi-sensing intelligent automobile lamp according to claims 2 to 7, characterized in that: The two cameras of the visible light binocular vision system are installed in the left and right headlights and the left and right taillights of the car respectively, which effectively increases the lateral distance between the two cameras. It can not only improve the detection and recognition distance and accuracy of the binocular vision system, but also increase its field of view and reduce the visual blind spot by stitching the images of the left and right cameras. By constructing a wide-angle binocular vision system and a telephoto / zoom binocular vision system, the perception and recognition of the driving environment within different viewing distances can be realized. After fusion and reconstruction with the thermal infrared binocular vision system, special targets such as pedestrians and animals can be effectively detected under visible light or invisible to the human eye, enhancing driving safety under low visibility conditions such as rain, fog and night. A car license plate with a standard size is used instead of a black and white chessboard as a known recognition target. The spatial position and distance of the corresponding car are determined by extracting and calculating the position, shape and number of pixels of the standard license plate in a single camera image, and each camera of the binocular vision system is quickly calibrated and automatically calibrated under various driving conditions.

9. A method for using a multifunctional and multi-sensing intelligent automobile lamp according to claims 2 to 8, characterized in that: The wide-angle camera module, telephoto / zoom camera module, thermal infrared camera module, red, blue and green laser projection module, laser radar module selected according to needs and ultrasonic / millimeter wave radar module are all integrated and installed inside the car lamp.

Images