Split type passenger vehicle passive night vision auxiliary driving system, method and application

Abstract

The application provides a split passenger vehicle passive night vision auxiliary driving system, method and application. The method comprises the following steps: converting the received scene infrared radiation signal of the working state of the vehicle into an optical signal, receiving an analog image according to the infrared detector image output timing sequence, and transmitting the converted original digital infrared image in real time after analog-digital conversion; after the obtained original infrared image of the working state of the vehicle is preprocessed, target recognition is performed, the target in the scene is automatically recognized, a detection frame is superimposed on the detected target in the image, and the image with the superimposed detection frame is displayed. The passive night vision camera can perform infrared imaging on the scene in front of the driver, can ensure clear imaging in the night, rain, snow and fog weather, can recognize pedestrians, two-wheeled vehicle and three-wheeled vehicle riders, vehicles and other targets by using the passive night vision controller, and can automatically prompt the driver of the collision risk, thereby improving the safety of the driver driving the vehicle in the low-illumination environment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of vehicle-mounted night vision auxiliary driving, and particularly discloses a split type passenger vehicle passive night vision auxiliary driving system, method and application. BACKGROUND

[0002] Traffic accidents account for a high proportion of abnormal deaths worldwide, and many serious traffic accidents occur at night. In low-visibility conditions such as night, foggy weather, rainy and snowy weather, the driver's field of vision is greatly limited, which is the main cause of traffic accidents.

[0003] Infrared thermal imaging is not affected by electromagnetic interference, has a long action distance, can image in the absence of light, is not affected by adverse weather such as rain, snow and fog, and can achieve clear imaging 24 hours a day. The night vision auxiliary driving system based on infrared thermal imaging can significantly improve the environmental perception ability of the driver and will become an important part of the L2-L5 level automatic driving system.

[0004] Through the above analysis, the problems and defects of the prior art are:

[0005] (1) The night vision auxiliary driving systems on the market generally only have imaging functions, only collect infrared images into the vehicle central platform, and display them on the liquid crystal screen, lack the ability to identify pedestrians, bicycles, motorcycle riders and cars, and cannot warn potential collision risks according to speed and distance information.

[0006] (2) In the prior art, the passive night vision controller cannot identify pedestrians, two-wheeled vehicles, cars and other targets while processing infrared information, resulting in a delay in infrared image processing and target identification, and the driver cannot be automatically prompted about collision risks, which makes the safety of driving a vehicle in low-illumination environments poor.(3) The prior art lacks embedded implementation of a deep learning target identification framework, and generally uses traditional machine learning frameworks such as Adaboost and SVM, which have low recognition rates and high false positives. SUMMARY

[0007] In view of the above defects or deficiencies in the prior art, the present application aims to provide a split type passenger vehicle passive night vision auxiliary driving system, method and application, particularly a split type passive night vision auxiliary driving system based on infrared thermal imaging.

[0008] In a first aspect, a split type passenger vehicle passive night vision auxiliary driving method includes:

[0009] S1, the passive night vision camera converts the received vehicle working state scene infrared radiation signal into photoelectric signal, receives analog image according to the infrared detector image output timing, and transmits the converted original digital infrared image in real time after analog-digital conversion;

[0010] S2, the passive night vision controller identifies the target after the obtained vehicle working state original infrared image is preprocessed and enhanced in detail; automatically identify the target in the scene, and superimpose the detection frame on the detected target in the image, display the image superimposed with the detection frame; obtain the vehicle speed information, predict the distance of the detected pedestrian target, calculate the collision time combined with the vehicle speed information, and when the collision time is less than the set danger threshold, superimpose the danger warning icon in the image and give a collision danger prompt.

[0011] In step S1, the passive night vision camera converts the analog electrical signal converted by the detector circuit from the scene infrared radiation into a digital signal through AD sampling;

[0012] At the same time, the passive night vision camera uses the infrared information processing circuit to respond to the IIC control signal sent by the passive night vision controller to configure the bias voltage of the infrared detector, control the shutter and heating ring, and generate a serial digital image sending timing. The digital image and the correction parameter and the bad cell list are sent to the passive night vision controller in a serial format for data processing.

[0013] In step S2, the target in the scene is automatically identified, and the detection frame is superimposed on the detected target in the image, which specifically includes: receiving vehicle CAN information, obtaining real-time working state of the vehicle, predicting the distance of the target, and evaluating the collision risk combined with the vehicle speed and distance prediction. The default danger threshold is set to 3.5s, and if the time to collision is less than a certain danger threshold according to the current vehicle speed and distance estimate, the alarm prompt icon is displayed by superimposition.

[0014] Further, the alarm prompt icon is displayed by superimposition, which includes:

[0015] After power-on, self-checking is performed by comparing the check code saved in the non-volatile memory at power-off with the data read from the non-volatile memory at power-off to determine whether the internal fixed data of the passive night vision camera and the file data of the passive night vision controller are correct;

[0016] If the data is abnormal, stop the driving assistance process and send an alarm signal to the vehicle CAN network;

[0017] If the data is normal, the driving assistance process is entered, and the shutter, heating ring and image data are detected in real time, if abnormal, the driving assistance process is stopped and an alarm signal is sent to the vehicle CAN network, if a collision danger is detected, an alarm signal is sent to the vehicle CAN network, and an alarm prompt icon is superimposed on the infrared display image video.

[0018] In another aspect, the present application provides a split passenger vehicle passive night vision auxiliary driving system, comprising:

[0019] The passive night vision camera is installed in the vehicle front grille and is used for digital signal conversion of received vehicle working state scene infrared radiation signals, receiving original infrared images based on the electrical signals according to the image output timing sequence of the infrared detector, and transmitting the converted original infrared images to the passive night vision controller in real time.

[0020] The infrared detector material is a long-wave uncooled microbolometer, which can receive infrared thermal radiation in the scene, convert the thermal signal into an electrical signal, and generate a grayscale image; the infrared information processing circuit processes the electrical signal converted by the infrared detector, converts the signal into serial data, and transmits it to the passive night vision controller.

[0021] The passive night vision controller is installed in the vehicle cab and is used for target recognition after infrared image preprocessing and infrared image detail enhancement of the acquired vehicle working state original infrared images, automatically recognizing vehicle, pedestrian, two-wheeled vehicle and three-wheeled vehicle rider targets in the scene, superimposing a detection frame on the detected target in the image, displaying the image with the superimposed detection frame, and further acquiring vehicle speed information, predicting the distance of the detected pedestrian target, calculating the collision time combined with the vehicle speed information, and superimposing a danger warning icon in the image and giving a collision danger prompt when the collision time is less than a certain danger threshold.

[0022] Further, the passive night vision camera comprises:

[0023] The optical system comprises a shutter, a heating ring, an optical window and an optical lens;

[0024] The shutter needs to be installed between the optical window and the optical lens;

[0025] The heating ring is pasted on the inside of the optical window;

[0026] The uncooled movement core assembly comprises an infrared detector, an infrared information processing circuit, a detector circuit and an external connector;

[0027] The infrared detector adopts a long-wave uncooled microbolometer, which is used for receiving infrared thermal radiation in the scene, converting the thermal signal into an electrical signal, and generating a grayscale image;

[0028] The infrared information processing circuit adopts a microcontroller for logic control, is used for responding to IIC control signals sent by the passive night vision controller, configuring bias voltage for the infrared detector, controlling the shutter and the heating ring, and is also used for generating a serial digital image sending timing, sending the digital image and correction parameters and a bad cell list in an LVDS serial format to the passive night vision controller through an external connector for data processing.

[0029] The detector circuit includes a detector configuration module, an analog signal conversion module, a focal plane temperature reading module and a detector image signal processing module; is used for providing a circuit interface for the detector, performing detector configuration, performing AD sampling conversion of the output analog signal into a digital signal, collecting the focal plane temperature and superimposing the focal plane temperature into the image data;

[0030] The detector circuit is connected with the infrared detector according to the detector manual, performs IIC communication with the infrared detector, completes infrared detector configuration and informs the infrared detector to start working, outputs an analog signal, and an AD sampling chip (i.e. the analog signal conversion module) on the detector circuit converts the analog signal into a digital signal, while the detector circuit can read a focal plane temperature signal from a pin of the infrared detector, and the focal plane temperature signal is superimposed with the digital signal, is serialized through LVDS, and is sent to the passive night vision controller through an external connector.

[0031] In order to avoid the influence of optical lens reflection on the non-uniformity of the infrared image, the entire passive night vision camera is divided into two cabin bodies, the cabin body in which the infrared detector is located is called a heat insulation cabin, and needs to have good thermal radiation consistency; the control circuit cabin includes a circuit board composed of the infrared information processing circuit and the detector circuit and an external connector, and the isolation design of the two cabin bodies ensures that the thermal radiation cannot be rapidly conducted to the infrared detector, thereby affecting the image quality.

[0032] Further, the passive night vision controller comprises:

[0033] The information processing module is used for performing target recognition on the preprocessed infrared image after infrared image preprocessing and infrared image detail enhancement, automatically recognizing pedestrians, two-wheeled and three-wheeled vehicle riders and automobile targets in a scene, superimposing a detection frame on the detected target in the image, transmitting the image with the superimposed detection frame to the vehicle information entertainment system in a GMSL format, and finally displaying on a liquid crystal screen;

[0034] The power supply module filters and converts an external input 12v power supply into a voltage required for the operation of each module, and ensures the normal operation of the circuit function;

[0035] Interface circuit, including GMSL interface, LVDS interface, CAN interface hardware device, for realizing information interaction of passive night vision controller and passive night vision camera, vehicle CAN network and vehicle-mounted liquid crystal display;

[0036] Controller processing circuit, circuit carrier of information processing module operation, two-point correction and bad cell replacement are carried out to the received original infrared image, and then digital detail enhancement is carried out;The enhanced image is subjected to deep learning target detection, and the detection target is displayed in the form of superimposed detection frame, and the target below the danger threshold is prewarned and superimposed warning icon is outputted;

[0037] Peripheral circuit, clock, memory, power failure non-volatile storage peripheral hardware required for data processing of controller processing circuit

[0038] On the other hand, the application provides a receiving user input program storage medium, the stored computer program makes the electronic equipment execute the split type passenger vehicle passive night vision auxiliary driving method.

[0039] On the other hand, the application provides a computer device, the computer device includes memory and processor, the memory stores computer program, the computer program is executed by the processor, so that the processor executes the split type passenger vehicle passive night vision auxiliary driving method.

[0040] In combination with all the above technical solutions, the application has the following advantages and positive effects:

[0041] First, in view of the technical problems existing in the prior art and the difficulty in solving the problems, the technical solutions to be protected by the application and the results and data in the research and development process are closely combined, the technical problems solved by the application are analyzed in detail and deeply, and some creative technical effects brought after the problems are solved are described as follows: the purpose of the application is to provide a split type passive night vision auxiliary driving system solution, which converts the infrared radiation information of the target and scene in the observation area of the vehicle driver into real-time thermal imaging video, and runs a target detection algorithm on the video content to automatically identify pedestrians, two-wheeled vehicles, cars and other targets in the scene, generates a warning signal by integrating vehicle CAN information, and outputs the video information and warning signal to the vehicle information entertainment system (IVI) and displays it on the liquid crystal display. The real vehicle installation and operation effect is shown in the liquid crystal display screen display effect diagram of Fig. 5(a), and the actual pedestrian effect diagram corresponding to the liquid crystal display screen display content is shown in Fig. 5(b).

[0042] Second, the technical solution is regarded as a whole or from the perspective of the product, the technical effect and advantages of the technical solution to be protected by the application are described as follows: the infrared imaging of the scene in front of the driver is performed by the passive night vision camera, clear imaging in night, rain, snow, fog and haze weather can be ensured, pedestrians, two-wheeled vehicle and three-wheeled vehicle riders, automobiles and other targets are identified by the passive night vision controller, and the driver is automatically prompted about the collision risk, thereby effectively improving the safety of the driver driving the vehicle in a low-illumination environment.

[0043] Third, as the creative evidence of the claims of the application, it is also embodied in the following important aspects: (1) the passive night vision system of the application is currently only used on foreign high-end vehicles, and there is no application case on domestic vehicles. With the continuous decline of the price of infrared detectors, the passive night vision system is expected to become a standard sensor of intelligent networked vehicles in the future, and the market size will break through 100 million yuan. (2) The application has been put into use in a domestic high-end vehicle, which becomes the first domestic vehicle equipped with a passive night vision system. (3) The application realizes the embedding of target recognition technology based on deep learning, so that the infrared night vision equipment becomes an edge computing device with real-time target recognition capability, which helps the passive night vision system to be widely promoted on various passenger vehicles and commercial vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0044] Other features, objects and advantages of the application will become more apparent through reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0045] Figure 1 is a schematic diagram of the split type passenger vehicle passive night vision auxiliary driving system provided by the embodiment of the application;

[0046] Figure 2 is a circuit design framework diagram of the passive night vision camera provided by the embodiment of the application;

[0047] Figure 3 is an interface circuit framework schematic diagram provided by the embodiment of the application;

[0048] Figure 4 is a flow chart of the split type passenger vehicle passive night vision auxiliary driving method provided by the embodiment of the application;

[0049] Fig. 5(a) is a display effect diagram of the liquid crystal display screen in the running of the actual vehicle provided by the embodiment of the application;

[0050] Fig. 5(b) is an actual pedestrian effect diagram of the display content of the liquid crystal display screen provided by the embodiment of the application;

[0051] Fig. 6(a) is a mounting effect diagram of the application on a certain domestic vehicle provided by the embodiment of the application Figure 1 ;

[0052] Figure 6(b) shows the effect of installing the present invention on a certain domestic car model according to an embodiment of the present invention. Figure 2 ;

[0053] Figure 7 This is a hardware connection diagram of a passive night vision camera provided in an embodiment of the present invention;

[0054] In the diagram: 1. Passive night vision camera; 1-1. Infrared detector; 1-2. Infrared information processing circuit; 1-2-1. Circuit board; 1-2-2. Connector; 1-3. Shutter; 1-4. Heating ring; 1-5. Optical window; 1-6. Optical lens; 1-7. Detector circuit; 1-8. External connector; 2. Passive night vision controller; 2-1. Information processing module; 2-2. Power supply module; 2-3. Interface circuit; 2-4. Controller processing circuit; 2-5. Peripheral circuit. Detailed Implementation

[0055] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] I. Explanation of the Implementation Example:

[0058] Example 1

[0059] like Figure 1 As shown, the split-type passenger vehicle passive night vision assisted driving system provided in this embodiment of the invention comprises a passive night vision camera 1 and a passive night vision controller 2.

[0060] The passive night vision camera 1 is installed in the front grille of the vehicle and is used to convert the received infrared radiation signal of the vehicle's working state into a digital signal. Based on the electrical signal, it receives the original infrared image according to the image output timing of the infrared detector 1-1, and transmits the converted original infrared image to the passive night vision controller 2 in real time.

[0061] The passive night vision controller 2 is installed in the vehicle cab, and is used for target recognition after original infrared image of vehicle working state obtained is preprocessed and enhanced in details, automatically recognizing targets in the scene, superimposing a detection frame on the detected target in the image, and displaying the image with the superimposed detection frame; and is also used for obtaining vehicle speed information, predicting the distance of the detected pedestrian target, calculating the collision time in combination with the vehicle speed information, and superimposing a danger warning icon in the image when the collision time is less than a certain danger threshold (such as 3.5s), prompting the driver to pay attention to the collision danger.

[0062] The passive night vision camera 1 includes an infrared detector 1-1 and an infrared information processing circuit 1-2. The infrared detector 1-1 is made of long-wave uncooled microbolometer, can receive infrared thermal radiation in the scene, convert the thermal signal into an electrical signal, and thus generate a gray-scale image. The infrared information processing circuit 1-2 processes the electrical signal converted by the infrared detector 1-1, converts the signal into serial data, and transmits the data to the passive night vision controller 2.

[0063] The passive night vision camera 1 is installed in the vehicle front grille, and is mainly used for receiving infrared radiation of the scene, converting the thermal radiation signal into an electrical signal, receiving the original infrared image according to the image output timing sequence of the infrared detector 1-1, and converting the image into LVDS format and transmitting the image to the passive night vision controller 2.

[0064] The passive night vision camera 1 can also receive control instructions from the passive night vision controller 2 to control the infrared detector 1-1, the shutter 1-3 and the heating ring 1-4. The passive night vision camera 1 is designed to have an IP6K9K dustproof and waterproof level, and can adapt to harsh working conditions outside the vehicle.

[0065] The passive night vision controller 2 receives the original infrared image transmitted by the passive night vision camera 1, runs preprocessing algorithms such as infrared image preprocessing and infrared image detail enhancement, generates a low-noise and high-quality 8-bit infrared image, and sends the preprocessed infrared image to a deep learning module 2-1 for target recognition. The deep learning module 2-1 can automatically recognize targets such as pedestrians, two-wheeled vehicles and cars in the scene, superimpose a detection frame on the detected target in the image, transmit the image with the superimposed detection frame to the vehicle information entertainment system in GMSL format, and finally display the image on the liquid crystal screen.

[0066] The passive night vision controller 2 can also receive vehicle CAN information, obtain vehicle working state, predict target distance, and evaluate collision risk in combination with vehicle speed and distance prediction. When a collision risk occurs, the passive night vision controller 2 displays a warning prompt icon on the liquid crystal screen to prompt the driver. The passive night vision controller 2 is installed in the vehicle cab, and is designed to have an IP52 dustproof and waterproof level.

[0067] Embodiment 2

[0068] Based on the passive night vision auxiliary driving system of the split passenger vehicle described in embodiment 1, the passive night vision camera 1 is composed of an optical system and a non-refrigeration core assembly, the optical system including: a shutter 1-3, a heating ring 1-4, an optical window 1-5, and an optical lens 1-6;

[0069] The shutter 1-3 is installed between the optical window 1-5 and the optical lens 1-6;

[0070] The heating ring 1-4 is pasted to the inside of the optical window 1-5; when the temperature of the optical window 1-5 is lower than 2℃, heating is started, and when it is higher than 7℃, heating is stopped.

[0071] The non-refrigeration core assembly includes an infrared detector 1-1, an infrared information processing circuit 1-2, a detector circuit 1-7, and an external connector 1-8;

[0072] The infrared detector 1-1 adopts a long-wave non-refrigeration microbolometer, which is used to receive infrared thermal radiation in the scene, convert the thermal signal into an electrical signal, and generate a grayscale image;

[0073] The infrared information processing circuit 1-2 adopts a microcontroller for logical control, which is used to respond to the IIC control signal sent by the passive night vision controller 2, configure the bias voltage for the infrared detector 1-1, control the shutter 1-3 and the heating ring 1-4, and also generate a serial digital image sending timing, send the digital image and the correction parameter and the bad cell list in the LVDS serialization format to the passive night vision controller 2 through the external connector 1-8 for data processing;

[0074] The detector circuit 1-7 is mainly used to provide a circuit interface for the detector, configure the detector, and AD sample and convert the output analog electrical signal into a 14-bit digital signal;

[0075] The detector circuit 1-7 includes a detector configuration module, an analog signal conversion module, a focal plane temperature reading module, and a detector image signal processing module; it is used to provide a circuit interface for the detector, configure the detector, AD sample and convert the output analog signal into a digital signal, collect the focal plane temperature and superimpose it into the image data;

[0076] The detector circuit 1-7 is connected to the infrared detector 1-1 according to the detector manual, communicates with the infrared detector 1-1 through IIC, completes the configuration of the infrared detector 1-1, informs the infrared detector 1-1 to start working, outputs an analog signal, and the AD sampling chip on the detector circuit 1-7 converts the analog signal into a digital signal; at the same time, the detector circuit 1-7 can read the focal plane temperature signal from the pin of the infrared detector 1-1, and superimpose it with the digital signal, perform LVDS serialization, and send it to the passive night vision controller 2 through the external connector 1-8.

[0077] In order to avoid the influence of the reflection of the optical lens 1-6 on the non-uniformity of the infrared image, the entire passive night vision camera 1 is divided into two cabins, the cabin in which the infrared detector 1-1 is located is called a heat insulation cabin, which needs to have good thermal radiation consistency; the cabin of the control circuit includes a circuit board 1-2-1 composed of an infrared information processing circuit 1-2 and a detector circuit 1-7 and an external connector 1-8, and the isolation design of the two cabins ensures that the thermal radiation cannot be rapidly conducted to the infrared detector 1-1, thereby affecting the image quality.

[0078] For example, the infrared information processing circuit 1-2 mainly includes a control module of the heating ring 1-4, a control module of the shutter 1-3, a data serialization module, adopts a microcontroller for simple logic control, and saves the detector configuration parameters in a power-down non-volatile memory. On the one hand, in response to the IIC control signal sent by the controller, the detector bias voltage, the shutter 1-3 and the heating ring 1-4 are controlled, and on the other hand, a serialization digital image sending timing is generated, and the digital image, the correction parameter and the bad cell list are sent to the rear-end controller in an LVDS serialization format for data processing.

[0079] The detector circuit 1-7 mainly includes a detector configuration module, an analog signal conversion module, a focal plane temperature reading module and a detector image signal processing module, which are mainly used for providing a circuit interface for the detector, configuring the detector, and AD sampling and converting the output analog signal into a 14-bit digital signal, collecting the focal plane temperature and superimposing it into the image data.

[0080] The circuit design framework of the passive night vision camera 1 is shown in Figure 2 , and the specific hardware connection diagram is shown in Figure 7 .

[0081] The circuit cabin of the infrared information processing circuit 1-2 includes a circuit board 1-2-1 and a connecting piece 1-2-2, and the isolation design of the two cabins ensures that the thermal radiation cannot be rapidly conducted to the detector, thereby affecting the image quality. The shutter 1-3 is installed between the optical window 1-5 and the lens, so as to avoid the influence of the reflection of the lens on the non-uniformity of the infrared image.

[0082] Embodiment 3

[0083] Based on the split type passenger vehicle passive night vision auxiliary driving system described in embodiment 1, further, the passive night vision controller 2 considers waterproof design for the passive night vision controller 2 connector, and a heat dissipation pad is arranged on the main processing chip, which is convenient for heat conduction during the working process of the chip.

[0084] As shown in Figure 1 , the passive night vision controller 2 includes an information processing module 2-1, an electrical module 2-2, an interface circuit 2-3, a controller processing circuit 2-4 and a peripheral circuit 2-5 thereof.

[0085] The information processing module 2-1 is used for performing infrared image preprocessing on the raw infrared image transmitted by the passive night vision camera 1, performing target recognition on the preprocessed infrared image after infrared image detail enhancement, automatically recognizing pedestrians, two-wheeled and three-wheeled vehicle riders, and automobile targets in the scene, superimposing a detection frame on the detected target in the image, transmitting the image superimposed with the detection frame to the vehicle information entertainment system in the GMSL format, and finally displaying on the liquid crystal screen.

[0086] The power supply module 2-2 filters and converts the externally input 12v power into the voltage required for the operation of each module, to ensure the normal operation of the circuit function.

[0087] The interface circuit 2-3 mainly includes GMSL interface, LVDS interface, CAN interface hardware devices, as shown in FIG. 2, for realizing the information interaction of the passive night vision controller 2 with the passive night vision camera 1, the vehicle CAN network, and the vehicle-mounted liquid crystal display screen. Figure 3

[0088] The controller processing circuit 2-4 adopts a multi-core heterogeneous SoC with FPGA as the core, to realize the functions of infrared image preprocessing, image enhancement, target recognition, CAN information first launch, passive night vision camera 1 control, and GMSL format video data transmission. The peripheral circuit 2-5 provides the clock, memory, power failure non-volatile storage, and other peripheral hardware support required for data processing by the information processing circuit 2-4.

[0089] The passive night vision controller 2 adopts a multi-core heterogeneous processor as the core device of the controller, which is mainly used for image data acquisition of the camera, image processing and target recognition, and outputs video and alarm signals to the outside through the CAN bus and the vehicle control system. The passive night vision controller 2 communicates with the vehicle CAN network, acquires vehicle information in real time, and sends the working state and self-checking state of the passive night vision system to the outside through the CAN network.

[0090] The controller processing circuit 2-4 is responsible for data reception of the passive night vision camera 1, two-point correction and bad cell replacement are performed on the received 14-bit raw infrared image, and then digital detail enhancement is performed to convert it into an 8-bit image. The converted 8-bit image is sent to the information processing module 2-1 to run deep learning target detection, and is output in the form of superimposed detection frame. A distance estimation algorithm is run, combined with the speed information transmitted by the vehicle CAN network, to predict the collision time and give a warning to the target below the danger threshold and superimpose a warning icon. The infrared video superimposed with the detection and alarm information is sent to the vehicle information entertainment system in the GMSL format, and finally displayed on the vehicle-mounted liquid crystal display screen.

[0091] Embodiment 4

[0092] ​The split passenger vehicle passive night vision auxiliary driving method as shown in Figure 4 includes the following steps:

[0093] S1, the passive night vision camera 1 converts the received vehicle working state scene infrared radiation signal into photoelectric signal, receives the analog image according to the infrared detector 1-1 image output time sequence, and transmits the converted original digital infrared image in real time after analog-digital conversion.

[0094] The above process is: converting the thermal radiation signal into an electrical signal, and then converting the digital video signal through AD digital-analog conversion;

[0095] S2, the passive night vision controller 2 identifies the target after the obtained vehicle working state original infrared image is preprocessed and enhanced in detail, automatically identifies the target in the scene, superimposes a detection frame on the detected target in the image, displays the image superimposed with the detection frame, obtains the vehicle speed information, predicts the distance of the detected pedestrian target, calculates the collision time combined with the vehicle speed information, and when the collision time is less than a certain dangerous threshold, superimposes a danger warning icon in the image, and prompts the driver to pay attention to the collision danger.

[0096] In an embodiment of the present application, in step S1, the scene infrared radiation signal is converted into a digital signal by the detector circuit 1-7.

[0097] At the same time, the infrared information processing circuit 1-2 responds to the IIC control signal sent by the passive night vision controller 2 to control the bias voltage and operating temperature of the infrared detector 1-1, and generates a serial digital image sending time sequence, and sends the digital image and correction parameters and bad cell list in a serial format to the passive night vision controller 2 for data processing.

[0098] In an embodiment of the present application, in step S2, in addition to automatically identifying the target in the scene and superimposing a detection frame on the detected target in the image, the following steps are also performed: receiving vehicle CAN information, obtaining real-time working state of the vehicle, predicting the distance of the target, and evaluating the collision risk combined with the vehicle speed and distance prediction. A dangerous threshold is set by default, and if the time to collision is less than a certain threshold (3.5 seconds by default) according to the current vehicle speed and distance estimate, an alarm prompt icon is displayed by superimposition.

[0099] Embodiment 5

[0100] The split passenger vehicle passive night vision auxiliary driving method as shown in

[0101] After the passive night vision system is powered on, a self-checking process is run to determine whether the internal fixed data of the passive night vision camera 1 and the file data of the passive night vision controller 2 are correct, if the data is abnormal, the driving assistance process is stopped and an alarm signal is sent to the vehicle CAN network, if the data is normal, the driving assistance process is entered, and the shutter 1-3, the heating ring 1-4 and the image data are detected in real time, if an abnormality is found, the driving assistance process is stopped and an alarm signal is sent to the vehicle CAN network, if a collision danger is detected, an alarm signal is sent to the vehicle CAN network, and an alarm prompt icon is superimposed on the infrared display image video.

[0102] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0103] The information interaction, execution process and the like between the above devices / units can be based on the same concept as the method embodiments of the present application, and the specific functions and brought technical effects can be referred to the method embodiments part, which will not be repeated here.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can be referred to the corresponding process in the above method embodiments.

[0105] II. Application Embodiments

[0106] Application Example 1

[0107] The embodiment of the present application provides a split embedded information processing device, which comprises a passive night vision camera 1 and a passive night vision controller 2.

[0108] The embodiment of the present application provides a kind of image information acquisition equipment, passive night camera 1, including optical window 1-5, optical lens 1-6, infrared thermal imaging sensor, embedded processor, memory, heating ring 1-4 and shutter 1-3.Install in the front grille of vehicle, and the infrared thermal radiation signal of the scene in front of vehicle is collected, and is transmitted to passive night controller 2, heating ring 1-4 is used for when optical window 1-5 frost, heating automatic defrosting is carried out, shutter 1-3 is used for periodic type to block lens provides uniform radiation scene, completes infrared image non-uniformity correction, to improve infrared image quality.

[0109] The embodiment of the present application provides a kind of image information processing equipment, passive night controller 2, including a multi-core heterogeneous SoC and memory, for receiving the infrared image raw data transmitted by passive night camera 1, carries out image preprocessing, image enhancement, and sends image enhancement data to target detection module to identify vehicle, two-wheeled and three-wheeled vehicle rider, pedestrian, while communicating with vehicle CAN network, receives vehicle information to carry out collision warning.

[0110] A typical work flow of the present application example is as follows:

[0111] (1) after system power-on enters initialization process, carries out power-on self-test, and the storage data of passive night camera 1 and passive night controller 2 are self-checked, if self-checking is normal, then enter normal system flow, otherwise report exception to vehicle machine through CAN network;

[0112] (2) after system normal work, passive night camera 1 collects infrared radiation signal, and the signal is transmitted to passive night controller 2, passive night controller 2 receives image signal, carries out image preprocessing, image enhancement and target detection, and superimposes detection frame to the target detected;

[0113] (3) passive night controller 2 obtains vehicle information in real time through CAN network, when target detection finds pedestrian, carries out distance estimation to pedestrian target, when judging collision time is less than a certain threshold, then superimposes alarm information to the driver and carries out early warning;

[0114] (4) in the system working process, passive night controller 2 reads the temperature of optical window 1-5 transmitted by passive night camera 1, when temperature is lower than 2 ℃, heating ring 1-4 is started, and when temperature is higher than 7 ℃, heating ring 1-4 is closed, ensure that optical window 1-5 does not frost, to affect infrared thermal radiation signal reception;

[0115] (5) In the working process of the system, the passive night vision controller 2 reads the focal plane temperature of the passive night vision camera 1 transmitted by the infrared detector 1-1, and when the temperature changes greater than a certain threshold, the shutter 1-3 is triggered to block the lens, providing uniform scene radiation for the infrared detector 1-1 to complete non-uniform correction, thereby improving the quality of the infrared image;

[0116] (6) In the working process of the system, the passive night vision controller 2 queries the working voltage of the passive night vision camera 1, the working voltage of the heating ring 1-4 and the working voltage of the shutter 1-3 motor in real time, and when the voltage values of the above modules are abnormal, the CAN network reports the abnormality to the vehicle machine.

[0117] The embodiment of the present application provides a computer device, which comprises at least one processor, a memory and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above method embodiments when executing the computer program.

[0118] Application example 2

[0119] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps in any of the above method embodiments.

[0120] The embodiment of the present application further provides an information data processing terminal, which is used to provide a user input interface to implement the steps in any of the above method embodiments when executed on an electronic device, and the information data processing terminal is not limited to a mobile phone, a computer or a switch.

[0121] The embodiment of the present application further provides a server, which is used to provide a user input interface to implement the steps in any of the above method embodiments when executed on an electronic device.

[0122] The embodiment of the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to implement the steps in any of the above method embodiments.

[0123] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be instructed by a computer program to relevant hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0124] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0125] III. Evidence of the effects of the embodiments

[0126] The split passenger vehicle passive night vision system is loaded on a certain domestic vehicle model, as shown in FIG. 6(a) the effect of loading on a certain domestic vehicle model Figure 1 ; FIG. 6(b) the effect of loading on a certain domestic vehicle model Figure 2 .

[0127] Experiments show that the split passenger vehicle passive night vision auxiliary driving system provided by the embodiment of the application can clearly image under low illumination conditions based on infrared thermal imaging technology and deep learning target detection technology, improve the driver's perception ability of the scene in front, and enhance the driving safety.

[0128] The above description is only the preferred embodiment of the application and the explanation of the applied technical principles. Those skilled in the art should understand that the scope of the application involved in the application is not limited to the technical solutions formed by the specific combination of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the application (but not limited to) having similar functions.

Claims

1. A split passenger vehicle passive night vision assisted driving method, characterized by, The method comprises: S1, the passive night vision camera (1) converts the received vehicle working state scene infrared radiation signal into photoelectric signal, receives analog image according to infrared detector (1-1) image output timing sequence, and transmits the converted original digital infrared image in real time after analog-digital conversion; S2, the passive night vision controller (2) acquires the vehicle working state original infrared image, carries out target recognition after infrared image preprocessing and infrared image detail enhancement, automatically identifies the target in the scene, superimposes the detection frame on the detected target in the image, displays the image superimposed with the detection frame, acquires the vehicle speed information, predicts the distance of the detected pedestrian target, calculates the collision time combined with the vehicle speed information, and when the collision time is less than the set danger threshold, superimposes the danger warning icon in the image and gives a collision danger prompt; Wherein, after acquiring the vehicle working state original infrared image, carrying out target recognition after infrared image preprocessing and infrared image detail enhancement, the steps include: Two-point correction and bad cell replacement are carried out on the received original infrared image, and then digital detail enhancement is carried out; the enhanced image is subjected to deep learning target detection; The passive night vision camera (1) is divided into a heat insulation cabin and a control circuit cabin, the infrared detector (1-1) is arranged in the heat insulation cabin, the infrared information processing circuit (1-2) and the detector circuit (1-7) are arranged in the control circuit cabin, and the two cabin bodies are isolated to prevent heat radiation conduction from affecting image quality; S3, the passive night vision controller (2) reads the temperature of the optical window (1-5) transmitted by the passive night vision camera (1), starts the heating ring (1-4) when the temperature is lower than 2℃, and stops the heating ring (1-4) when the temperature is higher than 7℃; and reads the focal plane temperature of the infrared detector (1-1), and when the focal plane temperature changes by more than a threshold value, triggers the shutter (1-3) to provide uniform scene radiation for non-uniform correction.

2. The split passenger vehicle passive night vision assisted driving method according to claim 1, characterized in that, In step S1, the passive night vision camera (1) converts the analog electrical signal converted by the detector circuit (1-7) from the scene infrared radiation into a digital signal through AD sampling.

3. The split passenger vehicle passive night vision assisted driving method according to claim 1, characterized in that, In step S1, the passive night vision camera (1) uses the infrared information processing circuit (1-2) to respond to the IIC control signal sent by the passive night vision controller (2) to set the bias voltage of the infrared detector (1-1), control the shutter (1-3) and the heating ring (1-4), and generate a serial digital image transmission timing, and send the digital image and the correction parameters and the bad cell list in a serial format to the passive night vision controller (2) for data processing.

4. The split passenger vehicle passive night vision assisted driving method of claim 1, wherein In step S2, the automatic identification of the target in the scene and the superimposition of the detection frame on the detected target in the image specifically include: Receive vehicle CAN information, acquire real-time working state of the vehicle, predict the distance of the target, evaluate the collision risk combined with the vehicle speed and distance prediction, set the default danger threshold to 3.5s, judge the time of collision according to the current vehicle speed and distance estimate value, and if the time is less than the danger threshold, display through superimposed alarm prompt icon.

5. The split passenger vehicle passive night vision assisted driving method of claim 4, wherein, Displaying through superimposed alarm prompt icon includes: After power on, self-checking is performed, and whether the internal fixed data of the passive night vision camera (1) and the file data of the passive night vision controller (2) are correct is judged by comparing the check code saved in the non-volatile memory at power off and the data read from the non-volatile memory at power off. If the data is abnormal, the driving assistance process is stopped, and an alarm signal is sent to the vehicle CAN network; If the data is normal, the driving assistance process is entered, and the shutter (1-3), the heating ring (1-4) and the image data are detected in real time. If abnormal, the driving assistance process is stopped, and an alarm signal is sent to the vehicle CAN network. If a collision danger is detected, an alarm signal is sent to the vehicle CAN network, and an alarm prompt icon is superimposed on the infrared display image video.

6. A split passenger vehicle passive night vision assisted driving system for implementing the split passenger vehicle passive night vision assisted driving method according to any one of claims 1-5, characterized in that, The split type passenger vehicle passive night vision auxiliary driving system comprises: The passive night vision camera (1) is installed in the vehicle front grille and is used for digital signal conversion of received vehicle working state scene infrared radiation signals, receiving original infrared images according to the image output timing sequence of the infrared detector (1-1), and transmitting the converted original infrared images to the passive night vision controller (2) in real time after analog-digital conversion. The passive night vision controller (2) is installed in the vehicle cab and is used for target recognition after infrared image preprocessing and infrared image detail enhancement of the obtained vehicle working state original infrared images, automatic recognition of targets in the scene, including vehicles, pedestrians, two-wheeled vehicle riders and three-wheeled vehicle riders, superimposition of a detection frame on the detected target in the image, display of the image with the superimposed detection frame, acquisition of vehicle speed information, distance prediction of the detected pedestrian target, calculation of the collision time combined with the vehicle speed information, superimposition of a danger warning icon in the image when the collision time is less than a set danger threshold, and collision danger prompting.

7. The split passenger vehicle passive night vision assisted driving system of claim 6, wherein, The passive night vision camera (1) comprises an optical system and a non-refrigeration movement core assembly. The optical system comprises a shutter (1-3), a heating ring (1-4), an optical window (1-5) and an optical lens (1-6); the shutter (1-3) is installed between the optical window (1-5) and the optical lens (1-6); and the heating ring (1-4) is pasted to the inner side of the optical window (1-5). The non-refrigeration movement core assembly comprises an infrared detector (1-1), an infrared information processing circuit (1-2), a detector circuit (1-7) and an external connector (1-8). The infrared detector (1-1) adopts a long-wave non-refrigeration micro-thermal radiometer and is used for receiving infrared thermal radiation in the scene, converting the thermal signal into an electrical signal and generating a gray-scale image; The infrared information processing circuit (1-2) adopts a microcontroller for logic control, is used for responding to an IIC control signal sent by the passive night vision controller (2), configuring a bias voltage for the infrared detector (1-1), controlling the shutter (1-3) and the heating ring (1-4), and is also used for generating a serial digital image sending timing sequence, sending the digital image and correction parameters and a bad cell list to the passive night vision controller (2) in an LVDS serial format through the external connector (1-8) for data processing. The detector circuit (1-7) is connected with the infrared detector (1-1) in hardware, and the infrared detector (1-1) is configured and informed to start working by IIC communication, analog signals are output, the AD sampling chip on the detector circuit (1-7) converts the analog signals into digital signals, and the detector circuit (1-7) is serialized by LVDS, and the passive night vision controller (2) is sent through the external connector (1-8).

8. The split passenger vehicle passive night vision assisted driving system of claim 6, wherein, The passive night vision controller (2) comprises: An information processing module (2-1) is configured to perform target identification on the preprocessed infrared image after infrared image preprocessing and infrared image detail enhancement, automatically identify targets in a scene, the targets including vehicles, pedestrians, two-wheeled and three-wheeled riders, superimpose a detection frame on the detected target in an image, and transmit the image with the superimposed detection frame to a vehicle information entertainment system in a GMSL format, and finally display on a liquid crystal screen. A power supply module (2-2) filters and converts an external input 12V power supply into a voltage required for the operation of each module, and ensures the normal operation of the circuit function. An interface circuit (2-3) comprises GMSL, LVDS and CAN interface hardware devices, and is configured to realize information interaction between the passive night vision controller (2) and the passive night vision camera (1), a vehicle CAN network and a vehicle liquid crystal display screen. A controller processing circuit (2-4) is a circuit carrier for the operation of the information processing module (2-1), performs two-point correction and bad cell replacement on the received original infrared image, and then performs digital detail enhancement; the enhanced image is subjected to deep learning target detection, the detected target is displayed in the form of a superimposed detection frame, and a target below a danger threshold is prewarned and output with a prewarning icon superimposed. A peripheral circuit (2-5) provides clock, memory and power-down non-volatile storage peripheral hardware required for data processing of the controller processing circuit (2-4). 9.A receiving user input program storage medium, wherein a stored computer program causes an electronic device to execute the split passenger vehicle passive night vision auxiliary driving method of any one of claims 1-5.

10. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor, so that the processor executes the split passenger vehicle passive night vision auxiliary driving method of any one of claims 1-5.

Images