Method and device for controlling light supplementing time of camera module

Abstract

The invention provides a method and a device for controlling light supplementing time of a camera module. The method and the device can be applied to intelligent automobiles in the field of automatic driving. The method comprises the steps that when a human face or a human body is detected in a first image shot before a previous frame, a first target area needing light supplement in the first image is determined, and the distance between a human face area or a human body area in the first target area and a light-free area in the first image is larger than or equal to a preset threshold value. And when the photosensitive chip is exposed in the current frame, indicating an infrared light source to supplement light in a first exposure time period corresponding to the current frame according to a first target photosensitive chip row determined by the first target area. Otherwise, indicating the infrared light source to supplement light in the whole exposure process of the current frame. According to the method provided by the embodiment of the invention, the working time of the infrared light source can be reduced, so that the heating value of the infrared camera module can be reduced.

Description

[0001] This application is a divisional application of the invention application with the application date of September 25, 2020, the application number of 202011020997.9, and the title of "Method and Device for Controlling Fill Light Time of Camera Module". Technical Field

[0002] This application relates to the field of autonomous driving, and more particularly, to a method and device for controlling the fill light time of a camera module. Background Art

[0003] Artificial intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, a theory, method, technology, and application system that can perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable the machines to have the functions of perception, reasoning, and decision-making. Research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc.

[0004] Autonomous driving is a mainstream application in the field of artificial intelligence. Autonomous driving technology relies on the collaborative cooperation of computer vision, radar, monitoring devices, and global positioning systems, etc., to enable motor vehicles to achieve autonomous driving without the need for active human operation. Autonomous driving vehicles use various computing systems to help transport passengers or goods from one location to another. Some autonomous driving vehicles may require some initial input or continuous input from an operator (such as a navigator, driver, or passenger). Autonomous driving vehicles allow the operator to switch from a manual operation mode to an autonomous driving mode or a mode in between. Since autonomous driving technology does not require humans to drive motor vehicles, theoretically it can effectively avoid human driving errors, reduce the occurrence of traffic accidents, and improve the transportation efficiency of roads. Therefore, autonomous driving technology has received increasing attention.

[0005] As the number of vehicles increases, the safety of autonomous driving has received more and more attention. The current autonomous driving technology cannot achieve full unmanned driving. The cockpit can be monitored in real time through a monitoring camera to improve the safety of autonomous driving. For example, an infrared (IR) camera can be used to detect driver fatigue, recognize the behaviors, gestures, and detect left-behind items of the driver or other passengers in the cockpit.

[0006] An infrared camera can work normally regardless of visible light, whether during the day or at night. When a traditional infrared camera is exposed, an infrared light source is used as the light source, and each row of photosensitive chips in the photosensitive chip is exposed row by row through a rolling shutter until all rows of photosensitive chips are exposed, thus completing a complete exposure. However, this infrared camera module generates a relatively high amount of heat. Summary of the Invention

[0007] This application provides a method and device for controlling the fill light time of a camera module, which can reduce the heat generation of an infrared camera module.

[0008] In a first aspect, a method for controlling the fill light time of a camera module is provided. The camera module includes a camera, the camera includes a photosensitive chip, and the camera is a cockpit monitoring camera or a driver monitoring camera. The method includes: performing face detection or human detection in a first image captured before the current frame. When a face or a human body is detected in the first image, determining a first target region in the first image, where the first target region is the region in the first image that needs to be filled with light, the first target region includes the face region or the human body region in the first image, and the distance between the face region or the human body region and the lightless region in the first image is greater than or equal to a preset threshold. Determining a first exposure period of a first target photosensitive chip row in the current frame according to the first target region, where the first target photosensitive chip row refers to the row of chips in the photosensitive chip that is used to generate the image content in the first target region. When exposing the photosensitive chip in the current frame, instructing the infrared light source to perform fill light according to the first exposure period. When no face or human body is detected in the first image, instructing the infrared light source to perform full-time fill light during the exposure process of the current frame.

[0009] Among them, the camera module may include a camera, and the camera may include a photosensitive chip.

[0010] In an embodiment of this application, when a human body or a face is detected in the first image, the first exposure period of the first target photosensitive chip row in the current frame can be determined according to the region in the first image that needs to be filled with light. When exposing the photosensitive chip in the current frame, instructing the infrared light source to perform fill light according to the first exposure period can reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module.

[0011] Meanwhile, the method for controlling the fill light time of the camera module in the embodiments of the present application does not add or change the hardware modules (or units) in the camera module. Instead, it determines the fill light period (e.g., the first exposure period) of the photosensitive chip in the current frame according to the first target area, and when the photosensitive chip is exposed in the current frame, fills light during the fill light period to reduce the working time of the infrared light source, so that the heat generation of the infrared camera module can be reduced without increasing costs.

[0012] Furthermore, since the method for controlling the fill light time of the camera module in the embodiments of the present application does not add or change the hardware modules (or units) in the camera module, it also does not increase the size of the camera module, which is beneficial to the configuration and use of the camera module in the vehicle.

[0013] In the embodiments of the present application, when no human body or face is detected in the first image, the infrared light source can fill light throughout the exposure period of the photosensitive chip when shooting the next frame of image, so as to obtain an image with good illumination in all areas, and then re-detect the human body or face in the obtained image.

[0014] Optionally, the camera may further include an infrared light source and a rolling shutter.

[0015] It should be noted that the infrared light source in the embodiments of the present application can be either the built-in infrared light source of the camera or an independent external infrared light source, and the embodiments of the present application do not limit this.

[0016] In the embodiments of the present application, a margin can be considered when determining the core area in the first image. For example, a threshold can be preset, and a margin can be set when determining the core area in the first image, so that the distance between the face frame and the lightless area (i.e., the area in the first image that does not require fill light) is greater than or equal to the preset threshold.

[0017] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: performing face detection or human body detection on the second image obtained in the current frame. When a face or a human body is detected in the second image, and the distance between the face area or the human body area in the second image and the lightless area in the second image is less than the preset threshold, determine the second target area in the second image, and the second target area is the area in the second image that requires fill light. Determine the second exposure period of the second target photosensitive chip row in the subsequent frame according to the second target area, where the second target photosensitive chip row refers to the chip row in the photosensitive chip that generates the image content in the second target area. When the photosensitive chip is exposed in the subsequent frame, instruct the infrared light source to fill light according to the second exposure period.

[0018] In the embodiment of the present application, by adjusting the fill light period of the photosensitive chip in the subsequent frame based on the second target area in the second image obtained from the current frame, the image effect captured by the camera module can be improved.

[0019] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: performing face detection or human body detection on the second image obtained from the current frame. When a face or a human body is detected in the second image, and the distance between the face area or the human body area in the second image and the lightless area in the second image is greater than or equal to the preset threshold, determining the third exposure period of the first target photosensitive chip row in the subsequent frame according to the first exposure period. When exposing the photosensitive chip in the subsequent frame, instructing an infrared light source to perform fill light according to the third exposure period.

[0020] In the embodiment of the present application, the fill light period (for example, the third exposure period) of the subsequent frame can be conveniently determined based on the first exposure period.

[0021] It should be noted that the first exposure period or the third exposure period in the embodiment of the present application can be represented by relative time or absolute time.

[0022] In the case where the exposure period is represented by relative time, for example, assuming that the exposure start time of the current frame is T 0 , the first exposure period can refer to: the start time is T 0 +T 1 , and the end time is T 0 +T 2 The time period, that is, the interval between the start time of the first exposure period and the exposure start time of the current frame is T 1 , and the interval between the end time of the first exposure period and the exposure start time of the current frame is T 2 .

[0023] Similarly, assuming that the exposure start time of the subsequent frame is T 3 , the third exposure period can refer to: the start time is T 3 +T 1 , and the end time is T 3 +T 2 The time period, that is, the interval between the start time of the third exposure period and the exposure start time of the subsequent frame is T 1 , and the interval between the end time of the third exposure period and the exposure start time of the subsequent frame is T 2 .

[0024] In the case where the exposure period is represented by absolute time, for example, assuming that the exposure start time of the current frame is T0 , the first exposure period may refer to: the start time is T 4 , and the end time is T 5 of the time period. At this time, the interval between the start time of the first exposure period and the start time of the exposure of the current frame is T 4 -T 0 = T 1 , and the interval between the end time of the first exposure period and the start time of the exposure of the current frame is T 5 -T 0 = T 2 .

[0025] Similarly, assuming that the start time of the exposure of the subsequent frame is T 3 , the third exposure period may refer to: the start time is T 6 , and the end time is T 7 of the time period, that is, the interval between the start time of the third exposure period and the start time of the exposure of the subsequent frame is T 6 -T 3 = T 1 , and the interval between the end time of the third exposure period and the start time of the exposure of the subsequent frame is T 7 -T 3 = T 2 .

[0026] Combined with the first aspect, in some implementation manners of the first aspect, the method further includes: performing face detection or human body detection on the second image obtained in the current frame; when the number of times of not detecting a face or a human body in the second image is less than or equal to N times, determining the third exposure period of the first target photosensitive chip row in the subsequent frame according to the first exposure period; when exposing the photosensitive chip in the subsequent frame, instructing an infrared light source to perform supplementary lighting according to the third exposure period, where N is an integer greater than 1; when the number of times of not detecting a face or a human body in the second image is greater than N times, instructing the infrared light source to perform full-time supplementary lighting during the exposure process of the subsequent frame.

[0027] Combined with the first aspect, in some implementation manners of the first aspect, the photosensitive chip includes a plurality of photosensitive chip rows, and the plurality of pixel rows in the first image correspond to the plurality of photosensitive chip rows. Among them, the determining the first exposure period of the first target photosensitive chip row in the current frame according to the first target area includes: determining the first target photosensitive chip row corresponding to the pixel row in the first target area in the first image. Determining the first exposure period of the first target photosensitive chip row in the current frame.

[0028] In an embodiment of the present application, multiple pixel rows in the first image correspond to the multiple photosensitive chip rows. According to the pixel rows in the first target area in the first image, the first target photosensitive chip row can be conveniently determined, so as to facilitate determining the first exposure period of the first target photosensitive chip row in the current frame.

[0029] In a second aspect, a device for controlling the fill light time of a camera module is provided. The camera module includes a camera, the camera includes a photosensitive chip, and the camera is a cockpit monitoring camera or a driver monitoring camera. The device includes: a first determination unit, configured to determine a first target area in the first image when a face or a human body is detected in the first image captured by the camera before the current frame. The first target area is the area in the first image that needs to be filled with light, and the first target area includes a face area or a human body area in the first image, and the distance between the face area or the human body area and the lightless area in the first image is greater than or equal to a preset threshold. A second determination unit, configured to determine the first exposure period of the first target photosensitive chip row in the current frame according to the first target area, where the first target photosensitive chip row refers to the chip row in the photosensitive chip that generates the image content in the first target area. An indication unit, configured to, when the photosensitive chip is exposed in the current frame, indicate the infrared light source to perform fill light according to the first exposure period. The indication unit is further configured to, when no face or human body is detected in the first image, indicate the infrared light source to perform full-time fill light during the exposure process of the current frame.

[0030] Wherein, the camera module may include a camera, and the camera may include a photosensitive chip.

[0031] In an embodiment of the present application, the first exposure period of the first target photosensitive chip row in the current frame is determined according to the area in the first image that needs to be filled with light. When the photosensitive chip is exposed in the current frame, the infrared light source is indicated to perform fill light according to the first exposure period, which can reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module.

[0032] At the same time, the method for controlling the fill light time of the camera module in the embodiment of the present application does not add or change the hardware modules (or units) in the camera module, but determines the fill light period (for example, the first exposure period) of the photosensitive chip in the current frame according to the first target area, and performs fill light during the fill light period when the photosensitive chip is exposed in the current frame, so as to reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module without increasing the cost.

[0033] Furthermore, since no hardware modules (or units) in the camera module are added or changed, the method for controlling the fill light time of the camera module in the embodiments of the present application does not increase the size of the camera module, which is beneficial to the configuration and use of the camera module in the vehicle.

[0034] In the embodiments of the present application, when no human body or face is detected in the first image, the infrared light source can fill light throughout the exposure period of the photosensitive chip when shooting the next frame of image, so as to obtain an image with good lighting in all areas, and then re-detect the human body or face in the obtained image.

[0035] Optionally, the camera may further include an infrared light source and a rolling shutter.

[0036] It should be noted that the infrared light source in the embodiments of the present application may be an infrared light source built in the camera or an independent external infrared light source, and the embodiments of the present application do not limit this.

[0037] In the embodiments of the present application, a margin may be considered when determining the core area in the first image. For example, a threshold may be preset, and a margin is set when determining the core area in the first image, so that the distance between the face frame and the lightless area (i.e., the area in the first image that does not require fill light) is greater than or equal to the preset threshold.

[0038] Combined with the second aspect, in some implementation manners of the second aspect, the first determination unit is further configured to determine a second target area in the second image when a human face or a human body is detected in the second image, and the distance between the human face area or the human body area in the second image and the lightless area in the second image is less than the preset threshold, where the second target area is the area in the second image that requires fill light. The second determination unit is further configured to determine a second exposure period of the second target photosensitive chip row in subsequent frames according to the second target area, where the second target photosensitive chip row refers to the chip row in the photosensitive chip for generating the image content in the second target area. The indication unit is further configured to, when exposing the photosensitive chip in subsequent frames, indicate the infrared light source to fill light according to the second exposure period.

[0039] In the embodiments of the present application, by adjusting the fill light period of the photosensitive chip in subsequent frames based on the second target area in the second image obtained based on the current frame, the effect of the image captured by the camera module can be improved.

[0040] In combination with the second aspect, in some implementations of the second aspect, the second determination unit is further configured to determine a second target area in the second image when a face or a human body is detected in the second image obtained in the current frame, and the distance between the face area or the human body area and the lightless area in the second image is less than the preset threshold, where the second target area is the area in the second image that needs to be filled with light. The indication unit is further configured to, when exposing the photosensitive chip in subsequent frames, instruct the infrared light source to fill the light according to the third exposure period.

[0041] In the embodiments of the present application, the fill light period (for example, the third exposure period) of subsequent frames can be conveniently determined based on the first exposure period.

[0042] It should be noted that the first exposure period or the third exposure period in the embodiments of the present application can be represented by relative time or absolute time.

[0043] In the case where the exposure period is represented by relative time, for example, assuming that the exposure start time of the current frame is T 0 , the first exposure period may refer to: the start time is T 0 +T 1 , and the end time is T 0 +T 2 The time period, that is, the interval between the start time of the first exposure period and the exposure start time of the current frame is T 1 , and the interval between the end time of the first exposure period and the exposure start time of the current frame is T 2 .

[0044] Similarly, assuming that the exposure start time of the subsequent frame is T 3 , the third exposure period may refer to: the start time is T 3 +T 1 , and the end time is T 3 +T 2 The time period, that is, the interval between the start time of the third exposure period and the exposure start time of the subsequent frame is T 1 , and the interval between the end time of the third exposure period and the exposure start time of the subsequent frame is T 2 .

[0045] In the case where the exposure period is represented by absolute time, for example, assuming that the exposure start time of the current frame is T 0 , the first exposure period may refer to: the start time is T 4 , and the end time is T 5 The time period, at this time, the interval between the start time of the first exposure period and the exposure start time of the current frame is T 4-T 0 = T 1 The interval between the end time of the first exposure period and the start time of the exposure of the current frame is T 5 -T 0 = T 2 .

[0046] Similarly, assume that the start time of the exposure of the subsequent frame is T 3 , the third exposure period may refer to: the start time is T 6 , and the end time is T 7 The time period, that is, the interval between the start time of the third exposure period and the start time of the exposure of the subsequent frame is T 6 -T 3 = T 1 , the interval between the end time of the third exposure period and the start time of the exposure of the subsequent frame is T 7 -T 3 = T 2 .

[0047] Combined with the second aspect, in some implementations of the second aspect, the second determination unit is further configured to, when the number of times that no face or human body is detected in the second image is less than or equal to N times, determine the third exposure period of the first target photosensitive chip row in the subsequent frame according to the first exposure period. And the indication unit is further configured to, when exposing the photosensitive chip in the subsequent frame, indicate the infrared light source to perform supplementary lighting according to the third exposure period, where N is an integer greater than 1. Or the indication unit is further configured to, when the number of times that no face or human body is detected in the second image is greater than N times, indicate the infrared light source to perform full-time supplementary lighting during the exposure process of the subsequent frame.

[0048] Combined with the second aspect, in some implementations of the second aspect, the photosensitive chip includes a plurality of photosensitive chip rows, and the plurality of pixel rows in the first image correspond to the plurality of photosensitive chip rows. Wherein, the second determination unit is specifically configured to: determine the first target photosensitive chip row corresponding to the pixel row in the first target area in the first image. Determine the first exposure period of the first target photosensitive chip row in the current frame.

[0049] In the embodiments of the present application, the plurality of pixel rows in the first image correspond to the plurality of photosensitive chip rows. According to the pixel rows in the first target area in the first image, the first target photosensitive chip row can be conveniently determined, so as to facilitate determining the first exposure period of the first target photosensitive chip row in the current frame.

[0050] In a third aspect, a camera module is provided. The camera module includes a storage medium and a central processing unit. The storage medium can be a non-volatile storage medium. A computer-executable program is stored in the storage medium. The central processing unit is connected to the non-volatile storage medium and executes the computer-executable program to implement the method in the first aspect or any possible implementation manner of the first aspect.

[0051] In a fourth aspect, a chip is provided. The chip includes a processor and a data interface. The processor reads instructions stored on a memory through the data interface and executes the method in the first aspect or any possible implementation manner of the first aspect.

[0052] Optionally, as an implementation manner, the chip may further include a memory. Instructions are stored in the memory. The processor is configured to execute the instructions stored on the memory. When the instructions are executed, the processor is configured to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0053] In a fifth aspect, a computer-readable storage medium is provided. The computer-readable medium stores program code for a device to execute. The program code includes instructions for executing the method in the first aspect or any possible implementation manner of the first aspect.

[0054] In a sixth aspect, a vehicle is provided. The vehicle includes the device for controlling the fill light time of the camera module described in the second aspect above or the camera module described in the third aspect above.

[0055] In the embodiments of the present application, when a human body or a human face is detected in the first image, the first exposure period of the first target photosensitive chip row in the current frame can be determined according to the area that needs fill light in the first image. When the photosensitive chip is exposed in the current frame, the infrared light source is instructed to perform fill light according to the first exposure period, which can reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module. And when no human body or human face is detected in the first image, the infrared light source can perform full-time fill light during the exposure of the photosensitive chip when taking the next frame of image, obtaining an image with good lighting in all areas, and then re-detecting the human body or human face in the obtained image. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic structural diagram of an autonomous vehicle provided by an embodiment of the present application.

[0057] Figure 2 It is a schematic structural diagram of an autonomous driving system provided by an embodiment of the present application.

[0058] Figure 3Schematic structural diagram of a camera module applicable to an embodiment of the present application.

[0059] Figure 4 Schematic block diagram of a method for controlling the fill light time of a camera module provided by an embodiment of the present application.

[0060] Figure 5 Schematic structural diagram of a photosensitive chip provided by an embodiment of the present application.

[0061] Figure 6 Schematic block diagram of a method for determining the fill light period of the current frame provided by an embodiment of the present application.

[0062] Figure 7 Schematic block diagram of a method for controlling the fill light time of a camera module provided by another embodiment of the present application.

[0063] Figure 8 Schematic block diagram of a method for calculating the row of the photosensitive chip corresponding to the face frame provided by an embodiment of the present application.

[0064] Figure 9 Schematic block diagram of a method for controlling the fill light time of a camera module provided by another embodiment of the present application.

[0065] Figure 10 Schematic block diagram of a method for calculating the row of the photosensitive chip corresponding to the human body frame provided by an embodiment of the present application.

[0066] Figure 11 Schematic block diagram of a device for controlling the fill light time of a camera module provided by an embodiment of the present application.

[0067] Figure 12 Schematic block diagram of a device for controlling the fill light time of a camera module provided by another embodiment of the present application. Detailed implementation manners

[0068] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0069] The technical solutions of the embodiments of the present application can be applied to various vehicles. The vehicle can specifically be a diesel locomotive, a smart electric vehicle, or a hybrid vehicle. Alternatively, the vehicle can also be a vehicle of other power types, etc. The embodiments of the present application are not limited thereto.

[0070] The vehicle in the embodiments of the present application can be an autonomous vehicle. For example, the autonomous vehicle can be configured with an autonomous driving mode, and the autonomous driving mode can be a full autonomous driving mode or a partial autonomous driving mode. The embodiments of the present application are not limited thereto.

[0071] The vehicle in the embodiments of the present application may also be configured with other driving modes, and the other driving modes may include one or more of a variety of driving modes such as a sport mode, an economy mode, a standard mode, an off-road mode, a snow mode, and a climbing mode. The autonomous vehicle may switch between the autonomous driving mode and the above-mentioned multiple (driver-driven vehicle) driving models, and the embodiments of the present application do not limit this.

[0072] Figure 1 is a functional block diagram of the vehicle 100 provided by the embodiments of the present application.

[0073] In one embodiment, the vehicle 100 is configured to be in a fully or partially autonomous driving mode.

[0074] For example, the vehicle 100 may control itself while in the autonomous driving mode, and may determine the current state of the vehicle and its surrounding environment through manual operations, determine the possible behaviors of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the possibility of the other vehicle performing the possible behaviors, and control the vehicle 100 based on the determined information. When the vehicle 100 is in the autonomous driving mode, the vehicle 100 may be set to operate without interacting with people.

[0075] The vehicle 100 may include various subsystems, such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, as well as a power source 110, a computer system 112, and a user interface 116.

[0076] Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the vehicle 100 may be interconnected by wire or wirelessly.

[0077] The propulsion system 102 may include components that provide powered movement for the vehicle 100. In one embodiment, the propulsion system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, for example, a hybrid engine composed of a gasoline engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy.

[0078] Examples of the energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other power sources. The energy source 119 may also provide energy for other systems of the vehicle 100.

[0079] The transmission device 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission device 120 may include a gearbox, a differential, and a drive shaft.

[0080] In one embodiment, the transmission device 120 may further include other components, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 121.

[0081] The sensor system 104 may include several sensors that sense information about the environment around the vehicle 100.

[0082] For example, the sensor system 104 may include a positioning system 122 (the positioning system can be a GPS system, a Beidou system, or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126, a lidar 128, and a camera 130. The sensor system 104 may also include sensors that monitor the internal systems of the vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an engine oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). Such detection and identification are key functions for the safe operation of the autonomous vehicle 100.

[0083] The positioning system 122 can be used to estimate the geographical location of the vehicle 100. The IMU 124 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope.

[0084] The radar 126 can use radio signals to sense objects within the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or forward direction of the objects.

[0085] The lidar 128 can use lasers to sense objects in the environment where the vehicle 100 is located. In some embodiments, the lidar 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0086] The camera 130 can be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 130 can be a static camera or a video camera. Among them, the camera 130 can also include an infrared camera, other cameras. For example, the camera 130 can include a cockpit monitoring camera in a camera monitor system (CMS) and a driver monitoring camera in a driver monitor system (DMS).

[0087] The control system 106 controls the operation of the vehicle 100 and its components. The control system 106 may include various elements, including a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

[0088] The steering system 132 is operable to adjust the forward direction of the vehicle 100. For example, in one embodiment, it may be a steering wheel system.

[0089] The throttle 134 is used to control the operating speed of the engine 118 and thus the speed of the vehicle 100.

[0090] The braking unit 136 is used to control the deceleration of the vehicle 100. The braking unit 136 may use friction to slow down the wheels 121. In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electricity. The braking unit 136 may also take other forms to slow down the rotational speed of the wheels 121 and thus control the speed of the vehicle 100.

[0091] The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the surrounding environment of the vehicle 100. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, and so on.

[0092] The route control system 142 is used to determine the driving route of the vehicle 100. In some embodiments, the route control system 142 may combine data from the sensors 138, GPS 122, and one or more pre - determined maps to determine the driving route for the vehicle 100.

[0093] The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise navigate around potential obstacles in the environment of the vehicle 100.

[0094] Of course, in one example, the control system 106 may additionally or alternatively include components other than those shown and described. Or some of the components shown above may be reduced.

[0095] The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through the peripheral device 108. The peripheral device 108 may include a wireless communication system 146, an on - vehicle computer 148, a microphone 150, and / or a speaker 152.

[0096] In some embodiments, the peripheral device 108 provides a means for a user of the vehicle 100 to interact with the user interface 116. For example, the in-vehicle computer 148 can provide information to the user of the vehicle 100. The user interface 116 can also operate the in-vehicle computer 148 to receive user input. The in-vehicle computer 148 can be operated via a touch screen. In other cases, the peripheral device 108 can provide a means for the vehicle 100 to communicate with other devices located within the vehicle. For example, the microphone 150 can receive audio from the user of the vehicle 100 (e.g., voice commands or other audio inputs). Similarly, the speaker 152 can output audio to the user of the vehicle 100.

[0097] The wireless communication system 146 can wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication. The wireless communication system 146 can utilize WiFi to communicate with a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 can communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system 146 can include one or more dedicated short range communications (DSRC) devices, which can include public and / or private data communication between vehicles and / or roadside stations.

[0098] The power source 110 can provide power to various components of the vehicle 100. In one embodiment, the power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such a battery can be configured to power various components of the vehicle 100. In some embodiments, the power source 110 and the energy source 119 can be implemented together, as in some all-electric vehicles.

[0099] Some or all of the functions of the vehicle 100 are controlled by the computer system 112. The computer system 112 can include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium such as the data storage device 114. The computer system 112 can also be multiple computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0100] The processor 113 can be any conventional processor, such as a commercially available CPU. Alternatively, the processor can be a special-purpose device such as an ASIC or other hardware-based processor. AlthoughFigure 1 The functional diagram shows a processor, a memory, and other components of computer 110 in the same block, but those of ordinary skill in the art should understand that the processor, computer, or memory may actually include multiple processors, computers, or memories within the same physical housing. For example, the memory may be a hard disk drive or other storage media located in a housing different from that of computer 110. Thus, a reference to a processor or computer will be understood to include a reference to a collection of processors or computers or memories that may or may not operate in parallel. Instead of using a single processor to perform the steps described herein, some components such as the steering component and the deceleration component may each have their own processor that only performs calculations related to component-specific functions.

[0101] In various aspects described herein, the processor may be located remote from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor disposed within the vehicle while others are executed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0102] In some embodiments, data storage device 114 may contain instructions 115 (e.g., program logic) that may be executed by processor 113 to perform various functions of vehicle 100, including those described above. Data storage device 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of propulsion system 102, sensor system 104, control system 106, and peripheral devices 108.

[0103] In addition to instructions 115, data storage device 114 may also store data, such as road maps, route information, the location, direction, speed of the vehicle, and other such vehicle data, as well as other information. Such information may be used by vehicle 100 and computer system 112 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0104] User interface 116, for providing information to or receiving information from a user of vehicle 100. Optionally, user interface 116 may include one or more input / output devices within the collection of peripheral devices 108, such as wireless communication system 146, vehicle-to-vehicle computer 148, microphone 150, and speaker 152.

[0105] The computer system 112 can control the functions of the vehicle 100 based on inputs received from various subsystems (e.g., the propulsion system 102, the sensor system 104, and the control system 106) and from the user interface 116. For example, the computer system 112 can utilize inputs from the control system 106 to control the steering unit 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144. In some embodiments, the computer system 112 can operate to provide control over many aspects of the vehicle 100 and its subsystems.

[0106] Optionally, one or more of the above components can be mounted or associated separately from the vehicle 100. For example, the data storage device 114 can exist partially or completely separately from the vehicle 100. The above components can be communicatively coupled together in a wired and / or wireless manner.

[0107] Optionally, the above components are just an example. In actual applications, the components in each of the above modules may be added or deleted according to actual needs. Figure 1 It should not be construed as a limitation on the embodiments of the present application.

[0108] An autonomous vehicle traveling on a road, such as the vehicle 100 above, can identify objects within its surrounding environment to determine adjustments to its current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, distance from the vehicle, etc., can be used to determine the speed adjustment required for the autonomous vehicle.

[0109] Optionally, the vehicle 100 or a computing device associated with the vehicle 100 (such as Figure 1 the computer system 112, the computer vision system 140, the data storage device 114) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of the others, so all the identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what steady state (e.g., accelerate, decelerate, or stop) the vehicle will need to adjust to based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 on the road being traveled, the curvature of the road, the proximity of static and dynamic objects, etc.

[0110] In addition to providing instructions to adjust the speed of an autonomous vehicle, the computing device can also provide instructions to modify the steering angle of vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from an object near the autonomous vehicle (e.g., a sedan in an adjacent lane on the road).

[0111] The above-mentioned vehicle 100 can be a sedan, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, a lawn mower, a recreational vehicle, a playground vehicle, construction equipment, a tram, a golf cart, a train, a trolley, etc., and the embodiments of the present application do not make special limitations.

[0112] Figure 2 It is a schematic diagram of an autonomous driving system provided by an embodiment of the present application.

[0113] As Figure 2 The shown autonomous driving system includes a computer system 101. Among them, the computer system 101 includes a processor 103, and the processor 103 is coupled to a system bus 105. The processor 103 can be one or more processors, and each processor can include one or more processor cores. A display adapter 107, the display adapter can drive a display 109, and the display 109 is coupled to the system bus 105. The system bus 105 is coupled to an input / output (I / O) bus 113 through a bus bridge 111. An I / O interface 115 is coupled to the I / O bus. The I / O interface 115 communicates with a variety of I / O devices, such as an input device 117 (e.g., a keyboard, a mouse, a touch screen, etc.), a media tray 121 (e.g., a CD-ROM, a multimedia interface, etc.). A transceiver 123 (which can send and / or receive radio communication signals), a camera 155 (which can capture static and dynamic digital video images), and an external USB interface 125. Among them, optionally, the interface connected to the I / O interface 115 can be a USB interface.

[0114] Among them, the processor 103 can be any conventional processor, including a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination of the above. Optionally, the processor can be a dedicated device such as an application specific integrated circuit (ASIC). Optionally, the processor 103 can be a neural network processor or a combination of a neural network processor and the above conventional processors.

[0115] Optionally, in various embodiments described herein, computer system 101 may be located remotely from the autonomous vehicle (e.g., computer system 101 may be located in the cloud or on a server) and may communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes described herein are executed on a processor disposed within the autonomous vehicle, and others are executed by a remote processor, including taking actions required to perform a single maneuver.

[0116] Computer 101 may communicate with software deployment server 149 via network interface 129. Network interface 129 is a hardware network interface, such as a network card. Network 127 may be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, network 127 may also be a wireless network, such as a WiFi network, a cellular network, etc.

[0117] Hard disk drive interface is coupled to system bus 105. The hardware drive interface is connected to a hard disk drive. System memory 135 is coupled to system bus 105. Data running in system memory 135 may include operating system 137 and application program 143 of computer 101.

[0118] The operating system includes parser 139 (shell) and kernel 141. The shell is an interface between the user and the kernel of the operating system. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waits for the user's input, interprets the user's input to the operating system, and processes various output results of the operating system.

[0119] Kernel 141 consists of those parts of the operating system that manage memory, files, peripherals, and system resources. Interacting directly with the hardware, the operating system kernel typically runs processes and provides inter-process communication, provides CPU time slice management, interrupts, memory management, IO management, etc.

[0120] Application program 143 includes a program related to controlling the fill light time of the camera module. For example, determining a first target area in a first image captured by the camera module before the current frame, where the first target area is the area in the first image that needs to be filled with light; determining a first exposure period of a first target photosensitive chip row in the current frame according to the first target area, where the first target photosensitive chip row refers to the chip row of the photosensitive chip used to generate the image content in the first target area; when exposing the photosensitive chip in the current frame, controlling the infrared light source to perform fill light only within the first exposure period through a rolling shutter.

[0121] The application 143 may also exist on the system of the software deployment server 149. In one embodiment, when the application 143 needs to be executed, the computer system 101 may download the application 143 from the software deployment server 149.

[0122] The sensor 153 is associated with the computer system 101. The sensor 153 is used to detect the environment around the computer 101, or the sensor 153 may also be used to monitor the situation inside the cockpit of an autonomous vehicle. Optionally, the computer 101 is located on an autonomous vehicle.

[0123] For example, the sensor 153 may include a driver monitoring camera in a driver monitor system (DMS), and the driver monitoring camera may be used for fatigue detection, face recognition, distraction detection, in-loop detection, etc. of the driver of an autonomous vehicle; or, the sensor 153 may also include a cockpit monitoring camera in a camera monitor system (CMS), and the cockpit monitoring camera may be used for behavior recognition, gesture recognition, and detection of left-behind items of the driver or other passengers inside the cockpit

[0124] For example, the application 143 may detect the image collected by the sensor 153 to determine the area that needs to be filled with light in the image, and determine the fill light period of the photosensitive chip during the exposure of the current frame in combination with the area that needs to be filled with light. At this time, by means of a rolling shutter, controlling the infrared light source to fill light during the fill light period can reduce the heat generation of the infrared camera module.

[0125] Figure 3 FIG. 300 is a schematic structural diagram of a camera module applicable to the embodiments of the present application. It should be understood that Figure 3 The shown camera module 300 is only an example and not a limitation. The camera module 300 may include more or fewer steps, which are not limited in the embodiments of the present application.

[0126] The camera module 300 may be used as a driver monitoring camera in a driver monitor system (DMS) to perform fatigue detection, face recognition, distraction detection, in-loop detection, etc. on the driver of an autonomous vehicle; or, the camera module 300 may also be used as a cockpit monitoring camera in a camera monitor system (CMS) to perform behavior recognition, gesture recognition, and detection of left-behind items, etc. on the driver or other passengers inside the cockpit.

[0127] Such as Figure 3As shown, the camera module 300 may include a lens 301 , a photosensitive chip 302 , an image signal processor (ISP) 303 , a central processing unit (CPU) / neural processing unit (NPU) 304 , an infrared (IR) light source 305 and a light source controller 306 .

[0128] Among them, the lens 301 may include a rolling shutter, the photosensitive chip 302 may be a complementary metal-oxide-semiconductor (CMOS), and the ISP 303 may be a static ISP (stack ISP) integrated on the CMOS. For example, the ISP 303 may be integrated on the photosensitive chip 302, or the ISP 303 may also be an independent ISP (discrete ISP), and the infrared light source 305 may be a light emitting diode (LED), or the infrared light source 305 may also be a vertical-extemal-cavity surface-emitting laser (VECSEL).

[0129] In the prior art, when exposing, a traditional infrared camera uses an infrared light source as a light source and exposes each row of photosensitive chips in the photosensitive chip row by row through a rolling shutter. Figure 5 As shown, the exposure time periods of the various photosensitive chip rows in the photosensitive chip are different, and a complete exposure is completed until all the photosensitive chip rows in the photosensitive chip are exposed.

[0130] Among them, infrared light source refers to a light source with a wavelength of 780 to 1400 nanometers (nm), which is invisible to the human eye. The infrared light source combined with the infrared camera can make the shooting unaffected by visible light, and can be shot normally regardless of day or night. In the exposure process of traditional infrared cameras, the infrared light source is always working, that is, the exposure time of the photosensitive chip (in a certain frame) is equal to the working time of the infrared light source. However, the infrared light source will generate heat when working, so the long-term operation of the infrared light source will cause the camera module to generate high heat.

[0131] Based on the above problems, the present application proposes a method for controlling the fill light time of a camera module. The first exposure period of the first target photosensitive chip row in the current frame is determined according to the area that needs fill light in the first image, and when the photosensitive chip is exposed in the current frame, fill light is only performed within the first exposure period, which can reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module.

[0132] The following will combine Figures 4 to 10 to elaborate in detail on the method for controlling the fill light time of the camera module in the embodiments of the present application.

[0133] Figure 4 is a schematic flowchart of the method 400 for controlling the fill light time of the camera module provided by the embodiments of the present application.

[0134] Figure 4 The method 400 shown may include step 410, step 420, and step 430. It should be understood that Figure 4 the method 400 shown is only an example and not a limitation. The method 400 may include more or fewer steps, which are not limited in the embodiments of the present application. The following will introduce these steps in detail respectively.

[0135] Figure 4 The method 400 shown may be executed by Figure 1 the camera module in the camera 130 in the vehicle 100 in Figure 2 or, the method 400 may also be executed by

[0136] the camera module in the camera 155 or the sensor 153 in the autonomous driving system in Figure 3 the camera module 300 shown.

[0137] S410, determine the first target area in the first image captured by the camera before the current frame.

[0138] Among them, the first target area may be the area that needs fill light in the first image.

[0139] Optionally, the content included in the area that needs fill light in the first image may be related to the use of the camera module. For example, the first target area in the first image may be determined according to a preset target object.

[0140] For example, the camera module can be used as a driver monitoring camera in a driver monitor system (DMS) to perform fatigue detection, face recognition, distraction detection, in-loop detection, etc. on the driver of an autonomous vehicle. Correspondingly, when the camera module is used as a driver monitoring camera, the preset target object can refer to the face area in the first image.

[0141] For another example, the camera module can also be used as a cockpit monitoring camera in a camera monitor system (CMS) to perform behavior recognition, gesture recognition, and detection of left-behind objects on the driver or other passengers in the cockpit. Correspondingly, when the camera module is used as a cockpit monitoring camera, the preset target object can refer to the human body area in the first image.

[0142] For yet another example, the camera can also be used as an external camera for detecting other surrounding vehicles to detect and identify other vehicles around the vehicle. Correspondingly, when the camera is used as an external camera for detecting other surrounding vehicles, the preset target object can refer to the vehicle (e.g., other vehicles around the vehicle) area in the first image.

[0143] Optionally, a deep learning algorithm or Haar operator can be used to detect the first image to determine the area in the first image that needs to be filled with light, i.e., the first target area.

[0144] For example, a neural network model can be used to determine the face area (or human body area) in the first image, i.e., the first target area in the first image that needs to be filled with light.

[0145] S420. Determine the first exposure period of the first target photosensitive chip row in the current frame according to the first target area.

[0146] Among them, the first target photosensitive chip row can refer to the chip row in the photosensitive chip used to generate the image content in the first target area.

[0147] Optionally, the photosensitive chip can include multiple photosensitive chip rows, the first image can include multiple pixel rows, and the multiple pixel rows in the first image can correspond to the multiple photosensitive chip rows.

[0148] For example, as Figure 6 shown, the photosensitive chip can include 960 photosensitive chip rows, the first image can include 960 pixel rows, and the 960 pixel rows included in the first image can correspond one-to-one to the 960 photosensitive chip rows included in the photosensitive chip.

[0149] Figure 6 Shown is a case where the number of rows of the photosensitive chip rows in the photosensitive chip is equal to the number of rows of pixel rows in the first image, that is, a case where the image generated after the photosensitive chip is exposed is not subjected to a stretching transformation.

[0150] It should be noted that the embodiments of the present application do not limit the number of rows of the photosensitive chip rows in the photosensitive chip and the number of rows of pixel rows in the first image. That is to say, in the embodiments of the present application, it is not limited that the number of rows of the photosensitive chip rows in the photosensitive chip must be equal to the number of rows of pixel rows in the first image, nor is it limited that the multiple pixel rows in the first image must correspond one-to-one to the multiple photosensitive chip rows.

[0151] Optionally, the determining the first exposure period of the first target photosensitive chip row in the current frame according to the first target area may include:

[0152] Determining the first target photosensitive chip row corresponding to the pixel rows in the first target area in the first image; determining the first exposure period of the first target photosensitive chip row in the current frame.

[0153] For example, as Figure 6 shown, it is possible to determine the pixel row a in the first image of the upper boundary of the first target area and the pixel row b in the first image of the lower boundary of the first target area; determining the photosensitive chip row n in the photosensitive chip corresponding to the pixel row a and the photosensitive chip row m in the photosensitive chip corresponding to the pixel row b, then all the photosensitive chip rows starting from the photosensitive chip row n to the photosensitive chip row m are the first target photosensitive chip rows.

[0154] Furthermore, the exposure period of the first target photosensitive chip row in the current frame can be calculated.

[0155] For example, assuming that the start time of exposure of the first row of photosensitive chip rows is T0, the time used for exposure of each row of photosensitive chip rows is T, and the time difference between the start times of exposure of adjacent two rows of photosensitive chip rows is t, then the start time T1 of the nth row of photosensitive chip rows is: T1 = T0 + n * t, and the start time T2 of the mth row of photosensitive chip rows is: T2 = T0 + T + m * t, that is Figure 6 the start time T1 of the infrared light source supplementary light and the end time T2 of the infrared light source supplementary light shown in

[0156] At this time, the exposure period (T1 - T2) of the first target photosensitive chip row in the current frame is the first exposure period.

[0157] S430, when exposing the photosensitive chip in the current frame, instruct the infrared light source to perform supplementary lighting according to the first exposure period.

[0158] In the embodiment of the present application, the first exposure period of the first target photosensitive chip row in the current frame is determined according to the area that needs supplementary lighting in the first image. When exposing the photosensitive chip in the current frame, instruct the infrared light source to perform supplementary lighting according to the first exposure period, which can reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module.

[0159] Meanwhile, the method for controlling the supplementary lighting time of the camera module in the embodiment of the present application does not add or change the hardware modules (or units) in the camera module. Instead, the supplementary lighting period (e.g., the first exposure period) of the photosensitive chip in the current frame is determined according to the first target area, and when exposing the photosensitive chip in the current frame, supplementary lighting is performed within the supplementary lighting period to reduce the working time of the infrared light source, thereby reducing the heat generation of the infrared camera module without increasing costs.

[0160] Furthermore, since the hardware modules (or units) in the camera module are not added or changed, the method for controlling the supplementary lighting time of the camera module in the embodiment of the present application does not increase the size of the camera module either, which is beneficial to the configuration and use of the camera module in the vehicle.

[0161] Optionally, the method 400 may further include step 432.

[0162] S432, determine the third exposure period of the first target photosensitive chip row in subsequent frames according to the first exposure period; when exposing the photosensitive chip in subsequent frames, instruct the infrared light source to perform supplementary lighting according to the third exposure period.

[0163] That is to say, when exposing the photosensitive chip in subsequent frames, control the infrared light source to perform supplementary lighting only during the period when the first target photosensitive chip row is exposed.

[0164] In the embodiment of the present application, based on the first exposure period, the supplementary lighting period (e.g., the third exposure period) of subsequent frames can be conveniently determined.

[0165] Optionally, the method 400 may further include step 434, step 436 and step 438.

[0166] S434, determine the second target area in the second image obtained in the current frame, and the second target area is the area that needs supplementary lighting in the second image.

[0167] S436. Determine a second exposure period of a second target photosensitive chip row in a subsequent frame according to the second target region, where the second target photosensitive chip row refers to the chip row in the photosensitive chip for generating the image content in the second target region.

[0168] S438. When the photosensitive chip is exposed in a subsequent frame, instruct an infrared light source to perform supplementary lighting according to the second exposure period.

[0169] In an embodiment of the present application, by adjusting the supplementary lighting period of the photosensitive chip in the subsequent frame based on the second target region in the second image obtained from the current frame, the image effect captured by the camera module can be improved.

[0170] Figure 7 is a schematic flowchart of a method 700 for controlling the supplementary lighting time of a camera module provided in an embodiment of the present application.

[0171] Figure 7 The method 700 shown can be executed by Figure 1 the camera module in the camera 130 in the vehicle 100 in, or, the method 700 can also be executed by Figure 2 the camera module in the camera 155 or the sensor 153 in the autonomous driving system in.

[0172] Among them, the camera module in the method 700 may include a photosensitive chip, an infrared light source, and a rolling shutter. For example, the camera module in the method 700 may be as shown in Figure 3 the camera module 300 in.

[0173] For example, the camera module in the method 700 can be used for an in-vehicle driver monitoring camera. This driver monitoring camera can be placed at the position behind the steering wheel or the position of the A-pillar of the vehicle, and is used for face recognition, fatigue detection, distraction detection, in-loop detection, etc. of the driver.

[0174] Figure 7 The method 700 shown may include steps 710 to 790. It should be understood that Figure 7 the method 700 shown is only an example and not a limitation. The method 700 may include more or fewer steps, which are not limited in the embodiments of the present application. The following will introduce these steps in detail respectively.

[0175] S710. During the exposure of the photosensitive chip, the infrared light source performs full-time supplementary lighting to obtain a first image.

[0176] For example, when shooting the first frame image in each second or when the camera module enters the reset state, the light source controller can control the infrared light source to provide supplementary light throughout the exposure period of the photosensitive chip to obtain the first image, so as to ensure that all areas in the first image can receive good illumination.

[0177] S720, perform face detection on the first image.

[0178] For example, the first image can be sent to the CPU / NPU. Correspondingly, the CPU / NPU can use deep learning algorithms or Haar operators to perform face detection on the first image.

[0179] If a face frame in the first image is detected, the face detection is successful, and S730 is executed.

[0180] If no face frame in the first image is detected (for example, as shown in part B of Figure 8 where the face area in the first image is blocked or there is no face area in the first image), the face detection fails, and S710 is executed. That is, when shooting the next frame image, the infrared light source provides supplementary light throughout the exposure period of the photosensitive chip to obtain an image with good illumination in all areas, and face detection is performed on the obtained image again.

[0181] S730, calculate the row of the photosensitive chip corresponding to the face frame.

[0182] Based on the face frame, the core area in the first image can be determined, and the row of the photosensitive chip corresponding to the core area in the first image can be calculated.

[0183] Optionally, a margin can be considered when determining the core area in the first image.

[0184] For example, a threshold can be preset, and a margin can be set when determining the core area in the first image, so that the distance between the face frame and the non-light area (i.e., the area in the first image that does not require supplementary light) is greater than or equal to the preset threshold.

[0185] As Figure 8 shown in part A of, the distance between the upper boundary of the face frame and the non-light area is greater than or equal to the preset threshold, and at the same time, the distance between the lower boundary of the face frame and the non-light area is greater than or equal to the preset threshold.

[0186] Among them, the core area is equivalent to Figure 4 the first target area in method 400 in, and specific reference can be made to the description of the embodiments in method 400, which will not be elaborated here.

[0187] S740, calculate the first supplementary light period corresponding to the row of the photosensitive chip.

[0188] Calculate the exposure period corresponding to the photosensitive chip row, that is, the first fill light period.

[0189] S750, during the exposure of the photosensitive chip, perform fill light within the determined fill light period to obtain a second image.

[0190] For example, the determined fill light period can be the first fill light period or the second fill light period.

[0191] S760, perform face detection on the second image.

[0192] As Figure 8 shown in part A, if a face frame in the second image is detected and the distance between the face frame and the non-light area (i.e., the area in the second image where fill light is not required) is greater than or equal to the preset threshold, then face detection is successful and S770 is executed.

[0193] As Figure 8 shown in part B, if no face frame in the second image is detected (for example, the face area in the second image is blocked or there is no face area in the second image), then face detection fails and belongs to failure type A, and S780 is executed to determine whether the number of face detection failures is less than 10 times.

[0194] As Figure 8 shown in part C, if a face frame in the second image is detected but the distance between the face frame and the non-light area (i.e., the area in the second image where fill light is not required) is less than the preset threshold, then face detection fails and belongs to failure type B, and S790 is executed.

[0195] S770, continue to capture the next frame of image based on the first fill light period.

[0196] During the process of capturing subsequent frames of images, the first fill light period can be maintained, that is, during the process of capturing subsequent frames of images, control the infrared light source to continue to perform fill light within the first fill light period corresponding to the photosensitive chip row.

[0197] S780, determine whether the number of face detection failures is less than 10 times.

[0198] If no face frame in the first image is detected and the number of face detection failures is less than 10 times, during the process of capturing subsequent frames of images, control the infrared light source to continue to perform fill light within the first fill light period corresponding to the photosensitive chip row; otherwise, when the number of face detection failures is greater than or equal to 10 times (or reset the first fill light period regularly per second), then S710 is executed.

[0199] S790. Determine a second fill light period based on the second image.

[0200] Perform face detection on the second image to obtain a face bounding box of the second image, calculate the photosensitive chip row corresponding to the face bounding box of the second image, and calculate the second fill light period corresponding to this photosensitive chip row (the photosensitive chip row corresponding to the face bounding box of the second image).

[0201] Subsequently, execute S750, and during the process of capturing images of subsequent frames, control the infrared light source to perform fill light within the second fill light period corresponding to the photosensitive chip row.

[0202] Figure 9 It is a schematic flowchart of method 900 for controlling the fill light time of a camera module provided by an embodiment of the present application.

[0203] Figure 9 The method 900 shown can be executed by Figure 1 the camera module in camera 130 of vehicle 100 in Figure 2 or, method 900 can also be executed by

[0204] the camera module in camera 155 or sensor 153 in the autonomous driving system in Figure 3 as shown by camera module 300 in

[0205] For example, the camera module in method 900 can be used for an in-vehicle cockpit monitoring camera. This cockpit monitoring camera can be placed below the rearview mirror inside the vehicle and is used for behavior recognition, gesture recognition, and detection of left-behind items for the driver and co-driver in the cockpit.

[0206] Figure 9 The method 900 shown can include steps 910 to 990. It should be understood that Figure 9 the method 900 shown is only an example and not a limitation. Method 900 can include more or fewer steps, which are not limited in the embodiments of the present application. The following will introduce these steps in detail respectively.

[0207] S910. During the exposure of the photosensitive chip, the infrared light source performs full-time fill light to obtain a first image.

[0208] For example, when capturing the first frame image per second or when the camera module enters the reset state, the light source controller can control the infrared light source to perform full-time fill light during the exposure of the photosensitive chip to obtain the first image, so as to ensure that all areas in the first image can obtain good illumination.

[0209] S920: Perform human body detection on the first image.

[0210] For example, the first image may be sent to the CPU / NPU, and accordingly, the CPU / NPU may use a deep learning algorithm or a Haar operator to perform human body detection on the first image.

[0211] If the human body frame in the first image is detected, the human body detection is successful, and S930 is executed.

[0212] If no human frame is detected in the first image (for example, Figure 10 As shown in part B in the figure, if the human body area in the first image is blocked or there is no human body area in the first image, human body detection fails and S910 is executed. That is, when shooting the next frame of image, the infrared light source fills the light during the exposure of the photosensitive chip to obtain an image with good lighting in all areas, and human body detection is performed again on the obtained image.

[0213] S930, calculating the photosensitive chip row corresponding to the human body frame.

[0214] Based on the human body frame, a core area in the first image can be determined, and a photosensitive chip row corresponding to the core area in the first image can be calculated.

[0215] Optionally, a margin may be taken into account when determining the core area in the first image.

[0216] For example, a threshold may be preset, and a margin may be set when determining the core area in the first image so that the distance between the human body frame and the dark area (ie, the area in the first image that does not require fill light) is greater than or equal to the preset threshold.

[0217] like Figure 10 As shown in part A, the distance between the upper boundary of the human body frame and the dark area is greater than or equal to a preset threshold, and at the same time, the distance between the lower boundary of the human body frame and the dark area is greater than or equal to the preset threshold.

[0218] The core area is equivalent to Figure 4 For details of the first target area in method 400, please refer to the description of the embodiment in method 400, which will not be repeated here.

[0219] S940, calculating the first fill light period corresponding to the photosensitive chip row.

[0220] The exposure period corresponding to the photosensitive chip row is calculated, that is, the first fill light period.

[0221] S950. During the exposure of the photosensitive chip, perform supplementary lighting within a determined supplementary lighting period to obtain a second image.

[0222] For example, the determined supplementary lighting period can be the first supplementary lighting period or the second supplementary lighting period.

[0223] S960. Perform human detection on the second image.

[0224] As Figure 10 shown in part A, if the human body frame in the first image is detected and the distance between the human body frame and the non-light area (i.e., the area in the first image that does not require supplementary lighting) is greater than or equal to the preset threshold, then the human detection is successful and S970 is executed.

[0225] As Figure 10 shown in part B, if the human body frame in the first image is not detected (for example, the human body area in the first image is blocked or there is no human body area in the first image), then the human detection fails and S980 is executed to determine whether the number of times of human detection failure is less than 10 times.

[0226] As Figure 10 shown in part C, if the human body frame in the first image is detected, but the distance between the human body frame and the non-light area (i.e., the area in the first image that does not require supplementary lighting) is less than the preset threshold, then the human detection fails and S990 is executed.

[0227] S970. Continue to capture the next frame of image based on the first supplementary lighting period.

[0228] During the process of capturing subsequent frames of images, the first supplementary lighting period can be maintained, that is, during the process of capturing subsequent frames of images, control the infrared light source to continue to perform supplementary lighting within the first supplementary lighting period corresponding to the photosensitive chip row.

[0229] S980. Determine whether the number of times of human detection failure is less than 10 times.

[0230] If the human body frame in the first image is not detected and the number of times of human detection failure is less than 10 times, during the process of capturing subsequent frames of images, control the infrared light source to continue to perform supplementary lighting within the first supplementary lighting period corresponding to the photosensitive chip row; otherwise, when the number of times of human detection failure is greater than or equal to 10 times (or reset the first supplementary lighting period regularly per second), then S910 is executed.

[0231] S990. Determine the second supplementary lighting period based on the second image.

[0232] Perform human detection on the second image to obtain the human bounding box of the second image, calculate the photosensitive chip row corresponding to the human bounding box of the second image, and calculate the second fill light period corresponding to this photosensitive chip row (the photosensitive chip row corresponding to the human bounding box of the second image).

[0233] Subsequently, execute S950, and during the process of capturing images of subsequent frames, control the infrared light source to perform fill light within the second fill light period corresponding to the photosensitive chip row.

[0234] Figure 11 It is a schematic block diagram of a device 1100 for controlling the fill light time of a camera module provided by an embodiment of the present application. It should be understood that Figure 11 The shown device 1100 for controlling the fill light time of the camera module is only an example, and the device 1100 of the embodiment of the present application may further include other modules or units. It should be understood that the device 1100 can execute Figure 4 、 Figure 7 and Figure 9 The respective steps in the method, and for the sake of avoiding repetition, will not be elaborated here.

[0235] Among them, the camera module may include a camera, and the camera may include a photosensitive chip, an infrared light source, and a rolling shutter. For example, the camera module may be as shown by the camera module 300 in Figure 3 .

[0236] In a possible implementation manner of the embodiment of the present application, the device 1100 for controlling the fill light time of the camera module may include:

[0237] A first determination unit 1110, configured to determine a first target area in a first image captured by the camera before the current frame, where the first target area is an area in the first image that needs to be filled with light;

[0238] A second determination unit 1120, configured to determine a first exposure period of a first target photosensitive chip row in the current frame according to the first target area, where the first target photosensitive chip row refers to the chip row in the photosensitive chip that is used to generate the image content in the first target area;

[0239] An indication unit 1130, configured to, when the photosensitive chip is exposed in the current frame, indicate the infrared light source to perform fill light according to the first exposure period.

[0240] Among them, the first determination unit and the first determination unit may be the same module or unit, and the present application embodiment does not limit this.

[0241] Optionally, the first determination unit is specifically configured to: determine the first target area in the first image according to a preset target object.

[0242] Optionally, the first target area is the face area in the first image.

[0243] Optionally, the photosensitive chip includes a plurality of photosensitive chip rows, and a plurality of pixel rows in the first image correspond to the plurality of photosensitive chip rows; wherein, the second determination unit 1120 is specifically configured to:

[0244] Determine the first target photosensitive chip row corresponding to the pixel row in the first target area in the first image; determine the first exposure period of the first target photosensitive chip row in the current frame.

[0245] Optionally, the first determination unit 1110 is further configured to: determine a second target area in the second image acquired in the current frame, where the second target area is the area in the second image that needs to be filled with light; the second determination unit 1120 is further configured to: determine the second exposure period of the second target photosensitive chip row in a subsequent frame according to the second target area, where the second target photosensitive chip row refers to the chip row in the photosensitive chip for generating the image content in the second target area; the indication unit 1130 is further configured to: when exposing the photosensitive chip in a subsequent frame, instruct the infrared light source to fill with light according to the second exposure period.

[0246] Optionally, the indication unit 1130 is further configured to: determine the third exposure period of the first target photosensitive chip row in a subsequent frame according to the first exposure period; when exposing the photosensitive chip in a subsequent frame, instruct the infrared light source to fill with light according to the second exposure period.

[0247] It should be understood that the device 1100 for controlling the light filling time of the camera module is embodied in the form of a functional module. The term "module" here can be implemented in software and / or hardware forms, and no specific limitation is made thereto. For example, a "module" can be a software program, a hardware circuit, or a combination of the two for implementing the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group of processors, etc.) for executing one or more software or firmware programs, a memory, a merged logic circuit, and / or other suitable components supporting the described functions.

[0248] As an example, the device 1100 for controlling the light supplement time of the camera module provided in the embodiments of the present application may be the camera module in the autonomous driving system, or may be the camera module configured in the vehicle head unit, or may also be the vehicle head unit (or processor) in the autonomous driving vehicle, or may also be the chip configured in the vehicle head unit for executing the method described in the embodiments of the present application.

[0249] Figure 12 FIG. 4 is a schematic block diagram of a device 800 for controlling the light supplement time of the camera module according to an embodiment of the present application. Figure 12 The device 800 shown includes a memory 801, a processor 802, a communication interface 803, and a bus 804. Among them, the memory 801, the processor 802, and the communication interface 803 are communicatively connected to each other through the bus 804.

[0250] The memory 801 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 801 may store a program. When the program stored in the memory 801 is executed by the processor 802, the processor 802 is used to execute each step of the method for controlling the light supplement time of the camera module in the embodiments of the present application. For example, it may execute Figure 4 , Figure 7 and Figure 9 each step of the embodiments shown.

[0251] The processor 802 may adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits for executing relevant programs to implement the method for controlling the light supplement time of the camera module in the method embodiments of the present application.

[0252] The processor 802 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the method for controlling the light supplement time of the camera module in the embodiments of the present application may be completed by the hardware integrated logic circuit or the instructions in software form in the processor 802.

[0253] The processor 802 may also be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0254] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 801, and the processor 802 reads the information in the memory 801, and combines its hardware to complete the functions required to be executed by the units included in the device for controlling the fill light time of the camera module in the embodiment of the present application, or execute the method for controlling the fill light time of the camera module in the method embodiment of the present application, for example, it can be executed Figure 4 , Figure 7 and Figure 9 The various steps / functions of the illustrated embodiment.

[0255] The communication interface 803 may use, but is not limited to, a transceiver or other transceiver device to implement communication between the apparatus 800 and other devices or a communication network.

[0256] The bus 804 may include a path for transmitting information between various components of the device 800 (eg, the memory 801 , the processor 802 , and the communication interface 803 ).

[0257] It should be understood that the device 800 shown in the embodiment of the present application can be a camera module in an autonomous driving system, or it can also be a camera module configured in a vehicle computer, or it can also be a vehicle computer (or processor) in an autonomous driving vehicle, or it can also be a chip configured in a vehicle computer, for executing the method described in the embodiment of the present application.

[0258] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0259] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0260] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0261] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship. The specific meaning can be understood by referring to the context.

[0262] In the present application, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0263] It should be understood that in various embodiments of the present application, the order of the serial numbers of the above processes does not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0264] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0265] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0266] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0267] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0268] In addition, the functional units in each embodiment of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0269] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0270] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for controlling the fill light time of a camera module, characterized in that, the camera module includes a camera, the camera includes a photosensitive chip, the camera is a cockpit monitoring camera or a driver monitoring camera, and the method includes: performing face detection or human body detection on a first image captured before the current frame; when a face or a human body is detected in the first image, determining a first target area in the first image, the first target area being the area in the first image that needs fill light, the first target area including the face area or the human body area in the first image, and the distance between the face area or the human body area and the lightless area in the first image being greater than or equal to a preset threshold; determining a first exposure period of a first target photosensitive chip row in the current frame according to the first target area, the first target photosensitive chip row referring to the chip row in the photosensitive chip for generating the image content in the first target area; when exposing the photosensitive chip in the current frame, instructing an infrared light source to perform fill light according to the first exposure period; when no face or human body is detected in the first image, instructing the infrared light source to perform full-time fill light during the exposure process of the current frame.

2. The method according to claim 1, characterized in that, the method further includes: performing face detection or human body detection on a second image obtained in the current frame; when a face or a human body is detected in the second image, and the distance between the face area or the human body area in the second image and the lightless area in the second image is less than the preset threshold, determining a second target area in the second image, the second target area being the area in the second image that needs fill light; determining a second exposure period of a second target photosensitive chip row in a subsequent frame according to the second target area, the second target photosensitive chip row referring to the chip row in the photosensitive chip for generating the image content in the second target area; when exposing the photosensitive chip in a subsequent frame, instructing an infrared light source to perform fill light according to the second exposure period.

3. The method according to claim 1, characterized in that, the method further includes: performing face detection or human body detection on a second image obtained in the current frame; when a face or a human body is detected in the second image, and the distance between the face area or the human body area in the second image and the lightless area in the second image is greater than or equal to the preset threshold, determining a third exposure period of the first target photosensitive chip row in a subsequent frame according to the first exposure period; when exposing the photosensitive chip in a subsequent frame, instructing an infrared light source to perform fill light according to the third exposure period.

4. The method according to claim 1, characterized in that, the method further includes: performing face detection or human body detection on a second image obtained in the current frame; When the number of times that no face or human body is detected in the second image is less than or equal to N times, determine the third exposure period of the first target photosensitive chip row in subsequent frames according to the first exposure period; when exposing the photosensitive chip in subsequent frames, instruct the infrared light source to perform supplementary lighting according to the third exposure period, where N is an integer greater than 1; When the number of times that no face or human body is detected in the second image is greater than N times, instruct the infrared light source to perform full-time supplementary lighting during the exposure process of the subsequent frames.

5. The method according to any one of claims 1 to 4, characterized in that the photosensitive chip includes a plurality of photosensitive chip rows, and a plurality of pixel rows in the first image correspond to the plurality of photosensitive chip rows; wherein, the determining the first exposure period of the first target photosensitive chip row in the current frame according to the first target area includes: determining the first target photosensitive chip row corresponding to the pixel row in the first target area in the first image; determining the first exposure period of the first target photosensitive chip row in the current frame.

6. An apparatus for controlling the supplementary lighting time of a camera module, characterized in that the camera module includes a camera, the camera includes a photosensitive chip, the camera is a cockpit monitoring camera or a driver monitoring camera, and the apparatus includes: a first determining unit, configured to determine a first target area in the first image when a face or a human body is detected in the first image captured by the camera before the current frame, the first target area being an area in the first image that needs supplementary lighting, the first target area including a face area or a human body area in the first image, and the distance between the face area or the human body area and the lightless area in the first image being greater than or equal to a preset threshold; a second determining unit, configured to determine the first exposure period of the first target photosensitive chip row in the current frame according to the first target area, the first target photosensitive chip row referring to the chip row in the photosensitive chip for generating the image content in the first target area; an instructing unit, configured to instruct the infrared light source to perform supplementary lighting according to the first exposure period when exposing the photosensitive chip in the current frame; the instructing unit is further configured to instruct the infrared light source to perform full-time supplementary lighting during the exposure process of the current frame when no face or human body is detected in the first image.

7. The apparatus according to claim 6, characterized in that the first determining unit is further configured to determine a second target area in the second image when a face or a human body is detected in the second image obtained in the current frame, and the distance between the face area or the human body area in the second image and the lightless area in the second image is less than the preset threshold, the second target area being an area in the second image that needs supplementary lighting; the second determining unit is further configured to determine the second exposure period of the second target photosensitive chip row in subsequent frames according to the second target area, the second target photosensitive chip row referring to the chip row in the photosensitive chip for generating the image content in the second target area; The indicating unit is further configured to, when exposing the photosensitive chip in subsequent frames, instruct the infrared light source to perform supplementary lighting according to the second exposure period.

8. The apparatus according to claim 6, wherein: The second determining unit is further configured to, when a face or a human body is detected in the second image obtained in the current frame, and the distance between the face area or the human body area in the second image and the lightless area in the second image is less than the preset threshold, determine a second target area in the second image, where the second target area is the area in the second image that needs supplementary lighting; The indicating unit is further configured to, when exposing the photosensitive chip in subsequent frames, instruct the infrared light source to perform supplementary lighting according to the third exposure period.

9. The apparatus according to claim 6, wherein: The second determining unit is further configured to, when the number of times that no face or human body is detected in the second image obtained in the current frame is less than or equal to N times, determine the third exposure period of the first target photosensitive chip row in subsequent frames according to the first exposure period; and the indicating unit is further configured to, when exposing the photosensitive chip in subsequent frames, instruct the infrared light source to perform supplementary lighting according to the third exposure period, where N is an integer greater than 1; or The indicating unit is further configured to, when the number of times that no face or human body is detected in the second image is greater than N times, instruct the infrared light source to perform full-course supplementary lighting during the exposure process of the subsequent frames.

10. The apparatus according to any one of claims 6 to 9, wherein: The photosensitive chip includes a plurality of photosensitive chip rows, and the plurality of pixel rows in the first image correspond to the plurality of photosensitive chip rows; wherein, the second determining unit is specifically configured to: determine the first target photosensitive chip row corresponding to the pixel row in the first target area in the first image; determine the first exposure period of the first target photosensitive chip row in the current frame.

11. A camera module, wherein: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method according to any one of claims 1 to 5.

12. A vehicle, wherein: The vehicle includes the apparatus according to any one of claims 6 to 10 or the camera module according to claim 11.

13. A computer-readable storage medium, wherein: Program instructions are stored in the computer-readable storage medium, and when the program instructions are run by a processor, the method according to any one of claims 1 to 5 is implemented.

14. A chip, wherein: The chip includes a processor and a data interface, and the processor reads instructions stored on a memory through the data interface to execute the method according to any one of claims 1 to 5.

Images