Exposure control system

Abstract

To achieve highly accurate exposure control while preventing an increase in the number of manufacturing steps for an apparatus.SOLUTION: An exposure control system according to the present invention includes an exposure control unit that obtains a multiplication value between an exposure time and a gain from a parameter based on an image captured by an imaging unit using an exposure control model that is trained through reinforcement learning and that receives inputs as parameters including the exposure time of the image captured by the imaging unit, a gain serving for adjusting brightness when an image is captured by the imaging unit, an average luminance value of a region of interest in the image, a target luminance value, and a difference between the average luminance value of the region of interest and the target luminance value.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an exposure control system. [Background technology]

[0002] BACKGROUND ART Conventionally, there has been known a technology that uses an image of a target area captured by an imaging unit such as a camera or an image sensor to detect objects captured within the target area, the posture of the object, the surrounding situation, etc., and uses the detected images for situation understanding, control, etc.

[0003] For example, Patent Document 1 discloses a technology for detecting a driver's condition by generating an image that improves the probability of detecting the driver's condition by combining an automatically corrected image from the imaging device with an image that has been corrected again, in order to avoid a decrease in the accuracy of detecting the driver's condition due to external light such as streetlights.

[0004] Furthermore, Patent Document 2 discloses a technology that uses a machine learning model to infer the position and brightness of a high-brightness subject based on training data, and calculates correction parameters from the inferred values, thereby achieving appropriate exposure control for extremely bright images, such as those in which sunlight is directly incident.

[0005] Furthermore, Patent Document 3 discloses a technology in which a surveillance camera is equipped with artificial intelligence to detect people and vehicles and learn image data to estimate the position, brightness, and speed of faces and vehicles, and determine the camera parameters of the surveillance camera to prevent a deterioration in surveillance accuracy. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2019-046141 [Patent Document 2] Japanese Patent Publication No. 2022-154658 [Patent Document 3] Japanese Patent Publication No. 2021-118478 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the above-mentioned conventional techniques have the following problems. The technology of Patent Document 1 performs correction based on the brightness difference of the acquired image, while the technology of Patent Document 2 uses machine learning to estimate changes in the external environment, such as the position and brightness of the subject, and utilizes artificial intelligence for object detection. In other words, these conventional technologies do not infer or calculate control parameters (e.g., exposure time, correction amount, etc.) for various imaging environments of the image sensor or camera. Therefore, it is difficult to perform exposure control with high precision using these conventional technologies.

[0008] Furthermore, the technology of Patent Document 2 infers the position and brightness of a highly bright subject through supervised learning, while the technology of Patent Document 3 detects people and vehicles and learns image data to estimate the position, brightness, and movement speed of faces and vehicles. For this reason, these conventional technologies require an enormous amount of time for processing, learning, and manipulating image data, and require the preparation of a large number of data sets of correct values ​​for input, which increases the manufacturing process of the device.

[0009] The present invention has been made in view of the above, and has as its object to provide an exposure control system that can achieve highly accurate exposure control while preventing an increase in the number of manufacturing processes for the device. [Means for solving the problem]

[0010] The exposure control system according to the present invention includes an exposure control unit that calculates the multiplication value of the exposure time and the gain from the parameters based on the image captured by the imaging unit using an exposure control model that is a trained model that is trained by reinforcement learning and that takes as input parameters including the exposure time of an image captured by the imaging unit, a gain that adjusts brightness when capturing an image by the imaging unit, an average luminance value of a region of interest in the image, a target luminance value, and the difference between the average luminance value and the target luminance value of the region of interest, and outputs the multiplication value of the exposure time and the gain. [Effects of the Invention]

[0011] The exposure control system according to the present invention can achieve highly accurate exposure control while preventing an increase in the number of manufacturing steps for the device. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a vehicle control system according to an embodiment. [Figure 2] FIG. 2 is a schematic diagram showing an example of how an image of a passenger seated in a front seat in a vehicle according to an embodiment is captured and an example of the arrangement of an image capturing unit. [Figure 3] FIG. 3 is a block diagram showing an example of the functional configuration of the exposure control device according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram illustrating an example of the configuration of an exposure control model according to an embodiment. [Figure 5] FIG. 5 is a diagram showing an example of the relationship between exposure time*gain, type, range of variation, and dimension in the exposure control model according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of registered data in the basic database according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of a procedure of a learning process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The configurations of the embodiments described below, as well as the actions and effects brought about by the configurations, are merely examples and are not limited to the following description.

[0014] FIG. 1 is a diagram illustrating an example of the configuration of a vehicle control system 1 according to an embodiment. A vehicle control system 1 according to this embodiment is mounted on a vehicle. As shown in Fig. 1, the vehicle control system 1 mainly includes an imaging unit 22, an ECU (Electronic Control Unit) 26, a plurality of in-vehicle devices 34, and a plurality of output devices 32. The ECU 26, the plurality of in-vehicle devices 34, and the plurality of output devices 32 are connected via an in-vehicle network 30.

[0015] The in-vehicle network 30 is configured as, for example, a CAN (Controller Area Network). The in-vehicle network 30 is electrically connected to various sensors that are referenced when a device control unit 28 of the ECU 26 (described later) executes control.

[0016] The in-vehicle equipment 34 is, for example, various equipment mounted on a vehicle, such as a sunroof opening / closing motor, etc. The output device 32 is, for example, an output device such as a speaker, an indicator light, a display device, etc. The imaging unit 22 is a digital camera incorporating an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS image sensor), etc. The imaging unit 22 can sequentially output moving image data (captured image data) at a predetermined frame rate.

[0017] FIG. 2 is a schematic diagram showing an example of how an image of a passenger 18 seated in a front row seat 14 (seat 14a) in a vehicle 10 according to the embodiment is captured and an example of the arrangement of the image capturing unit 22. In FIG.

[0018] The front row seat 14 (seat 14a) is composed of a seat cushion 14c, a seat back 14d, a headrest 14e, etc., and a frame supporting the seat cushion 14c is fixed to the floor of the passenger compartment 10a by rails (not shown) so that its position can be adjusted in the fore-and-aft direction of the vehicle. The front row seat 14 is also provided with various adjustment mechanisms such as a reclining mechanism, and is configured to allow adjustments to be made so that the posture of a passenger 18 (driver) wearing a seat belt 20 and gripping the steering wheel 12 can be easily adjusted to maintain a position suitable for driving.

[0019] The imaging unit 22 is disposed in a position from which the interior of the vehicle compartment 10a can be viewed in order to detect (recognize) the situation inside the vehicle compartment 10a. In this embodiment, the imaging unit 22 is disposed in approximately the center of the upper inside portion of the front windshield of the vehicle 10 (for example, near the rearview mirror). Note that the imaging unit 22 may be disposed in another position as long as it is in a position from which the interior of the vehicle compartment 10a can be viewed.

[0020] The imaging unit 22 is fixed with its viewing angle and posture adjusted so that its imaging area includes at least the passengers 18 seated in the front row seats 14 and the rear row seats (not shown). For example, as shown in FIG. 2, when a passenger 18 is seated in seat 14a (driver's seat), the imaging area of ​​the imaging unit 22 includes the face and at least the upper body of the passenger 18.

[0021] Returning to FIG. 1, the ECU 26 controls the imaging unit 22 to capture images, and also acquires the images captured by the imaging unit 22 to perform various controls such as exposure control, monitoring control, and vehicle control.

[0022] As shown in FIG. 1, the ECU 26 includes a device control unit 28, a CPU (Central Processing Unit) 38, a ROM 40 (Read Only Memory), a RAM 42 (Random Access Memory), an SSD 44 (Solid State Drive, flash memory), and the like.

[0023] The CPU 38 can read a program installed and stored in a nonvolatile storage device such as the ROM 40, and execute arithmetic processing in accordance with the program. The CPU 38 according to this embodiment implements the exposure control device 100, etc., by reading a program installed and stored in a storage device such as the ROM 40 and executing the program.

[0024] The RAM 42 temporarily stores various data used in the calculations performed by the CPU 38 . The SSD 44 is a rewritable non-volatile storage unit, and can store data even when the power supply to the ECU 26 is turned off.

[0025] The device control unit 28 controls an output device 32 and in-vehicle devices 34 electrically connected via an in-vehicle network 30 .

[0026] The device control unit 28, CPU 38, ROM 40, RAM 42, etc. may be integrated in the same package. The ECU 26 may be configured to use other logic operation processors, logic circuits, etc., such as a DSP (Digital Signal Processor), instead of the CPU 38. The SSD 44 may be replaced by an HDD (Hard Disk Drive), or the SSD 44 and HDD may be provided separately from the ECU 26.

[0027] Next, the exposure control device 100 that functions as a result of the CPU 38 executing a program will be described in detail. 3 is a block diagram showing an example of the functional configuration of the exposure control device 100 according to the embodiment, and also shows an SSD 44 as a storage unit.

[0028] 3, the exposure control device 100 according to this embodiment includes an exposure control unit 120 and a learning unit 110. The SSD 44 also stores a basic database 111 (hereinafter referred to as "basic DB 111") and an exposure control model 112. The exposure control device 100 is an example of an exposure control system.

[0029] The exposure control unit 120 performs exposure control for the imaging unit 22. Specifically, the exposure control unit 120 performs exposure control by using the exposure control model 112 stored in the SSD 44 to determine the multiplication value of the exposure time and the gain. That is, the exposure control unit 120 determines the six parameters based on the captured image input from the imaging unit 22. The exposure control unit 120 then inputs the determined six parameters to the exposure control model 112, and sets the multiplication value of the exposure time and the gain output from the exposure control model 112 as a new exposure parameter for the imaging unit 22 to perform exposure control.

[0030] Here, the parameters are parameters for the imaging unit 22. The parameters are six parameters including an exposure time, a gain, an average luminance value of a region of interest (ROI) of an image, a target luminance value, absolute ambient brightness information, and a difference between the ROI average luminance value and the target luminance value.

[0031] The exposure time is the exposure time for an image captured by the imaging unit 22. The gain is a parameter for adjusting the brightness when the imaging unit 22 captures an image.

[0032] The absolute ambient brightness information is a measure of the absolute brightness in the actual environment. The image output from the imaging unit 22 is an image adjusted by the exposure time and gain for the actual environment, and is input to the exposure control model 112. For this reason, the exposure control model 112 does not know what the actual lighting environment is like, and for example, even if the actual lighting environment is a dark environment at night, the input image may be a bright image adjusted by the exposure time and gain. For this reason, the exposure control model 112 may erroneously output an inference result of the appropriate exposure parameters (exposure time * gain) for the actual lighting environment.

[0033] For this reason, in this embodiment, a parameter called absolute environmental brightness information, which is a measure of absolute brightness in the actual environment, is used to accurately infer the exposure parameter (exposure time * gain) by the exposure control model 112. That is, by inputting absolute environmental brightness information as a parameter to the exposure control model 112, the exposure control model 112 is able to recognize the actual lighting environment conditions in addition to the image adjusted by the exposure time and gain, and is configured to calculate and output a more accurate exposure time * gain.

[0034] In this embodiment, as an example, the parameter shown in the following equation (1) is used as the absolute ambient brightness information.

[0035] Absolute ambient brightness information = average brightness of output image / (exposure time * gain) (1) However, the absolute ambient brightness information is not limited to this, and can be set arbitrarily as long as it is a parameter that indicates the scale of absolute brightness in the actual environment.

[0036] The exposure control model 112 stored in the SSD 44 is a trained model trained by the training unit 110 (described later) through deep reinforcement learning. The exposure control model 112 receives the six parameters and outputs a plurality of multiplication values ​​of the exposure time and the gain. In this embodiment, the exposure control model 112 outputs 58 multiplication values ​​of the exposure time and the gain (i.e., exposure time*gain) and their scores (Q values).

[0037] FIG. 4 is a schematic diagram showing an example of the configuration of the exposure control model 112 according to the embodiment. As shown in FIG. 4, the exposure control model 112 is a trained DNN (Deep Near Network) model that includes an input layer, two intermediate layers 1 and 2, and an output layer.

[0038] The input layer inputs the six parameters mentioned above. Hidden layer 1 inputs 6 parameters from the input layer and outputs 100 parameters to hidden layer 2. Hidden layer 2 inputs 100 parameters from hidden layer 1 and outputs 58 parameters to the output layer. The output layer outputs 58 multiplication values ​​of the exposure time and the gain (denoted as "exposure time * gain") along with the Q value of the multiplication value. Here, the Q value is called the score.

[0039] The exposure control model 112 considers the amount of change in the product of the exposure time and the gain through the intermediate layers 1 and 2, and when the environmental change is small, sets a small limit range of the amount of change (called "Interval") and outputs the product of the exposure time and the gain from the output layer to make finer adjustments to the environment. For example, Interval = 1.

[0040] On the other hand, when the environmental change is large, the exposure control model 112 sets a large limit range (Interval) of the amount of change and outputs the product of the exposure time and the gain from the output layer so that the product of the exposure time and the gain can be adjusted in the shortest number of times in response to the environmental change. For example, Interval = 200.

[0041] FIG. 5 is a diagram showing an example of the relationship between exposure time*gain, type, range of variation, and dimension in the exposure control model according to the embodiment. 5, in the example of the exposure control model 112 of this embodiment, a plurality of limit ranges (intervals) of the amount of change are set according to the magnitude of the environmental change, and the larger the interval, the larger the range of the amount of change that is set. Furthermore, the number of actions is determined according to the interval. The number of actions is the number of output patterns of the exposure control model 112.

[0042] Returning to FIG. 4, the learning unit 110 will now be described in detail. The learning unit 110 learns and constructs the exposure control model 112 . The learning unit 110 mainly includes an environment simulator 101, an evaluation unit 103, an inference unit 104, and a state change unit .

[0043] The environment simulator 101 simulates the digital camera and image sensor that constitute the imaging unit 22 in a learning environment, and is configured as a model that reproduces the exposure control model 112. The environment simulator 101 receives the above-mentioned six parameters, including the changed exposure time and gain, output from the state change unit 106, which will be described later. The environment simulator 101 then refers to the basic DB 111 stored in the SSD 44 to acquire the ROI average luminance value corresponding to the changed exposure time and gain.

[0044] The basic DB 111 is a database group in which exposure times, gains, and ROI average luminance values ​​collected under various environmental patterns are associated and registered. Specifically, the exposure time and gain are gradually changed, and the moving images captured by the imaging unit 22 are analyzed to obtain ROI average luminance values ​​as analysis data. The ROI average luminance values ​​are associated with all combinations of exposure time and gain and registered in the basic DB 111 in advance.

[0045] FIG. 6 is a diagram illustrating an example of registered data in the basic DB 111 according to the embodiment. As shown in FIG. 6, the basic DB 111 is made up of a group of databases (CSV) each of which has an ROI average luminance value associated with all combinations of exposure time and gain.

[0046] 4, the environment simulator 101 acquires an ROI average luminance value corresponding to the changed exposure time and gain from the basic DB 111. Then, the environment simulator 101 outputs the changed exposure time and gain (however, the exposure time and gain are the initial values ​​the first time), the ROI average luminance value at the exposure time and gain (i.e., the ROI average luminance value acquired from the basic DB 111), the target luminance value for control, and observation. Here, the environment simulator 101 sets the target luminance value to an arbitrary value among the six input parameters.

[0047] In this way, the environment simulator 101 focuses on the average brightness value of the ROI of the image, which is determined by the exposure time and gain, and simulates numerical information called parameters rather than simulating the image, thereby simplifying and reducing the processing load.

[0048] The evaluation unit 103 scores the behavior of the inference unit 104 as an agent based on the degree of reach to the target brightness value, whether the route is optimal, etc. Specifically, the evaluation unit 103 inputs the exposure time, gain, ROI average brightness value, and target brightness value, which are output as observations from the environment simulator 101, into a predetermined reward function to find the ROI average brightness value and target brightness value that maximize the output of the reward function. Then, the evaluation unit 103 outputs the target brightness value, the ROI average brightness value that maximizes the output of the reward function, the exposure time, and the gain to the inference unit 104.

[0049] The inference unit 104 receives the ROI average luminance value that maximizes the reward function, the target luminance value, the exposure time, and the gain, which are the evaluation results by the evaluation unit 103. The inference unit 104 also calculates the difference between the input ROI average luminance value and the target luminance value, and absolute environmental brightness information.

[0050] The inference unit 104 inputs and registers six parameters, including the ROI average luminance value that maximizes the reward function, the target luminance value, the exposure time, the gain, the difference between the calculated ROI average luminance value and the target luminance value, and absolute environmental brightness information, into the exposure control model 112. Then, the inference unit 104 acquires and outputs, as inferred values, multiple exposure times*gains (multiplication values ​​of exposure time and gain) and their scores (i.e., Q values) after the next change to be set from the exposure control model 112. The inference unit 104 is also referred to as an agent.

[0051] The state change unit 106 selects, as an action, the exposure time and gain that maximizes the score (Q value) from among the multiple exposure times*gains (multiplication of the exposure time and the gain) output as inference values ​​from the inference unit 104, and inputs the selected exposure time*gain to the environment simulator 101 as the changed exposure time and gain (i.e., the next exposure time and gain) together with the other four parameters.

[0052] The environment simulator 101 simulates the imaging unit 22 using the other four parameters as the next exposure time and gain input in this way, and outputs the changed exposure time and gain, the ROI average luminance value at the changed exposure time and gain, and the target luminance value for control, as described above.

[0053] Next, the learning process performed by the learning unit 110 of the exposure control device 100 according to this embodiment configured as above will be described. FIG. 7 is a flowchart illustrating an example of a procedure of a learning process according to the embodiment.

[0054] First, the state change unit 106 selects the pattern with the highest score (Q value) from among multiple patterns of exposure time*gain as the behavior inferred from the exposure control model 112 (S101). Next, the state change unit 106 outputs six parameters to the environment simulator 101: the post-change exposure time and post-change gain calculated from the selected highest-score exposure time*gain, the current ROI average luminance value, the target luminance value, absolute environment brightness information, and the difference between the ROI average luminance value and the target luminance value (S102).

[0055] The environment simulator 101 receives the post-change exposure time and post-change gain, the current ROI average luminance value, the target luminance value, absolute environment brightness information, and the difference between the ROI average luminance value and the target luminance value from the state change unit 106, and acquires the ROI average luminance value corresponding to the post-change exposure time and the post-change gain in the basic DB 111. Then, the environment simulator 101 outputs the post-change exposure time and gain (which are the initial values ​​in the first case), the ROI average luminance value acquired from the basic DB 111, and the control target luminance value as observations (S103).

[0056] Next, the evaluation unit 103 acquires from the environment simulator 101 the exposure time and gain after the change (initial values ​​if this is the first time) as an observation, the ROI average luminance value acquired from the basic DB 111, and the control target luminance value. Then, the evaluation unit 103 scores (evaluates) the behavior of the inference unit 104 as an agent based on the target luminance value and the ROI average luminance value (S104). Specifically, the evaluation unit 103 inputs the exposure time, gain, ROI average luminance value, and target luminance value into a reward function, and finds the ROI average luminance value and target luminance value that maximize the output of the reward function. Then, the evaluation unit 103 outputs the target luminance value, the ROI average luminance value that maximizes the output of the reward function, the exposure time, and the gain to the inference unit 104.

[0057] Next, the inference unit 104 receives the target brightness value, the ROI average brightness value that maximizes the output of the reward function, the exposure time, and the gain from the evaluation unit 103. The inference unit 104 then calculates the difference between the input ROI average brightness value and the target brightness value, as well as absolute environment brightness information (S105). The inference unit 104 then inputs and registers six parameters, including the ROI average brightness value that maximizes the reward function, the target brightness value, the exposure time, the gain, the difference between the calculated ROI average brightness value and the target brightness value, and the absolute environment brightness information, into the exposure control model 112 (S106). The inference unit 104 then acquires, as an output of the exposure control model 112, multiple patterns of exposure time*gain (the product of the exposure time and the gain) to be set next, along with their scores (Q values), as inference values ​​(S107).

[0058] Next, the inference unit 104 determines whether learning of the exposure control model 112 is complete (S108). Specifically, the inference unit 104 determines whether the number of repetitions has reached the number of actions, which is the dimension corresponding to the above-mentioned exposure time*gain. If the number of repetitions has not reached the number of actions and learning of the exposure control model 112 has not been completed (S108: No), the process returns to S101. Then, the learning unit 110 repeatedly executes the processes from S101 to S107.

[0059] On the other hand, if the number of repetitions reaches the number of actions and the learning of the exposure control model 112 is completed (S108: Yes), the process ends.

[0060] As described above, the exposure control device 100 according to this embodiment is equipped with an exposure control unit 120 that calculates the multiplication value of the exposure time and the gain from parameters based on the image captured by the imaging unit 22, using an exposure control model 112 that is a trained model that is trained by reinforcement learning and that takes as input parameters including the exposure time of the image captured by the imaging unit 22, a gain that adjusts the brightness when capturing an image by the imaging unit 22, the ROI average luminance value of the image, a target luminance value, and the difference between the ROI average luminance value and the target luminance value, and that outputs the multiplication value of the exposure time and the gain.

[0061] That is, in this embodiment, the exposure control model 112 determines an exposure parameter for the imaging unit 22, which is a multiplication value of the exposure time and the gain, and exposure control is performed based on the determined exposure parameter. Furthermore, the exposure control model 112 is a trained model trained by reinforcement learning. Therefore, according to this embodiment, there is no need to use a large amount of training data, and exposure control is performed using parameters for the imaging unit 22, so that highly accurate exposure control can be achieved while preventing an increase in the manufacturing process of the device.

[0062] The exposure control device 100 according to this embodiment further includes a learning unit 110 that learns the exposure control model 112 . The learning unit 110 is equipped with an environment simulator 101, which simulates the imaging unit 22, inputs the above parameters, outputs an ROI average luminance value, an exposure time, a gain, and a target luminance value, and is a model that reproduces the exposure control model 112; an evaluation unit 103 that inputs the ROI average luminance value and the target luminance value output from the environment simulator 101 into a predetermined reward function to determine the ROI average luminance value and the target luminance value that maximize the output of the reward function; an inference unit 104 that inputs the above parameters, including the ROI average luminance value and the target luminance value, which are the evaluation results by the evaluation unit 103, into the exposure control model 112 and outputs multiple exposure times*gains as inferred values; and a state change unit 106 that selects, as an action, the exposure time and gain of the exposure time*gain that maximizes the score from the multiple exposure times*gains output as inferred values, and inputs parameters including the selected exposure time and gain into the environment simulator 101.

[0063] Therefore, according to this embodiment, exposure control is performed using the exposure control model 112, which is a trained model trained by reinforcement learning. This eliminates the need to prepare a large amount of training data, i.e., data on correct values ​​for input to the trained model, and also prevents an increase in storage capacity. Furthermore, in this embodiment, parameters including exposure time, gain, ROI average luminance value of the image, target luminance value, and difference between the ROI average luminance value and the target luminance value, rather than the image itself captured by the imaging unit 22, are input to the exposure control model 112. This makes it possible to simplify and reduce the weight of the control process compared to when image data is used as input. Therefore, according to this embodiment, it is possible to achieve more accurate exposure control while preventing an increase in the number of manufacturing processes for the device.

[0064] Furthermore, in the exposure control device 100 according to this embodiment, the above parameters include absolute ambient brightness information that indicates the scale of absolute brightness in the actual environment. Therefore, according to this embodiment, exposure control is performed taking into account the brightness of the actual environment, thereby achieving more accurate exposure control.

[0065] Furthermore, in the exposure control device 100 according to this embodiment, the environment simulator 101 acquires, from an image captured in advance by the imaging unit 22, the ROI average luminance value associated with the exposure time and the gain, among the input parameters, in the basic DB 111, which associates the exposure time, the gain, and the ROI average luminance value. Therefore, according to this embodiment, the basic DB 111 is constructed using numerical information instead of an image, and the imaging unit is simulated, which simplifies and lightens the processing, and further prevents the number of manufacturing steps for the device from increasing.

[0066] In this embodiment, six parameters are input to the exposure control model 112, but this is not limiting. For example, five parameters excluding the absolute ambient brightness information may be used as input to the exposure control model 112.

[0067] In addition, in this embodiment, the exposure control is described for the case where the imaging unit 22 is used to monitor the occupant 18 such as the driver, but the present invention is not limited to this. For example, the exposure control device 100 according to this embodiment can also be applied to the exposure control of an imaging unit used to monitor the periphery of the vehicle 10.

[0068] In addition, in this embodiment, the exposure control device 100 and the SSD 44 are provided separately, but this is not limiting. For example, the exposure control device 100 may be configured to include a storage unit that stores the basic DB 111 and the exposure control model 112.

[0069] Furthermore, in this embodiment, the exposure control device 100 mounted on the vehicle 10 is provided with the learning unit 110 in addition to the exposure control unit 120, but this is not limited to this. For example, the exposure control device 100 may be provided with only the exposure control unit 120, and the learning unit 110 may be configured as a separate learning device that includes the exposure control device 100 and the basic DB 111, and an exposure control system may be constructed that includes the exposure control device 100 and the learning device.

[0070] In this case, the learning device and the basic DB 111 are not mounted on the vehicle 10, and the exposure control model 112 learned by the learning device can be configured to be incorporated into the SSD 44 of the exposure control device 100 in the vehicle 10.

[0071] The exposure control program executed by the CPU 38 of this embodiment may be configured to be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0072] Furthermore, the exposure control program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the exposure control program executed in this embodiment may be provided or distributed via a network such as the Internet.

[0073] Although the embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0074] 1...vehicle control system, 10...vehicle, 10a...vehicle cabin, 22...imaging unit, 26...ECU, 38...CPU, 44...SSD, 100...exposure control device, 101...environment simulator, 103...evaluation unit, 104...inference unit, 106...state change unit, 110...learning unit, 111...basic DB, 112...120...exposure control unit

Claims

1. an exposure control unit that calculates a multiplication value of the exposure time and the gain from the parameters based on the image captured by the imaging unit, using an exposure control model that is a trained model that is trained by reinforcement learning and that has inputs including parameters including an exposure time of an image captured by the imaging unit, a gain that adjusts brightness when capturing an image by the imaging unit, an average luminance value of a region of interest in the image, a target luminance value, and a difference between the average luminance value and the target luminance value of the region of interest, and that outputs a multiplication value of the exposure time and the gain; An exposure control system comprising:

2. a learning unit that learns the exposure control model, The learning unit an environment simulator that simulates the imaging unit, inputs the parameters, and outputs the average luminance value of the region of interest, the exposure time, the gain, and the target luminance value, and is a model that reproduces the exposure control model; an evaluation unit that inputs the average brightness value of the region of interest and the target brightness value output from the environmental simulator into a predetermined reward function to determine the average brightness value of the region of interest and the target brightness value that maximizes the output of the reward function; an inference unit that inputs the parameters including the average luminance value of the region of interest and the target luminance value, which are the evaluation results by the evaluation unit, into the exposure control model, and outputs a plurality of multiplication values ​​of the exposure time and the gain and scores of the multiplication values ​​as inference values; a state change unit that selects, as an action, the exposure time and the gain that result in the maximum score from among the scores of the multiple multiplication values ​​output as the inference value, and inputs the parameters including the selected exposure time and the gain into the environment simulator; The exposure control system of claim 1 , comprising:

3. The parameters include absolute ambient brightness information indicating an absolute brightness scale in an actual environment.

3. The exposure control system according to claim 2.

4. the environment simulator acquires, from the image captured in advance by the imaging unit, the average luminance value of the region of interest that is associated with the exposure time and the gain among the input parameters in a basic database that associates the exposure time, the gain, and the average luminance value of the region of interest; 3. The exposure control system according to claim 2.

Images