Exposure control method and electronic device for vehicle-mounted camera

Abstract

The present invention relates to the field of camera technology, and provides an exposure control method and electronic device for a vehicle-mounted camera. The method comprises: when a light change scene is detected within a preset range in front of a vehicle, obtaining at least two frames of images captured by a camera on the vehicle; based on the brightness ratio of each frame of the at least two frames of images, predicting the target brightness ratio of the frame image corresponding to the light change scene, the brightness ratio being the brightness ratio between the light change area and the ground lane area in the image; based on the target brightness ratio, determining the target parameter value of the exposure control parameter of the camera within a preset time period, the preset time period being determined based on the time when the camera enters the light change scene; controlling the camera to perform image acquisition based on the target parameter value within the preset time period. The present invention avoids overexposure or underexposure of the image, ensures that a clear and stable image can be obtained under various environmental conditions, and improves the adaptability and accuracy of the camera in dynamic light changes.

Description

Technical Field

[0001] The present invention relates to the technical field of camera control, and in particular to an exposure control method and electronic equipment of a vehicle-mounted camera. Background Art

[0002] With the rapid development of smart cars, the number and frequency of vehicle-mounted cameras, as key visual perception tools, are increasing. The perception data captured by the camera is crucial to the realization of autonomous driving functions, especially in environmental perception and obstacle detection. However, when dealing with scenes with changing light, such as entering and exiting tunnels and passing by tree-lined scenes, the camera has difficulty in quickly adapting to changes in light, resulting in overexposure or underexposure of the images collected by the camera, thus posing a safety hazard. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides an exposure control method and electronic device for a vehicle-mounted camera to solve the problem in the prior art that images captured by a camera are prone to overexposure or underexposure in a light-changing scenario.

[0004] According to a first aspect of an embodiment of the present invention, there is provided an exposure control method for a vehicle-mounted camera, comprising:

[0005] When a scene of light changes is detected within a preset range in front of the vehicle, at least two frames of images captured by a camera on the vehicle are acquired;

[0006] Based on the brightness ratio of each frame image in at least two frames of images, a target brightness ratio of the frame image corresponding to the light change scene is predicted, where the brightness ratio is the brightness ratio between the light change area and the ground lane area in the image;

[0007] Based on the target brightness ratio, a target parameter value of an exposure control parameter of the camera within a preset period is determined, and the camera is controlled to acquire images based on the target parameter value within the preset period. The preset period is determined based on the time when the camera enters a scene with changing light.

[0008] A second aspect of an embodiment of the present invention provides an exposure control device for a vehicle-mounted camera, comprising:

[0009] An acquisition module is configured to acquire at least two frames of images captured by a camera on the vehicle when a light change scene is detected within a preset range in front of the vehicle;

[0010] A prediction module is configured to predict a target brightness ratio of a frame image corresponding to a light change scene based on a brightness ratio of each frame image in at least two frames of images, wherein the brightness ratio is a brightness ratio between a light change area and a ground lane area in the image;

[0011] A determination module configured to determine a target parameter value of an exposure control parameter of a camera within a preset period of time based on the target brightness ratio, wherein the preset period of time is determined based on a time when the camera enters a scene with light changes;

[0012] The control module is configured to control the camera to collect images based on the target parameter value within a preset time period, and the preset time period is determined based on the time when the camera enters a scene with changing light.

[0013] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0014] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0015] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: when a light change scene is detected within a preset range in front of a vehicle, at least two frames of images captured by a camera on the vehicle are acquired, and based on the brightness ratio of the at least two frames of images captured, a target brightness ratio of the frame image corresponding to the light change scene corresponding to the camera is predicted, and a target parameter value of the camera exposure control parameter within a preset time period is determined based on the target brightness ratio, the preset time period is determined based on the time when the camera enters the light change scene, and the camera is controlled to perform image capture based on the target parameter value within the preset time period; the exposure control parameter is predicted in advance before the camera reaches the light change scene, and the predicted target parameter value of the exposure control parameter is adapted to the light intensity of the light change scene, so that the on-board camera can adjust the exposure control parameter of the on-board camera in a timely and accurate manner, and the captured image can accurately reflect the scene information of the light change scene, avoid capturing overexposed and underexposed images, ensure that clear and stable images can be obtained under various environmental conditions, and significantly improve the adaptability and accuracy of the camera in dynamic light changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of an exposure control method for a vehicle-mounted camera provided by an embodiment of the present invention;

[0018] Figure 2 is a schematic diagram of a fitting curve provided by an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of trajectory planning information provided by an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of a light change scene provided by an embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of an exposure control system according to an embodiment of the present invention;

[0022] Figure 6 is a flow chart of another exposure control method for a vehicle-mounted camera provided by an embodiment of the present invention;

[0023] Figure 7 is a schematic diagram of an exposure control device for a vehicle-mounted camera provided by an embodiment of the present invention;

[0024] Figure 8 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0026] An exposure control method and device for a vehicle-mounted camera according to an embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] In related technologies, since the image sensor of the camera has a limited dynamic range, the performance of the camera under extreme lighting conditions is limited. When the vehicle encounters scenes with large changes in light brightness such as entering or exiting a tunnel, passing by tree-lined trees, or urban high-rise buildings, it is often difficult to quickly adapt to the lighting changes, resulting in overexposure or underexposure of the collected images. Overexposure will cause the loss of details in the bright areas of the image, while underexposure will cause the loss of details in the shadow areas of the image, which can easily cause serious safety hazards for intelligent driving systems that rely on visual information for decision-making.

[0028] There are also attempts in related technologies to use High Dynamic Range Imaging (HDR) technology to expand the dynamic range of vehicle-mounted cameras, but high dynamic range technology requires the synthesis of multiple photos with different exposures, which not only increases processing time, but also causes artifacts or other visual defects in the synthesized image, resulting in low image quality. In addition, high dynamic range technology has high requirements on hardware and computing resources, which increases costs.

[0029] In addition, the current automatic exposure technology is mainly based on a passive closed-loop control system, that is, it relies on real-time feedback on the brightness of the existing image to adjust the exposure parameters. The passive closed-loop control system can adapt to different lighting conditions, but usually does not have the ability to predict future lighting changes, resulting in limitations in the response speed when facing scenes such as vehicles entering and exiting tunnels, passing through tree-lined areas, and urban high-rise buildings. There is still overexposure or underexposure.

[0030] Based on the above content, the present invention provides an exposure control method for a vehicle-mounted camera, which can identify scenes with light changes based on visual perception and high-precision positioning, and predict light changes in combination with the vehicle's motion state, thereby realizing active predictive exposure control for scenes with complex light (i.e., determining the exposure control parameters corresponding to the light change scene before reaching the light change scene), effectively avoiding the problem of overexposure or underexposure, thereby providing clearer and more reliable image data, and increasing the performance and safety of the intelligent driving system in complex lighting environments.

[0031] Figure 1 FIG. 1 is a flow chart of an exposure control method for a vehicle-mounted camera provided by an embodiment of the present invention. The exposure control method for a vehicle-mounted camera can be executed by a controller on the vehicle side (eg, a domain controller of the vehicle) or by the cloud. Figure 1 As shown, the exposure control method of the vehicle-mounted camera includes the following steps:

[0032] S101, when a light change scene is detected within a preset range in front of a vehicle, at least two frames of images captured by a camera on the vehicle are acquired.

[0033] The light change scene refers to a scene in which the difference between the light brightness at a later moment and the previous moment is greater than a preset value, which can be set according to the performance of the camera and is not specifically limited here. The difference value can be the difference between the light brightness at a later moment and the previous moment, or the ratio of the light brightness at a later moment to the previous moment, and accordingly, the preset value corresponds to the difference value.

[0034] A forward-looking sensor network on the vehicle, such as a light sensor and multiple cameras, can be used to detect whether the difference between the light brightness at a later moment and the light brightness at a previous moment within a preset range in front is greater than a preset value. If it is greater than the preset value, it is determined that the scene corresponding to the later moment is a light change scene. The light change scenes may include tunnel entrances and exits, tree-lined areas, high-rise buildings and other scenes.

[0035] It should be noted that the preset range in front of the vehicle can be a fan-shaped range with the preset road length in front of the vehicle as the radius and the road on which the vehicle is traveling as the central axis, and the angle of the fan-shaped range is the preset angle; of course, the preset range in front of the vehicle can also be the area covered by the preset road length in front of the vehicle. For example, as an example, the preset range in front of the vehicle can be a fan-shaped area with a radius of the preset road length and 15 degrees to the left and right of the road on which the vehicle is traveling as the central axis; or, the preset range in front of the vehicle is the road range covered by the road length of 50 meters in front of the vehicle.

[0036] The vehicle-mounted camera includes a front-view camera, a surround-view camera, a rear-view camera, etc. The camera in this embodiment can be any camera on the vehicle, that is, each camera on the vehicle, or one or more cameras on the vehicle. That is to say, taking any camera as an example, at least two frames of images collected by the camera can be obtained, so as to determine the target parameter value of the exposure control parameter of the camera within a preset time period.

[0037] The at least two frames of images acquired by the camera may be at least two frames of images acquired at a preset interval frame number before reaching the scene where the light changes. For instance, as an example, one image may be acquired at an interval of 10 frames 2 seconds before reaching the scene where the light changes. When the camera has a frame rate of 30 per second, 6 images may be acquired.

[0038] S102, predicting a target brightness ratio of a frame image corresponding to a light change scene based on a brightness ratio of each frame image in at least two frames of images.

[0039] The brightness ratio is the brightness ratio between the light changing area and the ground lane area in the image. The brightness ratio of each frame image in at least two frames of images is the brightness ratio between the light changing area and the ground lane area in each frame image, and the target brightness ratio is the brightness ratio between the light changing area and the ground lane area in the frame image corresponding to the light changing scene.

[0040] The brightness ratio may be determined by edge detection, feature extraction, brightness analysis, etc. The target brightness ratio of the frame image corresponding to the light changing scene may be used to reflect the illumination intensity of the light changing scene.

[0041] The target brightness ratio of the frame image corresponding to the light change scene is predicted by the brightness ratio of at least two frames of images, so that the light intensity change trend of the scene that the vehicle will pass through in the future can be determined by the target brightness ratio, thereby realizing the early identification of the lighting changes of the road to be traveled.

[0042] S103, determining a target parameter value of an exposure control parameter of the camera within a preset period of time based on the target brightness ratio, and controlling the camera to perform image acquisition based on the target parameter value within the preset period of time.

[0043] The time when the camera enters the scene with light changes during the preset period is determined; the exposure control parameters may include exposure time, analog gain and digital gain, etc.

[0044] It should be noted that the preset time period may include not only the time when the camera enters the light change scene, but also the first preset time length before the camera enters the light change scene, and / or the second preset time length after the camera enters the light change scene. For example, in one example, the preset time period includes 1 second before the camera enters the light change scene until 3 seconds after the camera enters the light change scene; in another example, the preset time period includes the time when the camera enters the light scene until 5 seconds after the camera enters the light change scene.

[0045] Since the target brightness ratio can reflect the change in light intensity of a scene with light changes, the target parameter value of the exposure control parameter that is adapted to the light intensity of the scene with light changes can be determined through the target brightness ratio. In this process, for example, based on the preset correspondence between the preset brightness ratio interval and the exposure control parameter adjustment strategy, the target adjustment strategy corresponding to the target brightness ratio can be determined, and the exposure control parameter of the camera can be adjusted based on the target adjustment strategy to obtain the target parameter value; for another example, based on the preset correspondence between the preset brightness ratio interval and the parameter values ​​of different exposure control parameters, the target parameter value of the exposure control parameter corresponding to the target brightness ratio is determined, wherein the preset correspondence between the preset brightness ratio interval and the parameter values ​​of different exposure control parameters can be obtained through experiments or historical data analysis.

[0046] In addition, when controlling the camera to acquire images based on the target parameter value within a preset time period, the target parameter value corresponding to the camera can be sent to the camera through the control network inside the vehicle when the current time reaches the start time of the preset time period, and the camera acquires images based on the target parameter value within the preset time period.

[0047] In this way, the target parameter value of the camera's exposure control parameter within a preset time period is determined through the target brightness ratio, so that the exposure control parameter of the camera when reaching a scene with changing light can adapt to the lighting conditions of the scene with changing light, thereby being able to capture clear, detail-rich images and avoiding the problem of image quality degradation due to overexposure or underexposure.

[0048] According to the technical solution provided in the embodiment of the present invention, it is possible to predict the exposure control parameters in advance before reaching the scene where the light changes, and the target parameter values ​​of the predicted exposure control parameters are adapted to the light intensity of the scene where the light changes, thereby avoiding the collection of overexposed and underexposed images, ensuring that clear and stable images can be obtained under various environmental conditions, and significantly improving the adaptability and accuracy of the camera in dynamic light changes.

[0049] In some embodiments, detecting a light change scene within a preset range in front of the vehicle includes:

[0050] Acquire a forward-looking image captured by a forward-looking camera, and based on the forward-looking image, detect whether there is a light change scene within a preset range ahead through a preset scene recognition model; and / or,

[0051] Based on the vehicle's current location information, navigation information and map information, detect whether there is a light change scene within a preset range ahead;

[0052] The preset front range includes: the road area range along the vehicle's driving direction within a preset time period in the future; or, the road area range corresponding to a preset road length in front of the vehicle along the vehicle's driving direction.

[0053] Before detecting whether there is a light change scene within a preset range in front of the vehicle, the vehicle's driving speed and gear position can be monitored in real time. When the vehicle's driving speed is greater than the preset speed and the current gear is the forward gear, it is considered that the light change will affect the image collected by the camera. At this time, it can be detected whether there is a light change scene within the preset range in front of the vehicle, avoiding the need to detect whether there is a light change scene within the preset range in front of the vehicle when the above conditions are not met, thereby reducing invalid computing power and power consumption and saving vehicle resources.

[0054] In addition, when detecting whether there is a scene of light change within a preset range ahead, one method is to obtain the front view image captured by the front view camera and input the front view image into a preset scene recognition model to detect whether there is a scene of light change within the preset range ahead. The scene recognition model can identify scenes of light change such as entering and exiting tunnels, passing by tree-lined areas, and urban high-rise buildings.

[0055] Among them, the forward-looking image collected by the forward-looking camera is obtained because the vehicle is moving forward most of the time, so only the scene recognition model based on forward-looking perception needs to be trained, which saves computing power and the amount of training calculations for the scene recognition model. In addition, it should be noted that the surround-view images collected by the surround-view camera can also be obtained, and the scene recognition model based on surround-view perception can be trained based on the surround-view images. It is also possible to select training sets corresponding to at least two types of perception data from forward-looking perception, surround-looking perception, and rear-view perception to train the scene recognition model. During the training process, scenes with changing light, such as entering and exiting tunnels, passing by tree-lined areas, and urban high-rise buildings, as well as other non-changing scenes, can be used as training sets to train the scene recognition model.

[0056] The scene recognition model may include a convolutional neural network model, a target detection model, or other models that can recognize image information, and is not specifically limited here.

[0057] In addition, when detecting whether there is a scene of light change within the preset range ahead, the second method, in addition to relying on the front-view camera, can also be detected based on the vehicle's current location information, navigation information and map information. In this process, the positioning module can determine the more accurate current location information of the vehicle, and match the map elements with the high-precision map, and then identify whether there is a scene of light change within the preset range ahead based on the data stored on the high-precision map and the navigation information.

[0058] The future preset time length may be 3s, 5s, 10s, etc., which is not specifically limited here. The preset road length may be 50 meters, 60 meters, etc., which is also not specifically limited. In addition, the area range may be determined in combination with factors such as the current speed and acceleration of the vehicle. Further, after the front preset range has been determined, if the vehicle travels out of the determined preset range, it is necessary to continue to determine the front preset range.

[0059] Furthermore, if any of the above two detection methods detects the existence of a light change scene, it can be determined that there is a light change scene within the preset range ahead; similarly, if both of the above two detection methods detect that there is no light change scene, it can be determined that there is no light change scene within the preset range ahead.

[0060] According to the technical solution provided in the embodiment of the present invention, it is possible to comprehensively and accurately identify light change scenes within a preset range ahead based on at least one of the above-mentioned detection methods, thereby saving computing power and energy consumption while avoiding omission of light change scenes, thereby enabling exposure control of the camera to improve driving safety.

[0061] In some embodiments, after detecting whether there is a light change scene within a preset range ahead based on the forward-view image by using a preset scene recognition model, the method further includes:

[0062] When the front view image is overexposed or underexposed, image data of a preset number of frames adjacent to the front view image and exposure parameters corresponding to each frame are obtained;

[0063] The image data of the front-view image, the image data of the preset number of frames and the exposure parameters are updated to the training set of the scene recognition model to obtain an updated training set, and the updated training set is used to optimize the scene recognition model.

[0064] The adjacent preset number of frames may be image data of preset frames before and after the current forward-looking image, with the current forward-looking image as the center.

[0065] In addition, overexposure means that the brightness of part or all areas of the image is too high due to excessive light, resulting in loss of details. Underexposure means that the image is dark overall and the details are blurred due to insufficient light.

[0066] In some examples, image data of the forward-view image, image data of a preset number of frames, and exposure parameters are integrated and updated into the training set of the scene recognition model to obtain an updated training set. The scene recognition model is retrained or fine-tuned based on the updated training set, so that the scene recognition model can learn and determine relevant data of light changes from the image data, exposure parameters and their associations, thereby achieving further training and optimization of the scene recognition model, and enabling the system to have the ability to evolve automatically, thereby enabling the scene recognition model to more accurately predict and respond to possible light change scenes.

[0067] Furthermore, the scene recognition model trained according to the updated training set can be applied to subsequent scene recognition with light changes. The performance of the scene recognition model can also be continuously monitored, new feedback data can be collected, and the scene recognition model can be iterated again or the performance of the scene recognition model can be improved.

[0068] In addition, it should be noted that after the power-off request is initiated, it is possible to check whether the training set is still being updated. If so, the untransmitted scene data (i.e., training data) is updated to the training set, and feedback is given to delay the power-off, and the power is turned off after the training set is updated.

[0069] According to the technical solution provided by the embodiments of the present invention, when the front view image is overexposed or underexposed, the scene recognition model is optimized based on the overexposed or underexposed front view image and its related data, so as to improve its adaptability and robustness under complex lighting conditions, thereby continuously improving the recognition efficiency of scenes with changing lighting and improving the long-term stability and reliability of the recognition of scenes with changing lighting.

[0070] In some embodiments, based on the brightness ratio of each frame image in at least two frames of images, predicting a target brightness ratio of a frame image corresponding to a light change scene includes:

[0071] Fitting the brightness ratio of each frame image to obtain a fitting curve;

[0072] Based on the fitting curve, the target brightness ratio of the frame image corresponding to the predicted arrival time is predicted, and the brightness ratio of the frame image corresponding to the predicted arrival time is determined as the target brightness ratio of the frame image corresponding to the light change scene corresponding to the camera; wherein the predicted arrival time is the time when the camera arrives at the light change scene.

[0073] When all conditions except light intensity remain unchanged, the higher the light intensity, the brighter the image captured by the camera. This is because when the light intensity is higher, more photons hit the image sensor, resulting in more light signals being converted into electrical signals. In the prior art, the control method of the camera exposure control parameters is closed-loop feedback control, that is, when the camera reaches a scene where light changes, it is necessary to perform post-processing analysis based on the actual image, and then adjust and correct the exposure control parameters; however, the closed-loop feedback control makes it impossible to determine the appropriate exposure control parameters in advance, and even if the speed of image data processing is enhanced and the response time is accelerated, it is still unable to handle light change scenes well. The technical solution provided by the embodiment of the present invention can determine the appropriate exposure control parameters before the camera reaches a scene where light changes, so it is necessary to predict future light intensity changes.

[0074] The brightness ratio between the light changing area and the ground lane area in each frame image may be determined according to the brightness of the light changing area and the ground lane area in each frame image.

[0075] In addition, when fitting the brightness ratio of each frame image, the least squares method can be used to fit the brightness ratio of each frame image to obtain a fitting curve, or the brightness ratio of each frame image can be fitted by linear regression, polynomial regression, exponential regression, etc. to obtain a fitting curve.

[0076] Then, when determining the target brightness ratio, interpolation prediction can be performed based on the fitting curve to obtain the target brightness ratio of the frame image corresponding to the predicted arrival time.

[0077] The predicted arrival time is the time it takes for the camera to arrive at the scene of light changes. For different cameras on the vehicle, cameras with different mapping positions in the length direction of the vehicle arrive at the scene of light changes at different times. The camera close to the front of the vehicle arrives at the scene of light changes first, that is, the predicted arrival time corresponding to the camera close to the front of the vehicle is less than the predicted arrival time corresponding to the camera close to the rear of the vehicle.

[0078] The time when any camera arrives at the scene with light changes can be based on any of the front-view cameras to determine the predicted arrival time of the front-view camera at the scene with light changes, and then the predicted arrival time corresponding to other cameras is determined based on the relative positional relationship between other cameras and the front-view camera, such as left-right relationship, front-back relationship, etc. Of course, in other examples, other cameras can also be selected as the benchmark.

[0079] In addition, when determining the predicted arrival time corresponding to any camera, it can be determined based on the vehicle's driving speed, acceleration, the distance between the vehicle's front and the light changing scene, and the distance between the camera and the vehicle's front; it can also be obtained by analyzing the images taken by the camera.

[0080] For example, Figure 2 This is a schematic diagram of a fitting curve provided by an embodiment of the present invention. Taking the number of frames of at least two frames of images as 6 frames as an example, the vertical axis in the figure is the brightness ratio of each frame of the image, the horizontal axis is the frame number i, and assuming that the frame number of the first frame collected is 1, the frame number of each frame of the image is obtained in sequence. The brightness ratio of the 6 frames is fitted by the least squares method, and the fitting curve can be obtained by the following formula: Ri=a*i 3 +b*i 2 +c*i+d, where i represents the frame number, a, b, c, and d are pre-set numerical parameters, and Ri represents the fitting point corresponding to the i-th frame image. The obtained fitting curve is as follows Figure 2 As shown in the middle curve, interpolation prediction can then be performed based on the fitting curve Ri to obtain the brightness ratio of frame number 7, that is, the target brightness ratio (the intersection value of the dotted line corresponding to the 7th frame in the figure and the fitting curve).

[0081] Furthermore, the predicted target brightness ratio can be verified and evaluated. After the actual target brightness ratio of the frame image corresponding to the predicted arrival time is collected, it is compared with the predicted target brightness ratio to determine the generalization ability.

[0082] According to the technical solution provided by the embodiment of the present invention, the target brightness ratio of the frame image corresponding to the time when the camera arrives at the light change scene can be predicted based on the fitting curve corresponding to the brightness ratio of at least two frames of images, thereby ensuring the accuracy of the predicted target brightness ratio.

[0083] In some embodiments, before predicting the brightness ratio of the frame image corresponding to the predicted arrival time based on the fitting curve, the method further includes:

[0084] Determine the distance of the target between the forward-looking camera and the scene with changing light;

[0085] Determine the target distance between each other camera and the light changing scene based on the spacing distance between each other camera and the front view camera in the vehicle body length direction and the target distance between the front view camera and the light changing scene, each other camera is a camera on the vehicle except the front view camera;

[0086] Obtain the trajectory planning information of the vehicle, and determine the predicted arrival time of any camera to the light change scene based on the trajectory planning information and the target distance corresponding to any camera.

[0087] The target distance between the front-view camera and the scene with changing light can be calculated based on the image captured by the camera, map information, and the real-time position information of the camera.

[0088] In addition, the spacing distance between each other camera and the front camera in the length direction of the vehicle body can be obtained through vehicle design drawing data or actual measurement data. When determining the spacing distance in actual measurement, for example, the front camera can be used as the coordinate origin, the vehicle driving direction can be used as the ordinate, and a rectangular coordinate system can be established, and the ordinate of each other camera can be determined as the spacing distance. After determining the spacing distance, the sum of the target distance corresponding to the front camera and the spacing distance can be determined as the target distance corresponding to each other camera. For example, assuming that the spacing distance between the rear view camera and the front view camera is 4.8m, and the distance between the front view camera and the light change scene is 50m, the distance between the rear view camera and the light change scene is 54.8m.

[0089] In addition, the vehicle's trajectory planning information includes the vehicle's expected driving path, speed curve, etc. Based on the trajectory planning information and the target distance corresponding to each camera, the predicted arrival time of each camera at the light change scene can be determined. In this process, the trajectory planning information can be converted into a two-dimensional correspondence between path and time. Since the path in the trajectory planning information is known, the corresponding predicted arrival time can be obtained.

[0090] For example, Figure 3 is a schematic diagram of trajectory planning information provided by an embodiment of the present invention, such as Figure 3 As shown in the figure, the trajectory planning information is represented as the correspondence between the path s and the time t. Based on the relationship between the position and time of the vehicle on the planned route, the target distance DIS can be calculated based on the Figure 3 Determine the predicted arrival time t1.

[0091] According to the technical solution provided by the embodiment of the present invention, it is possible to first determine the target distance between the forward-looking camera and the light-changing scene, and then determine the target distances between other cameras and the light-changing scene based on this, thereby avoiding the problem of high computing power and low efficiency caused by repeatedly calculating the distance using the same algorithm. While ensuring the accuracy of each target distance, the efficiency of determining the target distance is improved, thereby improving the efficiency of determining each predicted arrival time and saving computing power.

[0092] In some embodiments, the method of determining the target distance between the front-view camera and the scene with changing light includes any of the following:

[0093] Acquire real-time position information of any forward-looking camera, and determine a first distance between the forward-looking camera and the light-changing scene based on the map information and the real-time position information of the forward-looking camera, and determine the first distance as a target distance between the forward-looking camera and the light-changing scene;

[0094] Acquire image information of an image captured by the front-view camera, and determine a second distance between the front-view camera and the light-changing scene based on the image information, and determine the second distance as a target distance between the front-view camera and the light-changing scene;

[0095] Acquire the real-time position information of any forward-looking camera, determine a first distance between the forward-looking camera and the light-changing scene based on the map information and the real-time position information of the forward-looking camera, acquire image information of the image captured by the forward-looking camera, determine a second distance between the forward-looking camera and the light-changing scene based on the image information, and determine a target distance between the forward-looking camera and the light-changing scene based on the first distance, the second distance, the positioning confidence corresponding to the first distance, and the image perception confidence corresponding to the second distance.

[0096] Among them, this embodiment can obtain the real-time position information of any forward-looking camera, and the real-time position information includes longitude, latitude, altitude and the current direction of the vehicle.

[0097] The map information may be high-precision map data, which includes location information of scenes with changing light, such as road geometry, tunnel entrances and exits, shadows under viaducts, tree-lined roads, and densely packed high-rise buildings. When determining the first distance, the real-time location information of the front-view camera may be mapped to the map information, thereby determining the first distance between the front-view camera and the scene with changing light.

[0098] When determining the second distance between the front-view camera and the scene with changing light, the image data can be analyzed through computer vision techniques, such as edge detection, light intensity change analysis, image segmentation, etc., to identify areas in the image data where the light begins to change, such as areas where the brightness suddenly decreases or increases. Based on parameters such as the focal length and viewing angle of the front-view camera, the second distance between the front-view camera and the scene with changing light can be determined through algorithms such as perspective transformation or inverse perspective mapping.

[0099] The first distance or the second distance can be directly determined as the target distance between the front-view camera and the light-changing scene, avoiding the problem of calculation errors caused by determining the target distance by using both the first distance and the second distance when the first distance or the second distance is invalid, thereby achieving the reliability of the target distance calculation.

[0100] In addition, confidence evaluation is performed on the first distance and the second distance respectively. The confidence of the first distance may be affected by the map accuracy and positioning accuracy, and the confidence of the second distance may be affected by the accuracy of the image analysis algorithm and the calibration status of the camera.

[0101] When determining the target distance between the front-view camera and the light-changing scene according to the first distance, the positioning confidence corresponding to the first distance, the second distance, and the image perception confidence of the front-view camera, the following formula can be used for calculation:

[0102] DIS = (DIS1*C1+DIS2*C2) / (C1+C2); or,

[0103] DIS=(DIS1*C1 +DIS2*C2) / 2;

[0104] Among them, DIS represents the target distance, DIS1 represents the first distance, C1 represents the positioning confidence corresponding to the first distance, DIS2 represents the second distance, and C2 represents the image perception confidence.

[0105] The target distance is determined by jointly using the first distance and the second distance, thereby ensuring the accuracy of the target distance, and further ensuring the accuracy of the predicted arrival time of the camera arriving at the light change scene.

[0106] This embodiment ensures the diversity and accuracy of the target distance determination method by determining the first distance as the target distance between the front-view camera and the light change scene, or determining the second distance as the target distance between the front-view camera and the light change scene, or determining the target distance between the front-view camera and the light change scene based on the first distance and the second distance at the same time.

[0107] In some embodiments, it also includes:

[0108] Performing semantic segmentation on each of at least two frames of images to obtain a light change area and a ground lane area in the image;

[0109] The average brightness of the light change area and the average brightness of the ground lane area in each frame image are determined, and the ratio of the average brightness of the light change area and the average brightness of the ground lane area in each frame image is determined as the brightness ratio of each frame image.

[0110] Semantic segmentation is performed on each frame of image to identify the light change area and the ground lane area in each frame of image. Semantic segmentation can be performed on each frame of image through a deep learning model. The deep learning model can be a Mask Region-based Convolutional Neural Network (Mask R-CNN), Deep Learning for Scene Labeling (DeepLab), etc., which are used to identify different objects or areas in the image.

[0111] When determining the average brightness value of the light change area and the average brightness value of the ground lane area in each frame image, all pixels in each area can be traversed, their brightness values ​​are accumulated and divided by the total number of pixels in the area to obtain the average brightness value of the area.

[0112] It should be noted that the light change area is generally located in the middle of the image and changes with the real-time position information of the camera.

[0113] For example, Figure 4 is a schematic diagram of a tunnel scene provided by an embodiment of the present invention, such as Figure 4 As shown, area A represents the light change area, area B represents the ground lane area, the average brightness of area A in each frame image is recorded as LAi, and the average brightness of area B in each frame image is recorded as LBi. The brightness ratio corresponding to each frame image can be determined by the following formula: Ri=LAi / LBi, wherein Ri represents the brightness ratio of the i-th frame image, i represents the frame number, LAi represents the average brightness of the light change area, and LBi represents the average brightness of the ground lane area.

[0114] According to the technical solution provided by the embodiment of the present invention, the brightness ratio of each frame image is determined based on the average brightness of the light change area and the average brightness of the ground lane area in each frame image, thereby ensuring the accuracy of the brightness ratio.

[0115] In some embodiments, determining a target parameter value of an exposure control parameter of a camera within a preset period of time based on the target brightness ratio includes:

[0116] Determining a target adjustment strategy corresponding to the target brightness ratio based on a preset corresponding relationship between a preset brightness ratio interval and an exposure control parameter adjustment strategy;

[0117] The exposure control parameters of the camera are adjusted based on the target adjustment strategy to obtain a target parameter value; wherein the exposure control parameters include at least one of exposure duration, analog gain and digital gain.

[0118] Exposure time can be used to control how long the camera shutter is open, which directly affects the overall brightness of the image; analog gain can adjust the analog signal before it is converted to a digital signal to increase the signal strength and increase the brightness of the image; digital gain can adjust the image data after it is converted to a digital signal to increase the brightness of the image.

[0119] This embodiment can obtain the correspondence between the preset brightness ratio range and the exposure control parameter adjustment strategy based on a large amount of experimental data analysis, so as to cover various possible lighting conditions and shooting requirements. For example, when the brightness is relatively low, at least one of the exposure time, analog gain or digital gain is increased to enhance the image brightness. When the brightness ratio is too high, the opposite measures need to be taken to avoid overexposure.

[0120] In the preset corresponding relationship, when the preset brightness ratio interval is greater than 1, the exposure control parameter adjustment strategy is used to reduce at least one of the exposure time, analog gain and digital gain; when the preset brightness ratio interval is less than 1, the exposure control parameter adjustment strategy is used to increase at least one of the exposure time, analog gain and digital gain.

[0121] After predicting the target brightness ratio, a target adjustment strategy matching the target brightness ratio is determined according to the above preset corresponding relationship to ensure the accuracy and efficiency of the adjustment strategy; then the parameter value adjusted according to the target adjustment strategy is determined as the target parameter value, ensuring that the adjusted target parameter value is adapted to the target brightness ratio, that is, adapted to the change in light intensity of the light changing scene.

[0122] It should be noted that after completing the image acquisition for the preset period of time, the exposure control parameters are no longer actively adjusted until the next light change scene is identified.

[0123] According to the technical solution provided in the embodiment of the present invention, the exposure control parameters of the camera are determined according to the target brightness ratio, which can identify and adapt to light changes in advance, ensure that clear and stable images can be obtained under various environmental conditions, significantly improve the adaptability and accuracy of the camera in dynamic light changes, avoid overexposure or underexposure of the image, and improve driving safety.

[0124] In some embodiments, the preset correspondence between the preset brightness ratio interval and the exposure control parameter adjustment strategy includes:

[0125] In the case where the range of the preset brightness ratio interval is greater than or equal to 1, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is: the exposure duration of the current frame is the product of the exposure duration of the previous frame and the first preset multiple of the current frame image target brightness ratio, the analog gain of the current frame is the product of the analog gain of the previous frame and the first preset multiple of the current frame image target brightness ratio, the digital gain of the current frame is the product of the digital gain of the previous frame and the first preset multiple of the current frame image target brightness ratio, wherein the first preset multiple is less than 1, and the higher the value of the preset brightness ratio interval in different preset brightness ratio intervals, the smaller the first preset multiple;

[0126] When the range of the preset brightness ratio interval is less than 1, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is: the exposure duration of the current frame is the product of the exposure duration of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, the analog gain of the current frame is the product of the analog gain of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, and the digital gain of the current frame is the product of the digital gain of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, wherein the second preset multiple is greater than 1, and the higher the value of the preset brightness ratio interval in different preset brightness ratio intervals, the larger the second preset multiple.

[0127] The imaging of the camera is related to parameters such as exposure time, aperture, sensitivity and gain. Among them, the exposure time controls the light receiving time and can affect the image brightness. The image is dark when the exposure time is short, and the image is bright when the exposure time is long. It can also affect the capture ability and motion blur. The aperture can adjust the amount of light entering. The smaller the aperture value, the more light enters, the brighter the image and the blurred background. The larger the aperture value, the darker the image. The sensitivity (ISO) adjusts the sensitivity of the photosensitive element. High ISO is bright but has more noise, and low ISO is dark but has less noise. Gain includes analog gain, digital gain and image signal processor (ISP) gain, which amplify the signal at different image processing stages, affecting brightness and brightness. For vehicle-mounted cameras, the aperture and focal length are fixed, and the image effect can be adjusted by adjusting the exposure time (Shutter Time) and gain (Gain).

[0128] The brightness ratio in the preset brightness ratio interval refers to the ratio between the brightness value of the light change area in the image and the brightness value of the ground lane area.

[0129] In order to maintain the stability and rich details of the image captured by the camera in scenes with changing light, at least one of the exposure duration, analog gain and digital gain can be adjusted according to the preset correspondence between the preset brightness ratio range and the exposure control parameter adjustment strategy to adapt to the brightness changes in scenes with changing light and optimize the imaging effect.

[0130] In addition, when the system detects that the preset brightness ratio interval of the current scene is greater than or equal to 1, that is, the brightness value of the light change area is greater than the brightness value of the ground lane area, it indicates that the brightness of the current scene is dark, and the scene brightness change type is from a dark scene to a bright scene. In order to increase the image brightness and improve the dark details, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is:

[0131] Exposure duration adjustment: The exposure duration of the current frame is adjusted according to the exposure duration of the previous frame. The specific adjustment amount is the product of the target brightness ratio of the current frame image and a first preset multiple less than 1, which is used to prevent overexposure by reducing the exposure time and retain more details in the highlight area. As the value of the preset brightness ratio range increases, the scene becomes brighter (that is, as the brightness ratio increases, the amount of brightening increases), and the first preset multiple is further reduced to achieve more detailed exposure control;

[0132] Analog gain adjustment: Similar to exposure time, analog gain is also adjusted based on the analog gain of the previous frame and the first preset multiple of the target brightness ratio of the current frame image. The direction of analog gain adjustment is the same as that of exposure time adjustment (i.e., decrease). The adjustment of analog gain focuses more on the balance of signal amplification to compensate for the signal-to-noise ratio problem that may be caused by the reduction of exposure time, and maintain the overall brightness and contrast of the image within an acceptable range;

[0133] Digital gain adjustment: The adjustment of digital gain also follows the above logic, that is, it is adjusted based on the digital gain of the previous frame and the first preset multiple of the target brightness ratio of the current frame image. The digital gain operates in the digital domain to further fine-tune the image brightness to avoid introducing additional noise. As the preset brightness ratio increases, the adjustment of digital gain becomes more cautious to maintain the naturalness and clarity of the image.

[0134] In addition, when the preset brightness ratio interval is less than 1, that is, the brightness value of the light change area is less than the brightness value of the ground lane area, that is, the overall brightness of the current scene is brighter or the brightness is moderate to high, and the scene brightness change type is from a bright scene to a dark scene. For the same preset brightness ratio interval, the exposure control strategy takes the opposite operation to the preset brightness ratio interval greater than 1, specifically:

[0135] Exposure duration adjustment: increase the exposure duration of the current frame by the product of the exposure duration of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, and the second preset multiple is greater than 1, which is used to capture more light by extending the exposure time, thereby brightening the image, wherein as the value of the preset brightness ratio interval further decreases, the scene becomes darker (that is, as the brightness ratio decreases, the amount of darkening increases), and the second preset multiple decreases accordingly, so as to more significantly increase the exposure;

[0136] Analog gain adjustment: Similar to exposure time adjustment, analog gain is positively adjusted (increased) based on the second preset multiple of the target brightness ratio of the previous frame value and the current frame image. The increase in analog gain helps to amplify the signal to improve the overall brightness and detail performance of the image.

[0137] Digital gain adjustment: The digital gain is positively adjusted (i.e., increased) according to the second preset multiple of the target brightness ratio of the previous frame value and the current frame image, wherein the digital gain is used to further adjust the brightness in the digital domain to enhance the overall brightness and detail performance of the image.

[0138] For example, taking the current light change scene as a tunnel as an example, the average light intensity of the tunnel should be between 100-200 lux, and the light intensity of a long tunnel should not be less than 50 lux, the light intensity on a cloudy or overcast day outdoors is about 1000 lux, the light intensity on a sunny day outdoors is 10000 lux, and the light intensity under direct sunlight is 100000 lux, so the preset brightness ratio can be divided into 6 levels, namely 0.02, 0.2, 1, 5, and 50. Table 1 is a preset correspondence table between the preset brightness ratio intervals and the exposure control parameter adjustment strategy:

[0139] Table 1 Preset correspondence between preset brightness ratio intervals and exposure control parameter adjustment strategies

[0140]

[0141] As shown in Table 1, the change types include dark to bright and bright to dark. Each change type includes three brightness ratio intervals. R represents the target brightness ratio of the current frame image, t represents the exposure time of the current frame, t1 represents the exposure time of the previous frame, a represents the analog gain of the current frame, a1 represents the analog gain of the previous frame, d represents the digital gain of the current frame, and d1 represents the digital gain of the previous frame. In addition, the first preset multiple is 1-0.1*R, and the second preset multiple is 1+0.1*R.

[0142] When the light change scene is identified and the target brightness ratio is calculated to be between 1≤ R<5, that is, the camera is about to leave the tunnel and the external environment is cloudy or dusk, the imaging can be optimized by reducing the exposure time. The specific exposure time is calculated by the formula t=(1-0.1*R)*t1;

[0143] When the scene of light change is identified and the target brightness ratio is calculated to be between 5≤ R<50, that is, the camera is about to leave the tunnel and the external environment is sunny, the imaging can be optimized by reducing the exposure time and reducing the analog gain. The specific exposure time is calculated by the formula t=(1-0.1*R)*t1. In addition, the analog gain needs to be adjusted. The analog gain is calculated by the formula a=(1-0.1*R)*a1.

[0144] When the light change scene is identified and the target brightness ratio is calculated to be 50≤R, that is, the camera is about to leave the tunnel, and the tunnel is dim while the external environment is sunny, etc., the imaging can be optimized by reducing the exposure time, reducing the analog gain and the digital gain. The specific exposure time is calculated by the formula t=(1-0.1*R)*t1. In addition, the analog gain needs to be adjusted. The analog gain is calculated by the formula a=(1-0.1*R)*a1. Finally, the digital gain needs to be adjusted. The digital gain is calculated by the formula d=(1-0.1*R)*d1.

[0145] When the light change scene is identified and the target brightness ratio is calculated to be between 0.2≤R<1, that is, the camera is about to enter a tunnel, and the external environment is cloudy or dusk, etc., the imaging can be optimized by increasing the exposure time. The specific exposure time is calculated by the formula t=(1+0.1*R)*t1;

[0146] When the scene of light change is identified and the target brightness ratio is calculated to be between 0.02≤R<0.2, that is, the camera is about to enter a tunnel and the external environment is sunny, etc., the imaging can be optimized by increasing the exposure time and analog gain. The specific exposure time is calculated by the formula t=(1+0.1*R)*t1. In addition, the analog gain needs to be adjusted. The analog gain is calculated by the formula a=(1+0.1*R)*a1.

[0147] When identifying a scene with changing light and calculating the target brightness ratio R≤0.02, that is, the camera is about to enter a tunnel, and the tunnel is dim while the external environment is sunny, etc., the imaging is optimized by increasing the exposure time, analog gain, and digital gain. The specific exposure time is calculated by the formula t=(1+0.1*R)*t1. In addition, the analog gain needs to be adjusted. The analog gain is calculated by the formula a=(1+0.1*R)*a1. Finally, the digital gain needs to be adjusted. The digital gain is calculated by the formula d=(1+0.1*R)*d1.

[0148] According to the technical solution provided in the embodiment of the present invention, at least one of the exposure time, analog gain and digital gain of the camera can be adjusted in advance according to the preset correspondence between the preset brightness ratio interval and the exposure control parameter adjustment strategy, thereby ensuring the best performance of the image under different lighting conditions, avoiding overexposure and overcoming underexposure, and achieving high-quality image capture.

[0149] Figure 5 FIG. 1 is a schematic diagram of an exposure control system according to an embodiment of the present invention. Figure 5 As shown, the exposure control system involves an ADAS Domain Controller (ADC), a cockpit Domain Controller (CDC), a vehicle Domain Controller (VDC), sensors, and multiple cameras, wherein the multiple cameras include a front-view camera, a surround-view camera, and a rear-view camera, etc. The ADC includes a system on chip (SOC), a microcontroller unit (MCU), a positioning module, and a sensor, etc. The control flow in the exposure control system includes:

[0150] After the camera acquires an image through an optical lens and a sensor, it is serialized based on the serializer inside the camera. The serialized image is transmitted to the ADC based on a coaxial cable, and then deserialized by the deserializer and given to the perception module in the SOC. Serialization is the process of converting the analog signal acquired by the camera into a digital signal.

[0151] After receiving the positioning data sent by SOC, CDC sends the vehicle-related map data to each algorithm module of SOC;

[0152] The VDC transmits the vehicle drive system status information to the MCU through the controller area network, and the MCU forwards it to the SOC decision module and planning control module;

[0153] The MCU receives data collected by radar and other sensors, and after preliminary processing, transmits it to the SOC perception module through the controller area network;

[0154] The positioning module calculates the positioning information and transmits it to the MCU through the controller area network. The MCU forwards it to the fusion module in the SOC, which fuses the positioning information with the map information.

[0155] The perception module in the SOC receives data from each sensor to perform perception recognition and obtains a perception recognition result;

[0156] The fusion module in the SOC receives the perception recognition results, map information, and positioning information to perform lane line and target fusion;

[0157] The prediction module in the SOC receives data from various sensors to perform target data fusion and trajectory tracking prediction;

[0158] The decision module in the SOC receives the results of the fusion module and the prediction module, and makes behavioral decisions based on the above data;

[0159] The planning module in the SOC receives the results of the fusion module, the prediction module, and the decision module, and performs trajectory planning based on the above data;

[0160] The control module in the MCU receives the results of the planning module to perform vehicle control and outputs the results to the VDC to perform vehicle control;

[0161] The active predictive exposure control module in the SOC identifies light change scenarios based on the results of the perception module and the planning module to trigger active predictive exposure control, and solves the exposure control parameters of each camera based on the light change prediction, and transmits the exposure control parameters of each camera to the corresponding camera via the coaxial cable.

[0162] Figure 6 FIG. 1 is a flow chart of another exposure control method for a vehicle-mounted camera provided by an embodiment of the present invention. Figure 6 As shown, the following steps may be included in this process:

[0163] Initialize the sensor data in the vehicle. The initialization process includes reading the field of view (FOV), installation position, angle and other parameter information of each visual sensor in the vehicle, and self-checking whether the camera data link, radar data link, map data link and vehicle status information link are normal; if normal, and when monitoring that the current vehicle gear is in the forward gear, perform scene recognition detection.

[0164] Among them, scene recognition detection can be detected through at least one of the vehicle's motion information, the positioning information of the high-precision map, the high-precision map and the scene recognition model.

[0165] When a scene with light changes is detected within a preset range in front of the camera, it is determined whether to start active predictive exposure control.

[0166] If active predictive exposure control is started, the light change time is predicted and the light change intensity is predicted through the camera's perception results, the vehicle's planning results and motion information. According to the results of the light change time prediction and the light change intensity prediction, the target parameter value of the exposure control parameter and the time of issuing the target parameter value are determined, so that the exposure control parameter is issued to the corresponding camera, and the light change scene is recorded and stored in the light change scene library.

[0167] If active predictive exposure control is not started, the camera is controlled according to the passive automatic exposure control method, and the light change scene is recorded and stored in the light change scene library. The light change scene library is used to train the scene recognition model.

[0168] According to the technical solution provided by the embodiment of the present invention, it is possible to identify light changes in advance, ensure that clear and stable images can be obtained under all environmental conditions, and improve the adaptability and accuracy of the camera in dynamic light changes; because active predictive exposure control is turned on when the vehicle is in forward gear and there is a light change scene within a preset range ahead, invalid computing power and power consumption are reduced, and optimal utilization of energy is achieved, while the service life of the vehicle is extended and costs are reduced; the adjusted exposure control parameters can respond quickly to light changes, provide reliable visual perception, and improve driving safety. At the same time, the scene recognition model can be optimized and iterated according to the light change scenes that have been passed, thereby improving long-term stability and reliability.

[0169] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0170] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present invention, which will not be described in detail here.

[0171] The following are embodiments of the device of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention.

[0172] Figure 7 FIG. 1 is a schematic diagram of an exposure control device for a vehicle-mounted camera provided by an embodiment of the present invention. Figure 7As shown, the exposure control device 700 of the vehicle-mounted camera includes: an acquisition module 701, a prediction module 702, a determination module 703, and a control module 704, wherein:

[0173] The acquisition module 701 is configured to acquire at least two frames of images captured by a camera on the vehicle when a light change scene is detected within a preset range in front of the vehicle;

[0174] The prediction module 702 is configured to predict a target brightness ratio of a frame image corresponding to a light change scene based on a brightness ratio of each frame image in at least two frames of images, where the brightness ratio is a brightness ratio between a light change area and a ground lane area in the image;

[0175] A determination module 703 is configured to determine a target parameter value of an exposure control parameter of the camera within a preset period of time based on the target brightness ratio, where the preset period of time is determined based on the time when the camera enters a scene with light changes;

[0176] The control module 704 is configured to control the camera to collect images based on the target parameter value within a preset time period.

[0177] In some embodiments, the acquisition module 701 is configured to:

[0178] Acquire a forward-looking image captured by a forward-looking camera, and based on the forward-looking image, detect whether there is a light change scene within a preset range ahead through a preset scene recognition model; and / or,

[0179] Based on the vehicle's current location information, navigation information and map information, detect whether there is a light change scene within a preset range ahead;

[0180] The preset range ahead includes: the range of the road area along the vehicle's driving direction that the vehicle can travel within a preset time period in the future; or, the range of the road area corresponding to the preset road length ahead of the vehicle along the vehicle's driving direction.

[0181] In some embodiments, after the acquisition module 701 detects whether there is a light change scene within a preset range ahead based on the forward-view image through a preset scene recognition model, the acquisition module 701 is further configured to:

[0182] In the case where the front view image is overexposed or underexposed, image data of a preset number of frames adjacent to the front view image and exposure parameters corresponding to each frame are obtained; the image data of the front view image, the image data of the preset number of frames and the exposure parameters are updated to the training set of the scene recognition model to obtain an updated training set, and the updated training set is used to optimize the scene recognition model.

[0183] In some embodiments, the prediction module 702 is configured to: fit the brightness ratio of each frame image to obtain a fitting curve; based on the fitting curve, predict the brightness ratio of the frame image corresponding to the predicted arrival time, and determine the brightness ratio of the frame image corresponding to the predicted arrival time as the target brightness ratio of the frame image corresponding to the light change scene corresponding to the camera; wherein the predicted arrival time is the time when the camera arrives at the light change scene.

[0184] In some embodiments, prediction module 702 is configured to:

[0185] Determine the distance of the target between the forward-looking camera and the scene with changing light;

[0186] Determine the target distance between each other camera and the light changing scene based on the spacing distance between each other camera and the front view camera in the vehicle body length direction and the target distance between the front view camera and the light changing scene, each other camera is a camera on the vehicle except the front view camera;

[0187] Obtain the trajectory planning information of the vehicle, and determine the predicted arrival time of any camera to the light change scene based on the trajectory planning information and the target distance corresponding to any camera.

[0188] In some embodiments, prediction module 702 is configured to:

[0189] Acquire real-time position information of any forward-looking camera, and determine a first distance between the forward-looking camera and the light-changing scene based on the map information and the real-time position information of the forward-looking camera, and determine the first distance as a target distance between the forward-looking camera and the light-changing scene;

[0190] Acquire image information of an image captured by the front-view camera, and determine a second distance between the front-view camera and the light-changing scene based on the image information, and determine the second distance as a target distance between the front-view camera and the light-changing scene;

[0191] Acquire the real-time position information of any forward-looking camera, determine a first distance between the forward-looking camera and the light-changing scene based on the map information and the real-time position information of the forward-looking camera, acquire image information of the image captured by the forward-looking camera, determine a second distance between the forward-looking camera and the light-changing scene based on the image information, and determine a target distance between the forward-looking camera and the light-changing scene based on the first distance, the second distance, the positioning confidence corresponding to the first distance, and the image perception confidence corresponding to the second distance.

[0192] In some embodiments, prediction module 702 is configured to:

[0193] Semantic segmentation is performed on each of at least two frames of images to obtain light change areas and ground lane areas in the images; an average brightness value of the light change areas and an average brightness value of the ground lane areas in each frame of images are determined, and a ratio of the average brightness value of the light change areas in each frame of images to the average brightness value of the ground lane areas is determined as the brightness ratio of each frame of images.

[0194] In some embodiments, the determination module 703 is configured to:

[0195] Based on a preset correspondence between a preset brightness ratio interval and an exposure control parameter adjustment strategy, a target adjustment strategy corresponding to the target brightness ratio is determined; based on the target adjustment strategy, the exposure control parameters of the camera are adjusted to obtain a target parameter value; wherein the exposure control parameters include at least one of exposure duration, analog gain and digital gain.

[0196] In some embodiments, when the range of the preset brightness ratio interval is greater than or equal to 1, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is: the exposure duration of the current frame is the product of the exposure duration of the previous frame and the first preset multiple of the current frame image target brightness ratio, the analog gain of the current frame is the product of the analog gain of the previous frame and the first preset multiple of the current frame image target brightness ratio, the digital gain of the current frame is the product of the digital gain of the previous frame and the first preset multiple of the current frame image target brightness ratio, wherein the first preset multiple is less than 1, and the higher the value of the preset brightness ratio interval in different preset brightness ratio intervals, the smaller the first preset multiple;

[0197] When the range of the preset brightness ratio interval is less than 1, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is: the exposure duration of the current frame is the product of the exposure duration of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, the analog gain of the current frame is the product of the analog gain of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, and the digital gain of the current frame is the product of the digital gain of the previous frame and the second preset multiple of the target brightness ratio of the current frame image, wherein the second preset multiple is greater than 1, and the higher the value of the preset brightness ratio interval in different preset brightness ratio intervals, the larger the second preset multiple.

[0198] Figure 8 Schematic diagram of an electronic device 8 provided by an embodiment of the present invention. Figure 8 As shown, the electronic device 8 of this embodiment includes: a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the processor 801. When the processor 801 executes the computer program 803, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 801 executes the computer program 803, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0199] The electronic device 8 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 8 may include, but is not limited to, a processor 801 and a memory 802. Those skilled in the art will appreciate that Figure 8 The electronic device 8 is merely an example and does not limit the electronic device 8 . The electronic device 8 may include more or fewer components than those shown in the figure, or different components.

[0200] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0201] The memory 802 may be an internal storage unit of the electronic device 8, for example, a hard disk or memory of the electronic device 8. The memory 802 may also be an external storage device of the electronic device 8, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 8. The memory 802 may also include both an internal storage unit of the electronic device 8 and an external storage device. The memory 802 is used to store computer programs and other programs and data required by the electronic device.

[0202] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0203] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (such as a computer-readable storage medium). Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.

[0204] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. An exposure control method for a vehicle-mounted camera, characterized in that: include: When a light-changing scene is detected in a preset range in front of the vehicle, at least two frames of images captured by a camera on the vehicle are acquired, wherein the at least two frames of images are acquired before reaching the light-changing scene; Based on the brightness ratio of each frame image in the at least two frames of images, predicting a target brightness ratio of the frame image corresponding to the light change scene when the vehicle passes through, the brightness ratio being a brightness ratio between a light change area and a ground lane area in the image; Based on the target brightness ratio, determine a target parameter value of an exposure control parameter of the camera within a preset time period, and control the camera to perform image acquisition based on the target parameter value within the preset time period, wherein the preset time period is determined based on the time when the camera enters the light change scene, and the preset time period includes the time when the camera enters the light change scene, and also includes a first preset time length before the camera enters the light change scene and / or a second preset time length after the camera enters the light change scene; The step of predicting a target brightness ratio of a frame image corresponding to a scene where the vehicle passes through the light change scene based on the brightness ratio of each frame image in the at least two frames of images includes: Fitting the brightness ratio of each frame image to obtain a fitting curve; Based on the fitting curve, the brightness ratio of the frame image corresponding to the predicted arrival time is predicted, and the brightness ratio of the frame image corresponding to the predicted arrival time is determined as the target brightness ratio of the frame image corresponding to the camera when the vehicle passes through the light change scene; wherein the predicted arrival time is the time when the camera arrives at the light change scene.

2. The method according to claim 1, characterized in that Detects light changes in the preset range in front of the vehicle, including: Acquire a forward-looking image captured by a forward-looking camera, and based on the forward-looking image, detect whether the light change scene exists within a preset range ahead through a preset scene recognition model; and / or, Based on the current position information, navigation information and map information of the vehicle, detecting whether the light change scene exists within a preset range ahead; The preset front range includes: the road area range that the vehicle can travel within a preset time period in the future along the driving direction of the vehicle; or, the road area range corresponding to the preset road length in front of the vehicle along the driving direction of the vehicle.

3. The method according to claim 2, characterized in that After detecting whether there is the light change scene in a preset range ahead based on the front view image by using a preset scene recognition model, the method further includes: In the case where the front view image is overexposed or underexposed, acquiring image data of a preset number of frames adjacent to the front view image and exposure parameters corresponding to each frame; The image data of the forward-view image, the image data of the preset number of frames and the exposure parameters are updated into the training set of the scene recognition model to obtain an updated training set, and the updated training set is used to optimize the scene recognition model.

4. The method according to claim 1, characterized in that Before predicting the brightness ratio of the frame image corresponding to the predicted arrival time based on the fitting curve, the method further includes: Determining a target distance between a front-view camera and the light-changing scene; Determining a target distance between each other camera and the light changing scene based on a spacing distance between each other camera and the front-view camera in a vehicle body length direction and a target distance between the front-view camera and the light changing scene, wherein each other camera is a camera on the vehicle other than the front-view camera; The trajectory planning information of the vehicle is obtained, and based on the trajectory planning information and the target distance corresponding to any camera, a predicted arrival time of the any camera to the light change scene is determined.

5. The method according to claim 4, characterized in that The method of determining the target distance between the front-view camera and the light-changing scene includes: Acquire the real-time position information of any forward-looking camera, and determine a first distance between the forward-looking camera and the light-changing scene based on the map information and the real-time position information of the forward-looking camera, and determine the first distance as the target distance between the forward-looking camera and the light-changing scene; or, Acquire image information of the image captured by the front-view camera, and determine a second distance between the front-view camera and the light-changing scene based on the image information, and determine the second distance as a target distance between the front-view camera and the light-changing scene; or, Acquire the real-time position information of any forward-looking camera, determine a first distance between the forward-looking camera and the light-changing scene based on map information and the real-time position information of the forward-looking camera, acquire image information of the image captured by the forward-looking camera, determine a second distance between the forward-looking camera and the light-changing scene based on the image information, and determine a target distance between the forward-looking camera and the light-changing scene based on the first distance, the second distance, the positioning confidence corresponding to the first distance, and the image perception confidence corresponding to the second distance.

6. The method according to claim 1, characterized in that Also includes: Performing semantic segmentation on each of the at least two frames of images to obtain a light change area and a ground lane area in the image; Determine the average brightness of the light change area in each frame of the image and the average brightness of the ground lane area, and determine the ratio of the average brightness of the light change area in each frame of the image to the average brightness of the ground lane area as the brightness ratio of each frame of the image.

7. The method according to any one of claims 1, 4, 5 and 6, characterized in that The step of determining a target parameter value of an exposure control parameter of the camera within a preset time period based on the target brightness ratio includes: Determining a target adjustment strategy corresponding to the target brightness ratio based on a preset correspondence between a preset brightness ratio interval and an exposure control parameter adjustment strategy; The exposure control parameter of the camera is adjusted based on the target adjustment strategy to obtain the target parameter value; wherein the exposure control parameter includes at least one of exposure duration, analog gain and digital gain.

8. The method according to claim 7, characterized in that The preset corresponding relationship between the preset brightness ratio interval and the exposure control parameter adjustment strategy includes: In the case where the range of the preset brightness ratio interval is greater than or equal to 1, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is: the exposure duration of the current frame is the product of the exposure duration of the previous frame and the first preset multiple of the current frame image target brightness ratio, the analog gain of the current frame is the product of the analog gain of the previous frame and the first preset multiple of the current frame image target brightness ratio, and the digital gain of the current frame is the product of the digital gain of the previous frame and the first preset multiple of the current frame image target brightness ratio, wherein the first preset multiple is less than 1, and the higher the value of the preset brightness ratio interval in different preset brightness ratio intervals, the smaller the first preset multiple; When the range of the preset brightness ratio interval is less than 1, for the same preset brightness ratio interval, the exposure control parameter adjustment strategy is: the exposure duration of the current frame is the product of the exposure duration of the previous frame and the second preset multiple of the current frame image target brightness ratio, the analog gain of the current frame is the product of the analog gain of the previous frame and the second preset multiple of the current frame image target brightness ratio, and the digital gain of the current frame is the product of the digital gain of the previous frame and the second preset multiple of the current frame image target brightness ratio, wherein the second preset multiple is greater than 1, and the higher the value of the preset brightness ratio interval in different preset brightness ratio intervals, the larger the second preset multiple.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images