Camera control method and device, vehicle and storage medium

Abstract

The invention provides a camera control method and device, a vehicle and a storage medium, the method is applied to the vehicle, the vehicle comprises at least two types of cameras, and the method comprises the steps that the exposure center moment of each type of camera in the at least two types of cameras is acquired; determining a target moment based on the latest moment in the exposure center moments of the cameras of each type, calculating a difference value between the exposure center moments of the cameras of each type and the target moment, and obtaining a target delay duration of the exposure center moments of the cameras of each type; wherein the target moment is greater than or equal to the latest moment; and based on the target delay duration corresponding to each type of camera, controlling the acquisition moment of each type of camera for acquiring the target image, so that the timestamps of the target images acquired by each type of camera are the same, and the method enables each type of camera to realize time synchronization, and improves the accuracy of automatic driving control.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a camera control method, device, vehicle, and storage medium in the field of vehicle technology. Background Technology

[0002] Visual perception is an important component of autonomous driving. Currently, autonomous driving requires increasingly precise perception and control functions. Generally, multiple different cameras are installed on the vehicle to collect information about the vehicle's driving environment in order to meet the perception and control needs of autonomous driving.

[0003] Currently, there is a problem with inconsistent timestamps in images acquired by multiple different types of cameras. This can lead to stitching errors or perception errors in multiple images, thus affecting the accuracy of autonomous driving control of vehicles.

[0004] Therefore, improving the accuracy of autonomous driving control of vehicles has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a camera control method, device, vehicle, and storage medium. The method delays the image acquisition time of each camera, so that the exposure center point of each type of camera is at the same time, and each type of camera can achieve time synchronization, thereby improving the accuracy of autonomous driving control.

[0006] In a first aspect, a method for controlling a camera is provided, the method being applied to a vehicle including at least two types of cameras, the method comprising:

[0007] Obtain the exposure center moment for each of at least two types of cameras;

[0008] The target time is determined based on the latest time among the exposure center times of each type of camera. The difference between the exposure center time and the target time for each type of camera is calculated to obtain the target delay time of the exposure center time for each type of camera. The target time is greater than or equal to the latest time.

[0009] Based on the target delay duration corresponding to each type of camera, the acquisition time of target images by each type of camera is controlled so that the timestamps of target images acquired by each type of camera are the same.

[0010] In the above technical solution, the exposure center time of each type of camera is obtained; the target time is determined based on the latest time among the exposure center times of each type of camera; the target delay time of the exposure center time of each type of camera is obtained by calculating the difference between the exposure center time and the target time; and the acquisition time of the target image by each type of camera is controlled based on the target delay time corresponding to each type of camera. Since the exposure center time can be used as the timestamp of the image, by setting a unified target time and calculating the difference between the exposure center time and the target time of each type of camera, the image acquisition time of each type of camera can be delayed towards the target time based on the difference, so that each type of camera can acquire the target image at the same time, thereby synchronizing the timestamps of each type of camera. This achieves time synchronization of multiple types of cameras in the vehicle. Because of the time synchronization of multiple types of cameras in the vehicle, the accuracy of the stitched image generated based on the target images acquired by each type of camera can be ensured to be higher, thereby improving the accuracy of autonomous driving control of the vehicle.

[0011] In conjunction with the first aspect, in some possible implementations, the difference between the exposure center time and the target time for each type of camera is calculated, including:

[0012] Determine if the target time is an integer; if the target time is not an integer, round it to obtain an integer target time; calculate the difference between the exposure center time and the target integer time for each type of camera.

[0013] In the above technical solution, since the final image acquisition time of each type of camera is based on the target time, the target time is rounded down when it is not an integer. Rounding down the target time makes it easier to calculate the time difference corresponding to each type of camera, thus improving the ease of calculating the time difference corresponding to each type of camera.

[0014] Combining the first aspect and the above implementation methods, in some possible implementation methods, obtaining the exposure center time of each of at least two types of cameras includes:

[0015] Obtain the preset exposure duration and attribute information for each type of camera; the attribute information includes the total transmission time of one frame of image data and the transmission delay time of one frame of image data, and the transmission delay time is used to characterize the time difference between the time when the camera transmits image data and the time when it receives the image acquisition signal; based on the preset exposure duration and attribute information, calculate the exposure center time for each type of camera.

[0016] In conjunction with the first aspect and the above implementation methods, in some possible implementation methods, at least two types of cameras are included, namely, a front-view camera, a side-view camera, and a fisheye camera. The preset exposure time of each type of camera is obtained, including: determining the preset exposure time of the front-view camera based on the flash period of the traffic light; determining the preset exposure time of the side-view camera based on the current angular velocity of the vehicle; and determining the preset exposure time of the fisheye camera based on the brightness information of the current shooting environment.

[0017] In the above technical solutions, since the forward-facing camera is generally used to identify information ahead of the road (e.g., traffic light status), and the vehicle can be controlled to move or stop based on the status of the traffic lights, determining the preset exposure time of the forward-facing camera based on the flashing period of the traffic lights ensures that the forward-facing camera can capture images when the traffic lights are on, avoiding the problem of not being able to identify traffic lights when capturing images when the traffic lights are off. The side-facing camera is generally installed on the side of the vehicle, and the relative angular velocity between the vehicle and the surrounding environment is large when the vehicle is traveling at high speed. Therefore, determining the preset exposure time of the side-facing camera based on the current angular velocity of the vehicle can prevent motion blur caused by excessive exposure time. The fisheye camera is generally used for parking, and since there are many low-light scenes in underground parking garages, determining the preset exposure time of the fisheye camera based on the brightness information of the current shooting environment can brighten the image captured in low-light scenes, obtain a clearer image, and thus control the vehicle more accurately.

[0018] Combining the first aspect and the above implementation methods, in some possible implementation methods, the preset exposure time of the forward-looking camera is determined based on the flash period of the traffic light, including:

[0019] If traffic lights are present in the current shooting environment, the preset exposure time of the forward-facing camera is determined based on the flash period of the traffic lights in the current shooting environment; if traffic lights are not present in the current shooting environment, the preset exposure time of the forward-facing camera is determined based on the flash period of the traffic lights in the historical shooting environment.

[0020] In the above technical solution, if there are traffic lights in the current shooting environment, the preset exposure time of the forward-facing camera is determined based on the flash period of the traffic lights in the current shooting environment; this makes the determination of the preset exposure time more accurate and the recognition of the traffic light status more accurate. If there are no traffic lights in the current shooting environment, the preset exposure time of the forward-facing camera is determined based on the flash period of traffic lights in historical shooting environments. Since the flash period of each traffic light is usually a fixed period, the preset exposure time of the forward-facing camera obtained based on the flash period of traffic lights in historical shooting environments is more accurate when there are no traffic lights in the current environment.

[0021] Combining the first aspect and the above implementation methods, in some possible implementation methods, at least two types of cameras use progressive exposure mode.

[0022] In the above technical solutions, the progressive exposure mode can optimize image quality by precisely controlling the exposure time of each line; in addition, progressive exposure has a shorter exposure time, which is suitable for shooting scenarios with high frame rate requirements.

[0023] Combining the first aspect and the above implementation methods, in some possible implementation methods, based on the target delay duration corresponding to each type of camera, the acquisition time of the target image by each type of camera is controlled, including:

[0024] Replace the target parameter in each type of camera with the target delay duration corresponding to each type of camera; where the target parameter is used to characterize the time difference between the moment the image acquisition signal is received and the moment the exposure of the first row of pixels of the target image ends.

[0025] In the above technical solution, the target parameter in each type of camera is replaced with the target delay duration corresponding to each type of camera. Since the target parameter is the time difference between the moment the image acquisition signal is received and the moment the exposure of the first row of pixels of the target image ends, by replacing the target parameter with the target delay duration, the shooting time (i.e., the exposure time) of each type of camera can be delayed to the target time, thereby achieving uniformity of shooting time for various types of cameras; ensuring that the timestamps when various types of cameras acquire target images are the same, and achieving time synchronization of various types of cameras.

[0026] Secondly, a camera control device is provided, the device comprising:

[0027] The acquisition module is used to acquire the exposure center time of each of at least two types of cameras;

[0028] The calculation module determines the target time based on the latest time among the exposure center times of each type of camera, calculates the difference between the exposure center time and the target time for each type of camera, and obtains the target delay duration of the exposure center time for each type of camera; wherein, the target time is greater than or equal to the latest time.

[0029] The control module is used to control the acquisition time of target images by each type of camera based on the target delay duration corresponding to each type of camera, so that the timestamps of the target images acquired by each type of camera are the same.

[0030] In conjunction with the second aspect, in some possible implementations, the calculation module is specifically used to determine whether the target time is an integer time; if the target time is not an integer time, the target time is rounded down to obtain an integer target time; and the difference between the exposure center time of each type of camera and the target integer time is calculated.

[0031] Combining the second aspect and the above implementation methods, in some possible implementation methods, the acquisition module is specifically used to acquire the preset exposure duration and attribute information of each type of camera; wherein, the attribute information includes the total transmission duration of one frame of image data and the transmission delay duration of one frame of image data, and the transmission delay duration is used to characterize the time difference between the time when the camera transmits image data and the time when it receives the image acquisition signal; based on the preset exposure duration and attribute information, the exposure center time of each type of camera is calculated.

[0032] In combination with the second aspect and the above implementation methods, in some possible implementation methods, at least two types of cameras include a front-view camera, a side-view camera, and a fisheye camera. The acquisition module is specifically used to determine the preset exposure time of the front-view camera based on the flashing period of the traffic light; to determine the preset exposure time of the side-view camera based on the current angular velocity of the vehicle; and to determine the preset exposure time of the fisheye camera based on the brightness information of the current shooting environment.

[0033] Combining the second aspect and the above implementation methods, in some possible implementation methods, the acquisition module is specifically used to determine the preset exposure time of the forward-looking camera based on the flash period of the traffic lights in the current shooting environment if there are traffic lights in the current shooting environment; and to determine the preset exposure time of the forward-looking camera based on the flash period of the traffic lights in the historical shooting environment if there are no traffic lights in the current shooting environment.

[0034] Combining the second aspect and the above implementation methods, in some possible implementation methods, at least two types of cameras use progressive exposure mode.

[0035] Combining the second aspect and the above implementation methods, in some possible implementation methods, the control module is specifically used to replace the target parameters in each type of camera with the target delay duration corresponding to each type of camera; wherein, the target parameters are used to characterize the time difference between the moment when the image acquisition signal is received and the moment when the exposure of the first row of pixels of the target image ends.

[0036] Thirdly, a vehicle is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the vehicle to perform the methods described in the first aspect or any possible implementation thereof.

[0037] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0038] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof. Attached Figure Description

[0039] Figure 1 This is a camera runtime sequence diagram in a shutter external trigger mode provided in an embodiment of this application;

[0040] Figure 2 This is a runtime sequence diagram of a camera in readout external trigger mode provided in an embodiment of this application;

[0041] Figure 3 This is a schematic flowchart illustrating a camera control method provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the installation positions of various types of cameras provided in an embodiment of this application;

[0043] Figure 5 This is a runtime sequence diagram of various types of cameras provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of the structure of a camera control device provided in an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this application. Detailed Implementation

[0046] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0047] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0048] First, the nouns and terms appearing in the embodiments of this application will be explained.

[0049] Rolling shutter exposure mode: also known as line-by-line exposure mode, where the exposure time for each row of pixels in the image is the same, but the exposure start time for different rows is different. For example, if the exposure time for each row of pixels is set to 10ms, then the exposure time for the k-th row and the (k+1)-th row of pixels is also 10ms; however, the (k+1)-th row of pixels will start exposure 4.87µs later than the k-th row of pixels. Therefore, the exposure time range for the k-th row of pixels is from 0.00000ms to 10.00000ms, while the exposure time range for the (k+1)-th row of pixels is from 0.00487ms to 10.00487ms.

[0050] Shutter external trigger mode: an external trigger image output mode. Generally, trigger image output modes include internal trigger mode and external trigger mode. In internal trigger mode, the start and stop of image acquisition are controlled by software, and the image acquisition frame rate is controlled by software. In external trigger mode, after the camera starts acquiring images, it waits for external electrical signal pulses. One pulse is used to acquire one image, and the frame rate is controlled by the frequency of the pulses.

[0051] The advantage of external trigger mode is that multiple cameras can simultaneously acquire data through a single trigger source.

[0052] The following is combined Figure 1 This section introduces the shutter external trigger mode.

[0053] Figure 1 This is a camera runtime sequence diagram in a shutter external trigger mode provided in an embodiment of this application.

[0054] For example, such as Figure 1 As shown, in shutter external trigger mode, the exposure start time of the first row of pixels of the camera is aligned with the external trigger signal time. When the trigger signal is received, the first row of pixels of the image begins to be exposed. After a certain exposure time, the first row of exposure ends and image data transmission begins. The remaining rows of pixels are exposed line by line and the data transmission is completed.

[0055] The shutter-triggered mode has the following two characteristics: 1. The data transmission time varies with the exposure time, meaning the delay in receiving image data downstream changes with the exposure time; 2. When the exposure time changes from a long exposure time to a short exposure time, due to the large change in the exposure step size, the next frame of image data may be dropped because the previous frame of image data has not been completely transmitted. The shutter-triggered mode allows multiple cameras to have the same exposure start time.

[0056] Readout external trigger mode: Another external trigger mode for outputting graphs, described below in conjunction with... Figure 2 This section introduces the readout external trigger mode.

[0057] Figure 2 This is a camera runtime sequence diagram in a readout external trigger mode provided in an embodiment of this application.

[0058] For example, such as Figure 2 As shown, in readout external trigger mode, the end time of exposure of the first row of pixels in the camera is aligned with the time of the external trigger signal. The first row of pixels is exposed before the external trigger signal to ensure that the end time of exposure of the first row of pixels is aligned with the time of receiving the external trigger signal. The remaining rows of pixels are exposed row by row and data transmission is completed.

[0059] The readout external trigger mode has the following two characteristics: 1. The data transmission time does not change with the exposure time, that is, the delay of the downstream receiving the image data is stable; 2. It will not experience the frame dropping phenomenon similar to the shutter external trigger mode due to large changes in the exposure time of consecutive frames.

[0060] In the field of autonomous driving, sensors are mainly used to perceive the vehicle's surrounding environment. The vehicle makes correct control decisions based on the perception results. Among various sensors, cameras contain the most intuitive and rich information. Therefore, multiple cameras of different types are usually set up to meet different environmental perception needs.

[0061] Currently, the vision-centric bird's-eye view (BEV) of autonomous driving algorithms has strict requirements for the time synchronization of multiple cameras. Images taken at the same time need to be transformed into a unified overhead view for stitching. If the shooting times of multiple cameras are not synchronized, stitching errors and perception errors will occur, thus affecting the autonomous driving control of the vehicle.

[0062] Since the timestamp of an image is related to the time of receiving the trigger signal, the exposure time of the camera, and its attributes, such as the duration of transmitting a frame of image data, and since most multi-camera systems currently use dynamic exposure mode, the exposure time of different cameras varies, and their attributes also differ. Therefore, the timestamps of images captured by each camera cannot be exactly the same, which may lead to splicing or perception errors in multiple images, thus affecting the accuracy of autonomous driving control of the vehicle.

[0063] To address the aforementioned technical problems, embodiments of this application provide a camera control method, device, vehicle, and storage medium. This method delays the image acquisition time of each camera, ensuring that the exposure center point of each type of camera is at the same moment, enabling time synchronization for each type of camera and improving the accuracy of autonomous driving control.

[0064] Figure 3 This is a schematic flowchart of a camera control method provided in an embodiment of this application. This method can be applied to computing devices with computing processing functions in vehicles, such as microcontroller units (MCUs).

[0065] For example, such as Figure 3 As shown, the method 300 includes S310 to S330, which are described in detail below.

[0066] S210, acquire the exposure center time of each of at least two types of cameras.

[0067] Among them, at least two types of cameras may include front-view cameras, rear-view cameras, side-view cameras and fisheye cameras. Optionally, the front-view camera and the rear-view camera may be the same type of camera. It should be understood that different types of cameras have different resolutions.

[0068] Optionally, the number of each type of camera can be one or more, and this application embodiment does not specifically limit this.

[0069] Optionally, at least two types of cameras use progressive exposure mode, also known as rolling shutter mode. In progressive exposure mode, the exposure time of each row of pixels is the same, but the exposure start time of different rows is different. Since the camera does not capture images instantaneously, in order to make the timestamp of the captured image more accurate and reasonable, in progressive exposure mode, the time when half of the pixels in the middle row of the image are exposed is generally used as the timestamp of the image, that is, the exposure center time is used as the timestamp.

[0070] In the above technical solutions, the progressive exposure mode can optimize image quality by precisely controlling the exposure time of each line; in addition, progressive exposure has a shorter exposure time, which is suitable for shooting scenarios with high frame rate requirements.

[0071] It should be understood that the exposure center time is generally the time when the trigger signal is received plus the exposure center delay. When the trigger signal reception time is the same for different types of cameras, the exposure center time depends on the exposure center delay. However, the exposure time, data transmission time, and data transmission delay of different types of cameras are different. Therefore, the exposure center delay of different types of cameras is different, which leads to a certain difference in the exposure center time of different types of cameras.

[0072] For example, the exposure center time of the front-view camera is T1, the exposure center time of the side-view camera is T2, and the exposure center time of the fisheye camera is T3. The relationship between T1, T2, and T3 is not specifically limited in this embodiment of the application.

[0073] In some embodiments, the exposure center time can be determined based on the time of receiving the trigger signal (i.e., the image acquisition signal) and the delay duration of the exposure center, that is, T 曝光中心 =T 触发信号 +t 曝光中心延迟 It should be noted that in the embodiments of this application, T represents time and t represents duration.

[0074] The delay time of the exposure center is related to the exposure time and attribute information of the camera. Therefore, the delay time of the exposure center can be determined according to the preset exposure time and attribute information of each type of camera, and thus the exposure center time of the camera can be determined.

[0075] Specifically, the process of obtaining the exposure center time of each type of camera in at least two types of cameras can be as follows: obtain the preset exposure duration and attribute information of each type of camera; and calculate the exposure center time of each type of camera based on the preset exposure duration and attribute information.

[0076] The preset exposure time for different types of cameras can be set by those skilled in the art, and this application does not impose specific limitations on this.

[0077] The camera's attribute information includes the total transmission time of one frame of image data and the transmission delay time of one frame of image data. The transmission delay time characterizes the time difference between the moment the camera transmits image data and the moment it receives the image acquisition signal. It should be noted that the initiation and reception times of the image acquisition signal generally differ by a few microseconds; therefore, the initiation and reception times of the image acquisition signal can be considered as the same moment.

[0078] Optionally, the camera's attribute information can be obtained from a database that stores camera attribute information, where there is a one-to-one correspondence between camera types and attribute information.

[0079] Furthermore, with a preset exposure time of t 曝光 The total transmission time of one frame of image data is t. 数据传输 The transmission delay is t. 传输延迟 For example, the exposure center time of the camera can be obtained using the following formula:

[0080] T 曝光中心 =T 触发信号 +t 传输延迟 +t 数据传输 / 2-t 曝光 / 2;

[0081] Taking three types of cameras as an example, we can get three Ts. 曝光中心 They are T1, T2 and T3 respectively.

[0082] In some embodiments, the preset exposure time of different types of cameras can be set according to actual conditions. For example, in addition to recognizing lane lines and obstacles, front-view cameras can also recognize traffic lights (e.g., traffic lights). Therefore, the preset exposure time of the front-view camera can be determined based on the flashing period of the traffic lights. Side-view cameras are generally installed at the front and / or rear of the vehicle, forming a 45° angle with the vehicle's X-axis. The relative angular velocity between the vehicle and the surrounding environment is relatively large when the vehicle is moving. Therefore, the preset exposure time of the test camera can be determined based on the angular velocity of the vehicle. Fisheye cameras are generally installed at the front, rear, and sides of the vehicle for use when parking. Since there are many low-light scenes such as underground parking garages, the preset exposure time of the fisheye camera can be determined based on the brightness of the shooting environment.

[0083] Therefore, the process of obtaining the preset exposure time for each type of camera can be as follows: determine the preset exposure time for the forward-looking camera based on the flashing period of the traffic light; determine the preset exposure time for the side-looking camera based on the current angular velocity of the vehicle; and determine the preset exposure time for the fisheye camera based on the brightness information of the current shooting environment.

[0084] It should be understood that traffic lights typically flash before switching from green to red or vice versa. If the exposure time of the forward-facing camera does not meet the complete flashing cycle of the traffic light, it may capture an image of the traffic light being off, thus affecting the perception and judgment of autonomous driving. Therefore, to ensure the accuracy and safety of autonomous driving, the exposure time of the forward-facing camera must cover at least one complete flashing cycle of the traffic light. For example, if the flashing cycle of the traffic light is 10ms, then the exposure time must be at least 10ms.

[0085] In some embodiments, since the current shooting environment of the forward-viewing camera includes both the presence and absence of traffic lights, the process of determining the preset exposure time of the forward-viewing camera based on the flash period of the traffic lights can be as follows: if there are traffic lights in the current shooting environment, determine the preset exposure time of the forward-viewing camera based on the flash period of the traffic lights in the current shooting environment; if there are no traffic lights in the current shooting environment, determine the preset exposure time of the forward-viewing camera based on the flash period of the traffic lights in historical shooting environments.

[0086] For example, the flashing period of the traffic lights in the current shooting environment can be determined by the captured video image, and the determined flashing period value is stored in the database; if there are no traffic lights in the current shooting environment, the flashing period of traffic lights in historical shooting environments is obtained from the database, thereby determining the preset exposure time of the forward-looking camera.

[0087] In the above technical solution, if there are traffic lights in the current shooting environment, the preset exposure time of the forward-facing camera is determined based on the flash period of the traffic lights in the current shooting environment; this makes the determination of the preset exposure time more accurate and the recognition of the traffic light status more accurate. If there are no traffic lights in the current shooting environment, the preset exposure time of the forward-facing camera is determined based on the flash period of traffic lights in historical shooting environments. Since the flash period of each traffic light is usually a fixed period, the preset exposure time of the forward-facing camera obtained based on the flash period of traffic lights in historical shooting environments is more accurate when there are no traffic lights in the current environment.

[0088] Furthermore, if the vehicle's angular velocity is large (e.g., 100 km / h), an excessively long exposure time will result in motion blur. Therefore, a correspondence between angular velocity and exposure time can be set, as shown in Table 1 below. The exposure time is determined based on the vehicle's current angular velocity and this correspondence.

[0089] Table 1

[0090] Angular velocity (km / h) Preset exposure time V>＝100 10ms 50<＝V<100 15ms V<50 20ms … …

[0091] For example, as shown in Figure 1, V represents angular velocity. When the angular velocity is greater than or equal to 100 km / h, the exposure time can be determined to be 10 ms; when the angular velocity is greater than or equal to 50 km / h and less than 100 km / h, the exposure time can be determined to be 15 ms; when the angular velocity is less than 50 km / h, the exposure time can be determined to be 20 ms. It should be understood that the exposure time corresponding to the angular velocity in Figure 1 is only an example, and those skilled in the art can set it themselves. This application embodiment does not specifically limit it in this way.

[0092] Optionally, the vehicle's current angular velocity can be obtained by an angular velocity sensor installed on the vehicle, and the preset exposure time of the side-view camera can be determined based on the obtained current angular velocity value and by looking up Table 1.

[0093] For example, fisheye cameras are generally used in low-light scenes such as parking lots. Therefore, if the exposure time is too short, the image will be unclear and affect parking control. Therefore, a light sensor or camera can be used to obtain the brightness information of the shooting environment of the fisheye camera, and the exposure time can be determined based on the brightness information. For example, when the light is low, a longer exposure time (e.g., 15ms) can be set to improve the image brightness in low-light fields.

[0094] In the above technical solutions, since the forward-facing camera is generally used to identify information ahead of the road (e.g., traffic light status), and the vehicle can be controlled to move or stop based on the status of the traffic lights, determining the preset exposure time of the forward-facing camera based on the flashing period of the traffic lights ensures that the forward-facing camera can capture images when the traffic lights are on, avoiding the problem of not being able to identify traffic lights when capturing images when the traffic lights are off. The side-facing camera is generally installed on the side of the vehicle, and the relative angular velocity between the vehicle and the surrounding environment is large when the vehicle is traveling at high speed. Therefore, determining the preset exposure time of the side-facing camera based on the current angular velocity of the vehicle can prevent motion blur caused by excessive exposure time. The fisheye camera is generally used for parking, and since there are many low-light scenes in underground parking garages, determining the preset exposure time of the fisheye camera based on the brightness information of the current shooting environment can brighten the image captured in low-light scenes, obtain a clearer image, and thus control the vehicle more accurately.

[0095] S220: Determine the target time based on the latest time among the exposure center times of each type of camera, calculate the difference between the exposure center time and the target time for each type of camera, and obtain the target delay duration of the exposure center time for each type of camera.

[0096] Among them, the target time is greater than or equal to the latest time.

[0097] For example, if the order of T1, T2 and T3 in the obtained exposure center time is T2>T1>T3, then T2 can be taken as the target time, and the time difference between T2 and T2, the time difference between T1 and T2, and the time difference between T3 and T2 can be calculated respectively to obtain the target delay time of the exposure center time for each type of camera.

[0098] Alternatively, any time T4 greater than T2 can be used as the target time. Then, the time difference between T2 and T4, the time difference between T1 and T4, and the time difference between T3 and T4 can be calculated to obtain the target delay time of the exposure center time for each type of camera.

[0099] Taking the target time as T2 as an example, the time difference between T2 and T2 is △S1=0ms, so the target delay time of the side-view camera is 0ms; the time difference between T1 and T2 is △S2=0.5ms, so the target delay time of the front-view camera is 0.5ms; the time difference between T3 and T2 is △S3=1.5ms, so the target delay time of the fisheye camera is 1.5ms.

[0100] In some embodiments, to facilitate subsequent verification of the trigger signal, when calculating the target delay time of the exposure center time of each type of camera, the target time can be rounded up. Specifically, the process of calculating the difference between the exposure center time and the target time for each type of camera can be as follows: determine whether the target time is an integer; if the target time is not an integer, round the target time to obtain an integer target time; calculate the difference between the exposure center time and the target integer time for each type of camera.

[0101] In this context, integer timestamps are integer timestamps, such as XX hours 20 microseconds.

[0102] For example, if T2 is the target time and T2 = T 触发信号 +10.64ms can be considered as the target time not being an integer. Further, by rounding the target time, we obtain the integer target time as T. 触发信号 +11ms is used to calculate the target delay time for various types of cameras, based on this integer target time.

[0103] Similarly, when selecting any time greater than T2 as the target time, an integer time can be selected as the target time.

[0104] In the above technical solution, when the target time is not an integer, the target time is rounded down. Since the final image acquisition time of each type of camera is based on the target time, rounding down the target time facilitates the calculation of the time difference corresponding to each type of camera, improves the ease of calculating the time difference corresponding to each type of camera, and makes it easier for each type of camera to obtain an integer timestamp. Since the specific timestamp is related to the trigger time of the image acquisition signal, the timestamp based on the integer time facilitates the verification of the specific trigger time of the image acquisition signal.

[0105] S230, based on the target delay duration corresponding to each type of camera, controls the acquisition time of target images acquired by each type of camera, so that the timestamps of target images acquired by each type of camera are the same.

[0106] For example, after determining the target delay duration of each type of camera in S220, the camera is further controlled to delay the time of image acquisition by the target delay duration based on the target delay duration of each type of camera. For example, if the initial time of image acquisition by the front-view camera is T0, the front-view camera is controlled to acquire the image at the time T0+△S2, that is, the exposure of each row of pixels is delayed by △S2 based on the initial exposure time.

[0107] Thus, the start time of the exposure of the middle row pixels of the image will be delayed by the target delay time. Taking the target delay time in the above embodiment as an example, the middle row pixels of the front-view camera are delayed by 0.5ms, the middle row pixels of the side-view camera are delayed by 0ms, and the middle row pixels of the fisheye camera are delayed by 1.5ms. Through the additional delay, the exposure center time of each camera is the same, and the timestamp is also the same.

[0108] In some embodiments, the triggering mode for image output by various types of cameras is the readout external triggering mode.

[0109] It should be understood that in the original readout external trigger mode, the exposure end time of the first row of pixels is the same as the external trigger signal. In this embodiment, by setting an additional target delay time for each type of camera, there is a delay of the target delay time between the exposure end time of the first row of pixels and the reception time of the external trigger signal, thereby delaying the exposure center time of each type of camera by the target delay time. This process can be specifically described as: replacing the target parameter in each type of camera with the target delay time corresponding to each type of camera.

[0110] The target parameter is a parameter in the camera's register, used to characterize the time difference between the moment the camera receives the image acquisition signal (external trigger signal) and the moment the exposure of the first row of pixels in the target image ends.

[0111] It should be understood that after replacing the target parameter with the target delay duration, the time difference between the end of the exposure of the first row of pixels and the rising edge of the trigger signal has the target delay duration. For example, in the original readout external trigger mode, the time difference between the end of the exposure of the first row of pixels and the receiving time of the external trigger signal is 0. By modifying the target parameter, the time difference between the end of the exposure of the first row of pixels and the receiving time of the external trigger signal is the target delay duration. This allows the exposure center time of each camera to be delayed by the target delay duration based on the original time, so that the exposure center time of each camera is the same.

[0112] For example, the exposure center time of the original forward-facing camera is T1 = T 触发信号 +9.3ms; the original exposure center time of the side-view camera was T2 = T 触发信号 +10.6ms; original T3 = T 触发信号 +7.1ms;

[0113] The above calculations yield the following target delay times: 10.6ms - 9.3ms = 1.3ms; 10.6ms - 10.6ms = 0ms; and 10.6ms - 7.1ms = 3.5ms. Further, by replacing the target parameter of the front-view camera with 1.3ms, the exposure center time of the front-view camera is delayed by 1.3ms, resulting in an exposure center time of T1 = T... 触发信号 +9.3ms + 1.3ms = T 触发信号 +10.6ms; Replace the target parameter of the fisheye camera with 3.5ms, which delays the exposure center time of the fisheye camera by 3.5ms. The final exposure center time of the fisheye camera is T1 = T... 触发信号 +7.1ms + 3.5ms = T 触发信号 +10.6ms; thus ensuring that the final T1 = T2 = T3, meaning that the exposure center time of all types of cameras is consistent.

[0114] In the above technical solution, the readout external trigger mode can stabilize the image data transmission link latency and prevent frame loss caused by large differences in exposure time between adjacent frames. The target parameter in each type of camera is replaced with the target delay duration corresponding to each type of camera. Since the target parameter is the time difference between the moment the image acquisition signal is received and the moment the exposure of the first row of pixels in the target image ends, by replacing the target parameter with the target delay duration, the shooting time (i.e., the exposure time) of each type of camera can be delayed to the target time, thereby achieving uniformity of shooting time for various types of cameras. This ensures that the timestamps when various types of cameras acquire target images are the same, achieving time synchronization for various types of cameras.

[0115] Below, in conjunction with Figure 4 and Figure 5 This application will be described with reference to a specific embodiment.

[0116] Figure 4 This is a schematic diagram of the installation positions of various types of cameras provided in the embodiments of this application.

[0117] For example, such as Figure 4 As shown, this embodiment of the application adopts a camera-integrated driving and parking solution. Three 8-megapixel cameras are arranged in the front view to identify lane lines, obstacles, traffic lights, etc., in front of the vehicle. One 8-megapixel camera is arranged in the rear view to identify the environment behind the vehicle. Four 2-megapixel cameras are arranged in the side view for driving visual perception, and four 2.7-megapixel fisheye cameras are arranged for parking visual perception. Specifically, the three front-view cameras and one rear-view camera are of the same model, the four side-view cameras are of the same model, and the four fisheye cameras are of the same model.

[0118] For example, based on the preset exposure duration and attribute information of the front and rear 8MP cameras, the side 2MP camera, and the fisheye 2.7MP camera, the exposure center time is calculated respectively:

[0119] Front and rear 8MP cameras: T1 = T 触发信号 +t 传输延迟1 +t 数据传输1 / 2-t 曝光1 / 2=T 触发信号 +t1;

[0120] Side view of a 2-megapixel camera: T2 = T 触发信号 +t 传输延迟2 +t 数据传输2 / 2-t 曝光2 / 2=T 触发信号 +t2;

[0121] 2.7MP fisheye camera: T3 = T 触发信号 +t 传输延迟3 +t 数据传输3 / 2-t 曝光3 / 2=T 触发信号 +t3;

[0122] Taking t1 = 9.32ms, t2 = 10.65ms, and t3 = 7.10ms as examples, to align the exposure center time, t2 can be used as the baseline. For front and rear 8-megapixel cameras, an additional delay of 10.65 - 9.32 = 1.33ms can be added; for fisheye 2.7-megapixel cameras, an additional delay of 10.65 - 7.10 = 3.55ms can be added. This will ensure that the exposure center time of the three types of cameras is aligned.

[0123] One possible implementation is to round t2 to 11ms, and then add an additional delay of 11-9.32=1.68ms for the 8MP rear-view camera and an additional delay of 11-7.10=3.9ms for the 2.7MP fisheye camera. This will ensure that the exposure center of the three types of cameras is aligned. The specific timestamp required can be achieved by adjusting the trigger signal timing. For example, if a timestamp of XX hour 25ms is required, the trigger signal can be initiated at XX hour 14ms.

[0124] Figure 5 This is a runtime sequence diagram of various types of cameras provided in the embodiments of this application.

[0125] For example, such as Figure 5 As shown, taking the target delay time of the front-view camera as D1, the target delay time of the side-view camera as D2, and the target delay time of the fisheye camera as D3 as examples, in order to align the exposure center times of the three types of cameras, for the front-view camera, the target parameter in the front-view camera register indicating the time difference between the end time of the first row of pixels and the time of receiving the trigger signal is modified to D1, so that there is a delay of D1 between the end time of the first row of pixels and the time of receiving the trigger signal; for the side-view camera, the target parameter in the side-view camera register indicating the time difference between the end time of the first row of pixels and the time of receiving the trigger signal is modified to D2, so that there is a delay of D2 between the end time of the first row of pixels and the time of receiving the trigger signal; for the fisheye camera, the target parameter in the fisheye camera register indicating the time difference between the end time of the first row of pixels and the time of receiving the trigger signal is modified to D3, so that there is a delay of D3 between the end time of the first row of pixels and the time of receiving the trigger signal.

[0126] It should be understood that by setting an additional delay, the data transmission delay of the front-view camera will be delayed by D1 relative to the original data transmission delay. The exposure start time of each row of pixels will also be delayed by D1, thus the exposure center time will also be delayed by D1. By setting the above additional delay, the exposure center times of the front-view camera, the side-view camera and the fisheye camera are aligned and the timestamps are consistent.

[0127] In the above technical solution, 12 external trigger signals at exactly the same time are used to trigger 12 cameras for exposure. All 12 cameras adopt the readout external trigger mode to ensure stable data transmission link latency. Additional delay settings are made for the three types of cameras to create a certain delay between the end point of the first line of exposure and the external trigger signal, so that the timestamps of the exposure center points of each type of camera are aligned. In addition, the data from the 12 cameras can also complete the transmission of the entire frame almost simultaneously, ensuring that the downstream users can obtain the complete image information of the 12 cameras almost simultaneously. This can effectively avoid the risk of missed detection or false detection caused by the difference in data transmission latency between different cameras.

[0128] Figure 6 This is a schematic diagram of the structure of a camera control device provided in an embodiment of this application.

[0129] For example, such as Figure 6 As shown, the device 600 includes:

[0130] The acquisition module 610 is used to acquire the exposure center time of each of at least two types of cameras;

[0131] The calculation module 620 determines the target time based on the latest time among the exposure center times of each type of camera, calculates the difference between the exposure center time and the target time for each type of camera, and obtains the target delay duration of the exposure center time for each type of camera; wherein, the target time is greater than or equal to the latest time.

[0132] The control module 630 is used to control the acquisition time of each type of camera to acquire target images based on the target delay duration corresponding to each type of camera, so that the timestamps of the target images acquired by each type of camera are the same.

[0133] In some embodiments, the calculation module is specifically used to determine whether the target time is an integer time; if the target time is not an integer time, the target time is rounded down to obtain an integer target time; and the difference between the exposure center time of each type of camera and the target integer time is calculated.

[0134] In some embodiments, the acquisition module is specifically used to acquire the preset exposure duration and attribute information of each type of camera; wherein, the attribute information includes the total transmission duration of one frame of image data and the transmission delay duration of one frame of image data, and the transmission delay duration is used to characterize the time difference between the time when the camera transmits image data and the time when it receives the image acquisition signal; based on the preset exposure duration and attribute information, the exposure center time of each type of camera is calculated.

[0135] In some embodiments, at least two types of cameras include a front-view camera, a side-view camera, and a fisheye camera. The acquisition module is specifically used to determine the preset exposure time of the front-view camera based on the flashing period of the traffic light; to determine the preset exposure time of the side-view camera based on the current angular velocity of the vehicle; and to determine the preset exposure time of the fisheye camera based on the brightness information of the current shooting environment.

[0136] In some embodiments, the acquisition module is specifically used to determine the preset exposure time of the forward-looking camera based on the flashing period of the traffic lights in the current shooting environment if traffic lights exist in the current shooting environment; and to determine the preset exposure time of the forward-looking camera based on the flashing period of traffic lights in historical shooting environments if traffic lights do not exist in the current shooting environment.

[0137] In some embodiments, at least two types of cameras use progressive exposure mode.

[0138] In some embodiments, the control module is specifically used to replace the target parameter in each type of camera with the target delay duration corresponding to each type of camera; wherein, the target parameter is used to characterize the time difference between the moment when the image acquisition signal is received and the moment when the exposure of the first row of pixels of the target image ends.

[0139] Figure 7 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this application.

[0140] For example, such as Figure 7 As shown, the vehicle 700 includes a memory 701 and a processor 702. The memory 701 stores executable program code 7011, and the processor 702 is used to call and execute the executable program code 7011 to perform a camera control method.

[0141] Furthermore, this application also protects an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a camera control method provided in this application.

[0142] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0143] When each functional module is divided according to its corresponding function, the device may also include an acquisition module, a calculation module, a control module, etc. It should be noted that all relevant content regarding the steps involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0144] It should be understood that the device provided in this embodiment is used to execute the above-described camera control method, and therefore can achieve the same effect as the above-described implementation method.

[0145] When using an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module can be used to control and manage the vehicle's movements. The storage module can be used to support the vehicle in executing program code, etc.

[0146] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits shown in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0147] In addition, the device provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a camera control method provided in the above embodiments.

[0148] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described related method steps to implement a camera control method provided in the above embodiment.

[0149] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement a camera control method provided in the above embodiment.

[0150] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0151] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0152] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for controlling a camera, characterized in that, The method is applied to a vehicle that includes at least two types of cameras, and the method includes: Obtain the exposure center time for each of the at least two types of cameras; The target time is determined based on the latest time among the exposure center times of each type of camera, and the difference between the exposure center time of each type of camera and the target time is calculated to obtain the target delay time of the exposure center time of each type of camera; wherein, the target time is greater than or equal to the latest time. Based on the target delay duration corresponding to each type of camera, the acquisition time of the target image by each type of camera is controlled so that the timestamp of the target image acquired by each type of camera is the same.

2. The method according to claim 1, characterized in that, The calculation of the difference between the exposure center time and the target time for each type of camera includes: Determine whether the target time is an integer. If the target time is not an integer time, the target time is rounded down to obtain an integer target time; Calculate the difference between the exposure center time and the target integer time for each type of camera.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the exposure center time for each of the at least two types of cameras includes: Obtain the preset exposure time and attribute information for each type of camera; wherein, the attribute information includes the total transmission time of a frame of image data and the transmission delay time of the frame of image data, and the transmission delay time is used to characterize the time difference between the time when the camera transmits image data and the time when it receives the image acquisition signal; Based on the preset exposure duration and the attribute information, the exposure center time of each type of camera is calculated.

4. The method according to claim 3, characterized in that, The at least two types of cameras include a front-view camera, a side-view camera, and a fisheye camera. Obtaining the preset exposure time for each type of camera includes: The preset exposure time of the forward-looking camera is determined based on the flashing period of the traffic lights; Based on the current angular velocity of the vehicle, the preset exposure duration of the side-view camera is determined; Based on the brightness information of the current shooting environment, the preset exposure time of the fisheye camera is determined.

5. The method according to claim 4, characterized in that, The determination of the preset exposure time of the forward-looking camera based on the flash period of the traffic lights includes: If the traffic light exists in the current shooting environment, the preset exposure time of the forward-looking camera is determined according to the flash period of the traffic light in the current shooting environment; If the traffic light is not present in the current shooting environment, the preset exposure time of the forward-looking camera is determined based on the flashing period of the traffic lights in the historical shooting environment.

6. The method according to claim 1, characterized in that, The exposure mode of the at least two types of cameras is progressive exposure mode.

7. The method according to claim 6, characterized in that, The step of controlling the acquisition time of the target image by each type of camera based on the target delay duration corresponding to each type of camera includes: The target parameter in each type of camera is replaced with the target delay duration corresponding to each type of camera; wherein the target parameter is used to characterize the time difference between the moment when the image acquisition signal is received and the moment when the exposure of the first row of pixels of the target image ends.

8. A control device for a camera, characterized in that, The device includes: The acquisition module is used to acquire the exposure center time of each of at least two types of cameras; The calculation module determines the target time based on the latest time among the exposure center times of each type of camera, calculates the difference between the exposure center time of each type of camera and the target time, and obtains the target delay time of the exposure center time of each type of camera; wherein, the target time is greater than or equal to the latest time; The control module is used to control the acquisition time of the target image by each type of camera based on the target delay duration corresponding to each type of camera, so that the timestamp of the target image acquired by each type of camera is the same.

9. A vehicle, characterized in that, The vehicles include: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the vehicle to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 7.

Images