Image exposure parameter acquisition method and device, equipment and medium

Abstract

The invention discloses an image exposure parameter obtaining method and device, equipment and a medium, and relates to the field of image processing, and the method comprises the steps: a processor determines a current timestamp when obtaining a frame synchronization signal sent by a deserializer; obtaining a current frame exposure image sent by an image sensor in the camera, and determining image exposure starting time corresponding to the current frame exposure image based on the current timestamp and a preset time delay; wherein the current frame exposure image is an image obtained after a serializer in the camera controls an image sensor to start exposure based on a frame synchronization signal sent by a deserializer; the deserializer sends a frame synchronization signal to the serializer and the processor at the same time; the preset time delay is a difference value between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts to expose. Under the condition of not depending on the timestamp generation function of the image sensor, the image exposure starting time is calculated through the processor, the application range of the scheme is expanded, and the accuracy and the stability are higher.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, device, equipment and medium for acquiring image exposure parameters. Background Art

[0002] In an autonomous driving system, the vehicle needs to accurately perceive the surrounding environment through the camera to make real-time decisions. Based on this, it is crucial to synchronize the image exposure start time with the data of other sensors such as lidar, radar, IMU (Inertial Measurement Unit), etc.

[0003] At present, one way to obtain the exposure start time of an image is to generate a timestamp through the image sensor at the start of exposure and embed it into the image data stream to send the image and timestamp to the application together. However, this method is heavily dependent on the support of the image sensor in the camera. If the image sensor does not support it, the timestamp cannot be obtained. Therefore, it is not suitable for all cameras. In addition, the timestamp needs to be sent in an embedded manner, which will occupy the transmission bandwidth of MIPI (Mobile Industry Processor Interface) and limit the improvement of image resolution. Accordingly, after receiving each frame of the image, the application also needs to parse the embedded data to obtain the corresponding timestamp. Another way is to send a trigger signal to the camera through external software. The camera starts to expose after receiving the signal, and the software records the timestamp when the trigger signal is sent as the exposure start time of the image. However, this method of completely controlling the image exposure through software requires additional consumption of CPU (Central Processing Unit) resources, and the timestamp recorded by the software is not accurate, and there is a certain deviation from the actual exposure start time of the image. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an image exposure parameter acquisition method, device, equipment and medium, which can calculate the image exposure start time through a processor without relying on the timestamp generation function of the image sensor, which not only expands the application scope of the solution, but also has higher accuracy and stability and lower resource consumption. The specific solution is as follows:

[0005] In a first aspect, the present application provides a method for obtaining image exposure parameters, which is applied to a processor, comprising:

[0006] When acquiring the frame synchronization signal sent by the deserializer, determining the current timestamp;

[0007] Obtaining a current frame exposure image sent by an image sensor in a camera, and determining an image exposure start time corresponding to the current frame exposure image based on the current timestamp and a preset delay;

[0008] Among them, the current frame exposure image is the image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

[0009] Optionally, when acquiring the frame synchronization signal sent by the deserializer, determining the current timestamp includes:

[0010] Acquire a frame synchronization signal sent by a deserializer through a universal input / output interface in the processor, and output a target level different from a current level based on the frame synchronization signal;

[0011] When a level change is detected, an interrupt is triggered to determine the current timestamp.

[0012] Optionally, after determining the image exposure start time corresponding to the current frame exposure image, the method further includes:

[0013] Determining whether the deserializer works normally based on the current timestamp and / or the image exposure start time corresponding to the current frame exposure image;

[0014] If the deserializer works abnormally, the deserializer is reconfigured.

[0015] Optionally, determining whether the deserializer works normally based on the current timestamp includes:

[0016] If the current timestamp does not satisfy a first preset condition corresponding to the deserializer, it indicates that the deserializer is working abnormally;

[0017] Among them, the first preset condition includes that the current timestamp is within a first time range; the first time range is a range determined based on the previous timestamp, a preset time interval and a first preset fluctuation value; the previous timestamp is the time when the frame synchronization signal sent by the deserializer was last obtained.

[0018] Optionally, determining whether the deserializer works normally based on an image exposure start time corresponding to the current frame exposure image includes:

[0019] If the image exposure start time corresponding to the current frame exposure image does not meet the second preset condition corresponding to the deserializer, it indicates that the deserializer is working abnormally;

[0020] Among them, the second preset condition includes that the image exposure start time corresponding to the current frame exposure image is within a second time range; the second time range is a range determined based on the image exposure start time corresponding to the previous frame exposure image, a preset time interval and a second preset fluctuation value.

[0021] Optionally, after acquiring the current frame exposure image sent by the image sensor in the camera, the method further includes:

[0022] Determine a current image acquisition time when acquiring the current frame exposure image;

[0023] If the current image acquisition time does not meet the third preset condition corresponding to the camera, it indicates that the camera is working abnormally, and the camera is reconfigured;

[0024] Among them, the third preset condition includes that the current image acquisition time is within a third time range and the current image acquisition time is later than the image exposure start time corresponding to the current frame exposure image; the third time range is a range determined based on the previous image acquisition time, a preset time interval and a third preset fluctuation value; the previous image acquisition time is the time when the previous frame exposure image was acquired.

[0025] Optionally, the preset time interval is a value determined based on a frame rate of the image sensor;

[0026] Correspondingly, the deserializer sends the frame synchronization signal to the serializer and the processor simultaneously at a preset frequency; the preset frequency is a value determined based on the frame rate of the image sensor.

[0027] In a second aspect, the present application provides an image exposure parameter acquisition device, which is applied to a processor, comprising:

[0028] A timestamp determination module, used to determine a current timestamp when acquiring a frame synchronization signal sent by a deserializer;

[0029] An exposure time determination module, used to obtain a current frame exposure image sent by an image sensor in a camera, and determine an image exposure start time corresponding to the current frame exposure image based on the current timestamp and a preset delay;

[0030] Among them, the current frame exposure image is the image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

[0031] In a third aspect, the present application provides an electronic device, including a deserializer, a camera including a serializer connected to the deserializer, and further including:

[0032] Memory, used to store computer programs;

[0033] The processor connected to the deserializer is used to execute the computer program to implement the aforementioned image exposure parameter acquisition method.

[0034] Optionally, a plurality of cameras are provided, and the plurality of cameras are connected to the same deserializer.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which implements the aforementioned image exposure parameter acquisition method when executed by a processor.

[0036] In the present application, when the processor obtains the frame synchronization signal sent by the deserializer, it determines the current timestamp; obtains the current frame exposure image sent by the image sensor in the camera, and determines the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay; wherein the current frame exposure image is the image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

[0037] It can be seen that the present application sends a frame synchronization signal to the serializer and the processor at the same time through the deserializer. Since the serializer and the processor obtain the frame synchronization signal almost instantly, the current timestamp when the processor obtains the frame synchronization signal is basically the same as the time when the serializer obtains the frame synchronization signal, and the serializer will further control the image sensor to start exposure based on the frame synchronization signal to obtain the current frame exposure image. Considering that there is a delay from the serializer obtaining the frame synchronization signal to the image sensor starting exposure, and the delay is basically the same for the same camera, the processor can determine the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay. In this way, the system architecture of the present application based on the deserializer, the processor, and the camera including the image sensor and the serializer can be independent of the timestamp generation function of the image sensor, and the application scope of the present solution can be expanded by reducing the dependence on the image sensor, and the problem of high resource consumption and low image resolution caused by embedding the timestamp into the image data stream and sending it to the processor can be avoided; and the present application calculates the image exposure start time through the hardware method of the processor, and the accuracy and stability of the present solution will be higher than the method of controlling the image exposure by external software. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flow chart of a method for obtaining image exposure parameters disclosed in this application;

[0040] Figure 2 A system architecture diagram disclosed in this application;

[0041] Figure 3 This is an image exposure timing diagram disclosed in this application;

[0042] Figure 4 A camera structure diagram disclosed in this application;

[0043] Figure 5 This is a schematic diagram of the structure of an image exposure parameter acquisition device disclosed in this application;

[0044] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In an autonomous driving system, the vehicle needs to accurately perceive the surrounding environment through the camera to make real-time decisions. Based on this, it is very important to synchronize the image exposure start time with the data of other sensors (such as lidar, radar, IMU, etc.). To this end, this application provides a method for obtaining image exposure parameters. Through a system architecture based on a deserializer, a processor, and a camera including an image sensor and a serializer, the image exposure start time can be calculated by the processor without relying on the timestamp generation function of the image sensor, which not only expands the application scope of this solution, but also has higher accuracy and stability and lower resource consumption.

[0047] See also Figure 1 As shown, an embodiment of the present invention discloses a method for obtaining image exposure parameters, which is applied to a processor and includes:

[0048] Step S11, when acquiring the frame synchronization signal sent by the deserializer, determining the current timestamp;

[0049] Step S12, obtaining a current frame exposure image sent by the image sensor in the camera, and determining the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay; wherein the current frame exposure image is an image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

[0050] In the embodiment of the present invention, before the application processor calculates the exposure start time corresponding to the exposure image in real time, it is necessary to first configure the image sensor in the camera, specifically including mode configuration and signal trigger configuration. Among them, the processor in the present invention belongs to a system on chip (SOC), for example, a central processing unit CPU can be selected.

[0051] For mode configuration, image sensor exposure output generally has active mode (active exposure output, which can be automatically framed according to configuration) and passive mode (waiting for trigger exposure output, which can only frame after being triggered). The image sensor uses the active mode by default for general operation, and uses the passive mode when it needs to output data synchronously with other sensors.

[0052] For signal trigger configuration, in general, image sensors include the following three trigger modes: the first trigger mode is that the image sensor outputs a synchronization signal to other sensors, the second trigger mode is that the image sensor starts to read the image after receiving the synchronization signal, and the third trigger mode is that the image sensor opens the shutter operation to start image exposure after receiving the synchronization signal. The third trigger mode is used when it is necessary to output data synchronously with other sensors.

[0053] That is, before the application processor calculates the exposure start time corresponding to the exposure image in real time, the present invention first configures the image sensor in the camera to the passive mode and the third trigger mode, so that the image sensor starts the shutter operation to start image exposure after receiving the synchronization signal, thereby obtaining the corresponding exposure image.

[0054] It should be noted that the number of cameras in this application can be set according to actual needs, for example, it can be set to one or more; whether it is one camera or multiple cameras, they are all connected to the same deserializer. When the number of cameras is multiple, the specifications and parameters of the cameras must be consistent.

[0055] Taking the number of cameras as four as an example, the system architecture of the present application regarding the deserializer, the processor, and the camera including the image sensor and the serializer is as follows: Figure 2 As shown, the deserializer is connected to the GPIO (General-purpose input / output) pin of the processor through the MFP pin (Multi-function Pin) so that the deserializer can send a frame synchronization (FrameSynchronization) signal to the processor. For the MFP pin, it is generally recommended to use MFP2 or MFP10. Furthermore, the four cameras camera0~3 are connected to the deserializer through GMSL A~D (Gigabit Multimedia Serial Link), and each camera includes an image sensor and a serializer. It should be noted that no matter how many cameras there are, they can be connected to the deserializer and the processor in a connection method similar to the above.

[0056] After connecting the deserializer, processor, and camera including image sensor and serializer, you need to further configure the frame synchronization function of the deserializer, including the frequency of the frame synchronization signal and the frame synchronization flag. The frequency of the frame synchronization signal can be determined according to the frame rate of the image sensor. For example, if the frame rate of the image sensor is 25FPS (Frame Per Second), the frequency of the frame synchronization signal is 25HZ. At the same time, you also need to configure the frame synchronization function of the serializer, including the frame synchronization flag. It should be noted that the frame synchronization flag configured by the serializer needs to be consistent with the frame synchronization flag configured by the deserializer, so as to receive the frame synchronization signal sent by the deserializer.

[0057] After completing the above configuration, the deserializer sends a frame synchronization signal to the processor and the serializer in the camera at the same time according to the preset frequency, where the preset frequency is the frequency of the frame synchronization signal. Accordingly, when the processor obtains the frame synchronization signal sent by the deserializer, it determines the current timestamp. When the serializer obtains the frame synchronization signal sent by the deserializer, it controls the image sensor in the camera to start exposure based on the frame synchronization signal, and then the image sensor sends the current frame exposure image obtained after exposure to the processor. Furthermore, the processor obtains the current frame exposure image sent by the image sensor, and determines the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay, and then sends the current frame exposure image and the corresponding image exposure start time to the upper-layer application for processing.

[0058] It should be noted that, since the serializer and deserializer are interface circuits in high-speed data communication, they play an important role in short-distance chip interconnection, can effectively reduce the number of pins and tracks, and effectively improve the data communication rate; based on this, the present application uses short-distance communication between the serializer and the deserializer and short-distance communication between the deserializer and the processor, so that the current timestamp when the processor obtains the frame synchronization signal is basically the same as the time when the serializer obtains the frame synchronization signal. In addition, for cameras of the same specifications, the delay from the serializer obtaining the frame synchronization signal to the image sensor starting exposure is basically fixed for each exposure, so the preset delay determined based on the specification parameters of the camera can be loaded into the processor, so that the processor can determine the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay when the frame synchronization signal is obtained. In this way, the present application does not need to rely on the timestamp generation function of the image sensor, and expands the application scope of the present solution by reducing the dependence on the image sensor, and can avoid the problems of high resource consumption and low image resolution caused by embedding the timestamp into the image data stream and sending it to the processor.

[0059] Specifically, after the deserializer sends a frame synchronization signal to the processor and the serializer in the camera at the same time according to a preset frequency, the processor obtains the frame synchronization signal sent by the deserializer through the local general input and output interface, and outputs a target level different from the current level based on the frame synchronization signal. Further, when the processor detects a level change, it triggers an interrupt to determine the current timestamp. At the same time, when the serializer obtains the frame synchronization signal sent by the deserializer, it sends the frame synchronization signal to the image sensor, so that the image sensor starts the shutter operation to start image exposure when receiving the frame synchronization signal, and sends the current frame exposure image to the processor via the serializer and the deserializer.

[0060] Specifically, for image sensors, such as Figure 3 As shown, after receiving the frame synchronization signal sent by the external serializer, the image sensor processes the frame synchronization signal to generate a shutter trigger signal after an internal processing delay, and starts the shutter operation based on the shutter trigger signal to start image exposure; after the image exposure is completed, the current frame exposure image is obtained, and then the reading of the current frame exposure image is triggered to complete the reading of the current frame exposure image after an internal reading delay, and the read current frame exposure image is used as valid data, and a frame start mark is added to the head of the valid data, and a frame end mark is added to the tail of the valid data, so as to obtain a packaged frame of data, and the packaged frame of data is sent to the processor via the serializer and the deserializer.

[0061] Considering that the deserializer sends frame synchronization signals to the processor and the serializer in the camera at the same time at a preset frequency, and that the time required for the same processing operation is generally fixed, the time interval between two adjacent timestamps determined by the processor should be the preset time interval, and the time interval between the image exposure start times corresponding to two adjacent frames of exposure images should also be the preset time interval. The preset time interval is a value determined based on the frame rate of the image sensor; for example, if the frame rate of the image sensor is 25FPS, that is, 25 images per second, the preset time interval should be 40ms.

[0062] However, there may be slight fluctuations in the clock source of the frame synchronization signal sent by the deserializer, and there may also be small fluctuations in the interrupt processing of the processor, generally ranging from microseconds to nanoseconds. Therefore, a fluctuation value can be set to determine whether the time interval between two adjacent timestamps and / or the image exposure start times corresponding to two adjacent frames of exposure images is within the allowable fluctuation range, thereby determining whether the deserializer is working properly.

[0063] Specifically, the processor determines whether the deserializer is working properly based on the current timestamp and / or the image exposure start time corresponding to the current frame exposure image; if the deserializer is working abnormally, the deserializer is reconfigured; if the deserializer is working normally, the current frame exposure image and the corresponding image exposure start time are sent to the upper-layer application for processing.

[0064] In the process of determining whether the deserializer is working properly based on the current timestamp, it is determined whether the current timestamp satisfies the first preset condition corresponding to the deserializer; if the current timestamp does not satisfy the first preset condition corresponding to the deserializer, it indicates that the deserializer is working abnormally. Among them, the first preset condition includes that the current timestamp is within a first time range; the first time range is a range determined based on the previous timestamp, the preset time interval and the first preset fluctuation value; the previous timestamp is the time when the processor last obtained the frame synchronization signal sent by the deserializer.

[0065] Exemplarily, the lower limit value of the first time range = the previous timestamp + the preset time interval - the first preset fluctuation value, the upper limit value of the first time range = the previous timestamp + the preset time interval + the first preset fluctuation value. If the lower limit value of the first time range ≤ the current timestamp ≤ the upper limit value of the first time range, it indicates that the deserializer is working normally, otherwise the deserializer is working abnormally.

[0066] In the process of determining whether the deserializer is working normally based on the image exposure start time corresponding to the current frame exposure image, it is determined whether the image exposure start time corresponding to the current frame exposure image meets the second preset condition corresponding to the deserializer. If the image exposure start time corresponding to the current frame exposure image does not meet the second preset condition corresponding to the deserializer, it indicates that the deserializer is working abnormally. The second preset condition includes that the image exposure start time corresponding to the current frame exposure image is within a second time range; the second time range is a range determined based on the image exposure start time corresponding to the previous frame exposure image, the preset time interval and the second preset fluctuation value.

[0067] Exemplarily, the lower limit value of the second time range = the image exposure start time corresponding to the previous frame of exposure image + the preset time interval - the second preset fluctuation value, the upper limit value of the second time range = the image exposure start time corresponding to the previous frame of exposure image + the preset time interval + the second preset fluctuation value, if the lower limit value of the second time range ≤ the image exposure start time corresponding to the current frame of exposure image ≤ the upper limit value of the second time range, then it indicates that the deserializer is working normally, otherwise the deserializer is working abnormally.

[0068] Considering that the exposure image is sent from the image sensor to the processor for reception, it needs to be transmitted through the serializer and deserializer, received and processed by the processor hardware, etc., ideally, the time interval between two adjacent frames of exposure image arriving at the processor should be a fixed value. However, in actual situations, the time interval between two adjacent frames of exposure image arriving at the processor may fluctuate, but the fluctuation is generally in microseconds. Therefore, a fluctuation value can be set to determine whether the time interval between two adjacent frames of exposure image arriving at the processor is within the allowable fluctuation range, thereby determining whether the camera is working properly.

[0069] Specifically, after acquiring the current frame exposure image sent by the image sensor in the camera, the processor determines the current image acquisition time when acquiring the current frame exposure image, and judges whether the current image acquisition time satisfies the third preset condition corresponding to the camera; if the current image acquisition time does not satisfy the third preset condition corresponding to the camera, it indicates that the camera is working abnormally, and the camera is reconfigured; if the camera is working normally, the current frame exposure image and the corresponding image exposure start time are sent to the upper-level application for processing.

[0070] Among them, the third preset condition includes that the current image acquisition time is within the third time range and the current image acquisition time is later than the image exposure start time corresponding to the current frame exposure image; the third time range is a range determined based on the previous image acquisition time, the preset time interval and the third preset fluctuation value; the previous image acquisition time is the time when the processor acquires the previous frame exposure image.

[0071] Exemplarily, the lower limit value of the third time range = the last image acquisition time + the preset time interval - the third preset fluctuation value, and the upper limit value of the third time range = the last image acquisition time + the preset time interval + the third preset fluctuation value. If the lower limit value of the third time range ≤ the current image acquisition time ≤ the upper limit value of the third time range, and the current image acquisition time ≥ the image exposure start time corresponding to the current frame exposure image, then the camera is working normally, otherwise the camera is working abnormally.

[0072] It should be noted that the first preset fluctuation value, the second preset fluctuation value and the third preset fluctuation value can be set according to the actual fluctuation situation, and the first preset fluctuation value, the second preset fluctuation value and the third preset fluctuation value may be the same value or different values, which is not limited here.

[0073] When the deserializer is working normally, the present application can also record the time when the frame synchronization signal sent by the serializer is obtained this time through the image sensor, and determine the target time interval between the time when the frame synchronization signal sent by the serializer is obtained this time and the time when the frame synchronization signal sent by the serializer was obtained last time; if the difference between the target time interval and the preset time interval is greater than the fourth preset fluctuation value, it is determined that the serializer is working abnormally, and the image sensor sends a reset command to the serializer to reconfigure the serializer. In this way, the present application detects whether the serializer is working abnormally to avoid image exposure when the serializer is working abnormally, and avoids using the wrong exposed image, so as to improve the safety and reliability of subsequent work performed based on the exposed image.

[0074] Furthermore, since the specifications and parameters of the cameras are consistent, and the image exposure start time corresponding to the exposure image is the time determined by the processor based on the current timestamp when the frame synchronization signal sent by the deserializer is obtained and the preset delay, therefore, regardless of whether the number of cameras is one or more, that is, regardless of whether the current frame exposure image received by the processor this time is one or more, the processor only needs to calculate the image exposure start time once, that is, the processor does not need to calculate the image exposure start time once for each exposure image obtained this time, so that the present application can save the processor's computing resources to a certain extent.

[0075] It can be seen that the present application sends a frame synchronization signal to the serializer and the processor at the same time through the deserializer. Since the serializer and the processor obtain the frame synchronization signal almost instantly, the current timestamp when the processor obtains the frame synchronization signal is basically the same as the time when the serializer obtains the frame synchronization signal, and the serializer will further control the image sensor to start exposure based on the frame synchronization signal to obtain the current frame exposure image. Considering that there is a delay from the serializer obtaining the frame synchronization signal to the image sensor starting exposure, and the delay is basically the same for the same camera, the processor can determine the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay. In this way, the system architecture of the present application based on the deserializer, the processor, and the camera including the image sensor and the serializer can be independent of the timestamp generation function of the image sensor, and the application scope of the present solution can be expanded by reducing the dependence on the image sensor, and the problem of high resource consumption and low image resolution caused by embedding the timestamp into the image data stream and sending it to the processor can be avoided; and the present application calculates the image exposure start time through the hardware method of the processor, and the accuracy and stability of the present solution will be higher than the method of controlling the image exposure by external software.

[0076] by Figure 4 Taking the camera structure shown in the figure as an example, the camera proposed in this application is described in detail, including:

[0077] The camera is based on a lens group (LENS), a FLASH memory, an oscillator (OSC), a power management chip (Power Management IC, PMIC), an RF connector, an image sensor, and a serializer. Among them, the lens group is connected to the image sensor through a connector; the image sensor is connected to the FLASH memory through SPI (Serial Peripheral Interface), and the FLASH memory is used to store the configuration data and perception data of the image sensor; the image sensor is connected to the oscillator through the EXTCLK (external clock) pin; the image sensor is connected to the serializer through DVP (Digital Video Port) to transmit the exposure image to the serializer; the image sensor realizes bidirectional data transmission with the serializer through I2C (Inter-Integrated Circuit, a two-wire serial bus). In addition, the serializer is connected to another oscillator and to the RF connector; in the automotive field, the RF connector can use FAKRA (a connector used inside the car). The power management chip is used to power the image sensor, serializer, and RF connector.

[0078] Taking the above camera structure as an example, the serializer in the camera obtains the frame synchronization signal sent by the deserializer through the RF connector, and sends the frame synchronization signal to the image sensor. After receiving the frame synchronization signal, the image sensor triggers the shutter operation to start image exposure, and transmits the current frame exposure image to the serializer through the DVP interface. The serializer further sends the current frame exposure image to the deserializer through the RF connector, so that the deserializer sends the current frame exposure image to the processor.

[0079] Since the serializer and deserializer are interface circuits in high-speed data communication, they play an important role in short-distance chip interconnection, can effectively reduce the number of pins and tracks, and effectively improve the data communication rate; based on this, the present application can significantly improve the data transmission rate and the real-time performance of image exposure through short-distance communication between the serializer and the deserializer and short-distance communication between the deserializer and the processor, and realize the acquisition of the start time of image exposure through this hardware method, and the accuracy and stability will be higher than the software method.

[0080] See also Figure 5 As shown, an embodiment of the present invention discloses an image exposure parameter acquisition device, which is applied to a processor and includes:

[0081] A timestamp determination module 11, used to determine a current timestamp when acquiring a frame synchronization signal sent by a deserializer;

[0082] An exposure time determination module 12 is used to obtain a current frame exposure image sent by an image sensor in a camera, and determine an image exposure start time corresponding to the current frame exposure image based on the current timestamp and a preset delay;

[0083] Among them, the current frame exposure image is the image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

[0084] It can be seen that the present application sends a frame synchronization signal to the serializer and the processor at the same time through the deserializer. Since the serializer and the processor obtain the frame synchronization signal almost instantly, the current timestamp when the processor obtains the frame synchronization signal is basically the same as the time when the serializer obtains the frame synchronization signal, and the serializer will further control the image sensor to start exposure based on the frame synchronization signal to obtain the current frame exposure image. Considering that there is a delay from the serializer obtaining the frame synchronization signal to the image sensor starting exposure, and the delay is basically the same for the same camera, the processor can determine the image exposure start time corresponding to the current frame exposure image based on the current timestamp and the preset delay. In this way, the system architecture of the present application based on the deserializer, the processor, and the camera including the image sensor and the serializer can be independent of the timestamp generation function of the image sensor, and the application scope of the present solution can be expanded by reducing the dependence on the image sensor, and the problem of high resource consumption and low image resolution caused by embedding the timestamp into the image data stream and sending it to the processor can be avoided; and the present application calculates the image exposure start time through the hardware method of the processor, and the accuracy and stability of the present solution will be higher than the method of controlling the image exposure by external software.

[0085] In some specific embodiments, the timestamp determination module 11 includes:

[0086] A level output unit, configured to obtain a frame synchronization signal sent by a deserializer through a universal input / output interface in the processor, and output a target level different from a current level based on the frame synchronization signal;

[0087] The timestamp determination unit is used to trigger an interrupt to determine the current timestamp when a level change is detected.

[0088] In some specific embodiments, the image exposure parameter acquisition device further includes:

[0089] A normal operation determination module, configured to determine whether the deserializer is operating normally based on the current timestamp and / or the image exposure start time corresponding to the current frame exposure image;

[0090] The deserializer configuration unit is used to reconfigure the deserializer if the deserializer works abnormally.

[0091] In some specific embodiments, the normal operation determination module includes:

[0092] A first abnormality determination unit, configured to indicate that the deserializer is operating abnormally if the current timestamp does not satisfy a first preset condition corresponding to the deserializer;

[0093] Among them, the first preset condition includes that the current timestamp is within a first time range; the first time range is a range determined based on the previous timestamp, a preset time interval and a first preset fluctuation value; the previous timestamp is the time when the frame synchronization signal sent by the deserializer was last obtained.

[0094] In some specific embodiments, the normal operation determination module includes:

[0095] A second abnormality determination unit, configured to indicate that the deserializer is operating abnormally if the image exposure start time corresponding to the current frame exposure image does not satisfy a second preset condition corresponding to the deserializer;

[0096] Among them, the second preset condition includes that the image exposure start time corresponding to the current frame exposure image is within a second time range; the second time range is a range determined based on the image exposure start time corresponding to the previous frame exposure image, a preset time interval and a second preset fluctuation value.

[0097] In some specific embodiments, the image exposure parameter acquisition device further includes:

[0098] An acquisition time determining unit, used to determine a current image acquisition time when acquiring the current frame exposure image;

[0099] A camera configuration unit, configured to indicate that the camera is working abnormally and reconfigure the camera if the current image acquisition time does not satisfy a third preset condition corresponding to the camera;

[0100] Among them, the third preset condition includes that the current image acquisition time is within a third time range and the current image acquisition time is later than the image exposure start time corresponding to the current frame exposure image; the third time range is a range determined based on the previous image acquisition time, a preset time interval and a third preset fluctuation value; the previous image acquisition time is the time when the previous frame exposure image was acquired.

[0101] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0102] Figure 6 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, a communication bus 26, a deserializer 27, and a camera 28. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image exposure parameter acquisition method disclosed in any of the above embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0103] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of this application, and it is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and it is not specifically limited here. The deserializer 27 is used to send a frame synchronization signal to the processor 21 and the serializer 281 in the camera 28 at the same time according to a preset frequency; the serializer 281 is used to control the image sensor in the camera to start image exposure based on the frame synchronization signal.

[0104] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0105] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and may be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the image exposure parameter acquisition method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 may further include a computer program that can be used to complete other specific tasks.

[0106] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for obtaining image exposure parameters is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0107] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0108] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0110] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0111] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for obtaining image exposure parameters, characterized in that: Applicable to processors, including: When acquiring the frame synchronization signal sent by the deserializer, determining the current timestamp; Receive a current frame exposure image sent by an image sensor in a camera, and determine an image exposure start time corresponding to the current frame exposure image based on the current timestamp and a preset delay; Among them, the current frame exposure image is the image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

2. The method for acquiring image exposure parameters according to claim 1, characterized in that: The step of determining the current timestamp when acquiring the frame synchronization signal sent by the deserializer includes: Acquire a frame synchronization signal sent by a deserializer through a universal input / output interface in the processor, and output a target level different from a current level based on the frame synchronization signal; When a level change is detected, an interrupt is triggered to determine the current timestamp.

3. The method for acquiring image exposure parameters according to claim 1, characterized in that: After determining the image exposure start time corresponding to the current frame exposure image, the method further includes: Determining whether the deserializer works normally based on the current timestamp and / or the image exposure start time corresponding to the current frame exposure image; If the deserializer works abnormally, the deserializer is reconfigured.

4. The method for acquiring image exposure parameters according to claim 3, characterized in that: Determining whether the deserializer works normally based on the current timestamp includes: If the current timestamp does not satisfy a first preset condition corresponding to the deserializer, it indicates that the deserializer is working abnormally; Among them, the first preset condition includes that the current timestamp is within a first time range; the first time range is a range determined based on the previous timestamp, a preset time interval and a first preset fluctuation value; the previous timestamp is the time when the frame synchronization signal sent by the deserializer was last obtained.

5. The method for acquiring image exposure parameters according to claim 3, characterized in that: Determining whether the deserializer works normally based on the image exposure start time corresponding to the current frame exposure image includes: If the image exposure start time corresponding to the current frame exposure image does not meet the second preset condition corresponding to the deserializer, it indicates that the deserializer is working abnormally; Among them, the second preset condition includes that the image exposure start time corresponding to the current frame exposure image is within a second time range; the second time range is a range determined based on the image exposure start time corresponding to the previous frame exposure image, a preset time interval and a second preset fluctuation value.

6. The method for acquiring image exposure parameters according to claim 1, characterized in that: After obtaining the current frame exposure image sent by the image sensor in the camera, the method further includes: Determine a current image acquisition time when acquiring the current frame exposure image; If the current image acquisition time does not meet the third preset condition corresponding to the camera, it indicates that the camera is working abnormally, and the camera is reconfigured; Among them, the third preset condition includes that the current image acquisition time is within a third time range and the current image acquisition time is later than the image exposure start time corresponding to the current frame exposure image; the third time range is a range determined based on the previous image acquisition time, a preset time interval and a third preset fluctuation value; the previous image acquisition time is the time when the previous frame exposure image was acquired.

7. The method for acquiring image exposure parameters according to any one of claims 4 to 6, characterized in that: The preset time interval is a value determined based on the frame rate of the image sensor; Correspondingly, the deserializer sends the frame synchronization signal to the serializer and the processor simultaneously at a preset frequency; the preset frequency is a value determined based on the frame rate of the image sensor.

8. An image exposure parameter acquisition device, characterized in that: Applicable to processors, including: A timestamp determination module, used to determine a current timestamp when acquiring a frame synchronization signal sent by a deserializer; An exposure time determination module, used to obtain a current frame exposure image sent by an image sensor in a camera, and determine an image exposure start time corresponding to the current frame exposure image based on the current timestamp and a preset delay; Among them, the current frame exposure image is the image obtained after the serializer in the camera controls the image sensor to start exposure based on the frame synchronization signal sent by the deserializer; the deserializer sends the frame synchronization signal to the serializer and the processor respectively at the same time; the preset delay is the difference between the time when the serializer obtains the frame synchronization signal and the time when the image sensor starts exposure.

9. An electronic device, characterized in that: The invention comprises a deserializer, a camera including a serializer connected to the deserializer, and further comprises: Memory, used to store computer programs; A processor connected to the deserializer, configured to execute the computer program to implement the method for acquiring image exposure parameters as described in any one of claims 1 to 7.

10. The electronic device according to claim 9, characterized in that: A plurality of cameras are provided, and the plurality of cameras are connected to the same deserializer.

11. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the image exposure parameter acquisition method according to any one of claims 1 to 7.

Images