Camera time sequence control method and device based on FPGA

Abstract

The invention provides a camera time sequence control method and device based on an FPGA (Field Programmable Gate Array), relates to the field of electronic information, and solves the technical problems of low image acquisition time sequence control precision, large processing delay and insufficient system expansibility in the prior art. The method comprises the following steps: receiving an optical signal and converting the optical signal into an electric signal; configuring a register of the image sensor, wherein a working mode, exposure time and reading parameters are set; based on the configuration and the electric signal, parameterized time sequence control signals are generated in the FPGA module, and the time sequence control signals comprise a row clock, a field clock, a reset signal and a data effective signal; controlling the output of a time sequence control signal through a state machine and a counter, and collecting image data; analyzing and processing the image data to obtain processed image data, and writing the processed image data into a cache unit in the FPGA module; and transmitting the image data in the cache to an upper computer, and displaying a real-time image. The method and the device are used in the camera time sequence control process based on the FPGA.

Description

Technical Field

[0001] This application relates to the field of electronic information, and in particular to a camera timing control method and apparatus based on FPGA. Background Technology

[0002] With the widespread application of high-speed image acquisition in industrial inspection, scientific experiments, and intelligent vision systems, higher demands are being placed on the timing control, data acquisition, and real-time processing of image sensors. Existing image acquisition systems based on central processing units (CPUs) or digital signal processors (DSPs) have limitations in high-speed acquisition, precise timing control, and real-time processing. These limitations manifest as limited acquisition speed, low timing accuracy, large image processing latency, and insufficient system scalability, making it difficult to meet the real-time processing requirements of high frame rates and high resolution images. Especially in guide camera applications, achieving high signal-to-noise ratio imaging under weak starlight conditions further imposes stringent requirements on detector sensitivity, noise performance, and data processing efficiency. Therefore, there is an urgent need for a technical solution capable of achieving precise camera timing control and efficient image data processing under high frame rates and high resolution conditions to meet these application requirements. Summary of the Invention

[0003] This application provides a camera timing control method and apparatus based on FPGA, which solves the problems of low image acquisition timing control accuracy, large processing delay and insufficient system scalability in the prior art.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, an FPGA-based camera timing control method is provided, comprising: receiving an optical signal and converting it into an electrical signal; configuring the registers of an image sensor, including setting the operating mode, exposure time, and readout parameters; generating parameterized timing control signals in the FPGA module based on the configuration and the electrical signals, the timing control signals including a line clock, a field clock, a reset signal, and a data valid signal; controlling the output of the timing control signals through a state machine and a counter to acquire image data; parsing and processing the image data to obtain processed image data, and writing the processed image data into a buffer unit within the FPGA module; and transmitting the image data in the buffer to a host computer to display the real-time image.

[0006] In conjunction with the first aspect above, in one possible implementation, receiving an optical signal and converting it into an electrical signal includes: receiving the optical signal; converting the optical signal into electrical charge in an image sensor pixel via the photoelectric effect; amplifying the electrical charge via an amplifier; and converting the amplified electrical charge into a digital electrical signal via an analog-to-digital converter.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, image data is acquired by controlling the output of timing control signals through a state machine and a counter. This includes: setting up a state machine in the FPGA module and defining an idle state, a running state, and an end state; in the idle state, detecting a module enable signal, and when the enable signal is valid, causing the state machine to transition from the idle state to the running state; in the running state, starting a counter to periodically count the timing control signals, and controlling the output of the line clock, field clock, reset signal, and data valid signal according to the counter value; determining whether the acquisition of one frame of image has been completed based on the counter value and a preset threshold, and when the condition is met, the state machine transitions to the end state; in the end state, based on the completion signal and the counter value, resetting the state machine to the idle state to prepare for the acquisition of the next frame of image; and driving the image sensor to output pixel data through timing control signals, acquiring image data and writing it into the buffer unit within the FPGA module.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, the image data is parsed and processed to obtain processed image data, including: reading the acquired image data from the cache unit within the FPGA module; performing pixel alignment and channel stitching on the image data to obtain first image data; performing region of interest extraction and pixel merging operations on the first image data according to preset processing parameters to obtain processed image data; formatting and packaging the processed image data, and writing it into the cache unit within the FPGA module.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: constraining the clock, logic grouping, and output delay using a synchronization constraint file in the FPGA module.

[0010] Secondly, an FPGA-based camera timing control device is provided, comprising: an image sensor module and an FPGA module;

[0011] The system comprises: an image sensor module for receiving light signals and converting them into electrical signals; and an FPGA module connected to the image sensor for: configuring the image sensor's registers, including setting the operating mode, exposure time, and readout parameters; generating parameterized timing control signals based on the configuration and electrical signals, including a line clock, a field clock, a reset signal, and a data valid signal; controlling the output of the timing control signals through a state machine and a counter to acquire image data; parsing and processing the image data to obtain processed image data, and writing the processed image data into a buffer unit within the FPGA module; and transmitting the image data from the buffer to a host computer for real-time image display.

[0012] In conjunction with the second aspect mentioned above, in one possible implementation, the FPGA module includes: a register configuration unit for configuring the image sensor's operating mode, exposure time, and readout parameters; a timing control unit for generating parameterized timing control signals based on the configuration, including a line clock, a field clock, a reset signal, and a data valid signal; a state machine and counter control unit for controlling the output of the timing control signals, acquiring image data, and switching states based on the counter value and the state machine state; an image processing unit for parsing, pixel alignment, channel stitching, region of interest extraction, and pixel merging of the acquired image data, and writing the processed image data into a buffer unit within the FPGA module; and a data transmission unit for transmitting the processed image data from the buffer unit within the FPGA module to a host computer for real-time display.

[0013] In conjunction with the second aspect mentioned above, in one possible implementation, the FPGA module also includes a synchronization constraint unit, which is used to constrain the clock, logic grouping, and output delay to ensure the timing stability and signal transmission accuracy of each functional module.

[0014] In conjunction with the second aspect above, in one possible implementation, the FPGA module further includes: an external interface unit for data communication with the image sensor, host computer, and other peripheral modules; a trigger control unit for receiving external trigger signals and controlling the state machine and counter to start or stop image acquisition; and a power management unit for controlling the power-on sequence, power supply voltage, and power stability of the FPGA module and the image sensor.

[0015] Thirdly, this application provides an electronic device, including: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the methods described in the first aspect and any possible implementation thereof. This electronic device may be an electronic device or a chip within an electronic device.

[0016] Fourthly, this application provides a camera timing control system based on an FPGA, including an image sensor module and an FPGA module. The image sensor module receives light signals and converts them into electrical signals. The FPGA module, connected to the image sensor, is used to: configure the image sensor's registers, including setting the operating mode, exposure time, and readout parameters; generate parameterized timing control signals based on the configuration and electrical signals, including a line clock, a field clock, a reset signal, and a data valid signal; control the output of the timing control signals through a state machine and a counter; acquire image data; parse and process the image data to obtain processed image data, and write the processed image data into a buffer unit within the FPGA module; transmit the image data in the buffer to a host computer for real-time image display.

[0017] Fifthly, this application provides a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0018] In a sixth aspect, this application provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0019] This application provides a camera timing control method and apparatus based on FPGA. By generating parameterized timing control signals within the FPGA and combining them with a state machine and counters, precise control of image acquisition is achieved, enabling high-precision management of the line clock, field clock, reset signal, and valid data signals, thereby improving image acquisition speed and frame rate. Simultaneously, image data parsing, pixel alignment, channel stitching, and region of interest processing are performed internally within the FPGA, achieving low-latency real-time image processing and display. Constraints on clock, logic grouping, and output delays are implemented using a synchronization constraint file, ensuring timing stability and signal transmission accuracy. Combined with external trigger control and a high-speed cache interface, high-speed data transmission is achieved, improving the system's scalability and applicability, and solving the problems of low image acquisition timing control accuracy, large processing latency, and insufficient system scalability in existing technologies.

[0020] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0021] Figure 1 A system architecture diagram of a camera timing control system based on FPGA provided for embodiments of this application;

[0022] Figure 2 A flowchart illustrating a camera timing control method based on FPGA provided in an embodiment of this application;

[0023] Figure 3 A flowchart illustrating another FPGA-based camera timing control method provided in this application embodiment;

[0024] Figure 4 A schematic diagram of sensor control timing signals for a camera timing control method based on FPGA provided in this application embodiment;

[0025] Figure 5 A schematic diagram of another FPGA-based camera timing control process provided for an embodiment of this application;

[0026] Figure 6 A schematic diagram of another FPGA-based camera timing control simulation waveform provided for an embodiment of this application;

[0027] Figure 7 A schematic diagram of the structure of a camera timing control device based on FPGA provided in an embodiment of this application;

[0028] Figure 8 A physical diagram of a camera timing control device based on FPGA provided for an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0031] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0032] The FPGA-based camera timing control method provided in this application embodiment can be applied to, for example... Figure 1In the FPGA-based camera timing control system 100 shown, as follows... Figure 1 As shown, the system includes an image sensor module and an FPGA module. The image sensor module receives light signals and converts them into electrical signals. The FPGA module, connected to the image sensor, is used for: configuring the image sensor's registers, including setting the operating mode, exposure time, and readout parameters; generating parameterized timing control signals based on the configuration and electrical signals, including a line clock, a field clock, a reset signal, and a data valid signal; controlling the output of the timing control signals through a state machine and a counter to acquire image data; parsing and processing the image data to obtain processed image data, and writing the processed image data into a buffer unit within the FPGA module; and transmitting the buffered image data to a host computer for real-time image display.

[0033] To address the technical problems of low image acquisition timing control accuracy, large processing delay, and insufficient system scalability in existing technologies, this application provides an FPGA-based camera timing control method. The method includes: receiving an optical signal and converting it into an electrical signal; configuring the image sensor's registers, including setting the operating mode, exposure time, and readout parameters; generating parameterized timing control signals in the FPGA module based on the configuration and electrical signals, the timing control signals including a line clock, a field clock, a reset signal, and a data valid signal; controlling the output of the timing control signals through a state machine and a counter to acquire image data; parsing and processing the image data to obtain processed image data, and writing the processed image data into a buffer unit within the FPGA module; and transmitting the image data from the buffer to a host computer for real-time image display.

[0034] Figure 2 A flowchart illustrating an FPGA-based camera timing control method provided in this application embodiment is shown below. Figure 2 As shown, the method includes:

[0035] S201 receives optical signals and converts them into electrical signals.

[0036] Among them, the light signal consists of visible light or near-infrared light reflected or emitted by the scene or target object, and the image sensor is used to convert the light signal into a processable electrical signal.

[0037] In one possible implementation, the image sensor receives light signals through a pixel array, converts the light signals into electrical charges in each pixel using the photoelectric effect, amplifies the charges through an amplifier, and finally converts the amplified charges into digital electrical signals through an analog-to-digital converter, which are then output to the FPGA module.

[0038] It should be noted that the intensity, wavelength, and pixel response characteristics of the light signal will affect the accuracy of the converted electrical signal. Therefore, the image sensor should be selected with appropriate sensitivity and resolution according to the application scenario.

[0039] As an example, in this application embodiment, a CMOS sensor is used. Its core advantages are high integration, low power consumption, and fast response speed, making it suitable for application scenarios such as space stations where weight and power consumption are strictly limited. It converts weak light signals into digital electrical signals to achieve low-noise and high-sensitivity imaging.

[0040] S202, Configure the image sensor registers.

[0041] The image sensor's registers are used to set the sensor's operating mode, exposure time, and readout parameters, which directly affect the image acquisition quality and speed.

[0042] In one possible implementation, the FPGA module communicates via I... 2 The FPGA module communicates with the image sensor registers via the C or SPI interface, writing data to the registers according to the preset acquisition mode and application requirements. Register configuration includes: selecting global shutter or rolling shutter mode, setting exposure time, controlling gain, setting pixel readout order and resolution, etc. The FPGA module verifies the write status in real time when writing to the registers and performs rewrite or confirmation operations when necessary to ensure that the register configuration is correct.

[0043] As an example, the exposure time of a CMOS image sensor can be set to 10ms, the operating mode to global shutter, and the pixel readout order to progressive output, to adapt to high-speed image acquisition scenarios. Different sensor models may have different register mappings and control commands. In this embodiment, compatible control of different sensors can be achieved through a parameterized register mapping table; however, this application does not limit this aspect.

[0044] S203. Based on configuration and electrical signals, parameterized timing control signals are generated in the FPGA module.

[0045] The timing control signals include: line clock, field clock, reset signal, and data valid signal.

[0046] In one possible implementation, the FPGA module generates parameterized timing control signals through a logic module based on register configuration and input digital electrical signals; the line clock and field clock are used to control the pixel scanning order, the reset signal is used to initialize pixel charge accumulation, and the data valid signal is used to mark valid pixel data; the FPGA internally uses a programmable clock management unit in conjunction with a state machine to achieve high-precision timing generation and support fast switching between different frame rates and resolutions.

[0047] It should be noted that the accuracy of the timing control signal directly determines the frame rate and synchronization accuracy of the acquired image. Therefore, in the implementation, it is necessary to ensure that the signal delay is controllable and stable.

[0048] S204. Image data is acquired by controlling the output of timing control signals through a state machine and a counter.

[0049] In one possible implementation, a state machine is set up in the FPGA module, defining an idle state, a running state, and an end state. In the idle state, an enable signal is detected, and when the enable signal is valid, the state machine transitions from the idle state to the running state. In the running state, a counter is started to periodically count the timing control signals, and the output of the line clock, field clock, reset signal, and data valid signal is controlled according to the counter value. Based on the counter value and a preset threshold, it is determined whether the acquisition of one frame of image has been completed. When the condition is met, the state machine transitions to the end state. In the end state, based on the completion signal and the counter value, the state machine is reset to the idle state to prepare for the acquisition of the next frame of image. The image sensor is driven to output pixel data through the timing control signals, and the image data is acquired and written to the buffer unit in the FPGA module.

[0050] It should be noted that the state machine and counter can be parameterized according to different image resolutions and frame rates to ensure compatibility with different sensors and application scenarios.

[0051] S205. The image data is parsed and processed to obtain the processed image data, and the processed image data is written into the cache unit in the FPGA module.

[0052] In one possible implementation, the acquired image data is read from the cache unit within the FPGA module; the image data is pixel aligned and channel stitched to obtain first image data; the first image data is subjected to region of interest extraction and pixel merging operations according to preset processing parameters to obtain processed image data; the processed image data is formatted, packaged, and written into the cache unit within the FPGA module.

[0053] As an example, in this embodiment, the processing unit can support programmable parameter adjustment, enabling flexible configuration of different resolutions, region of interest sizes, and merging factors, thereby achieving high-speed real-time image processing, reducing data transmission burden, and improving image quality and system response speed.

[0054] S206. Transfer the image data in the cache to the host computer to display the real-time image.

[0055] The host computer is used to receive FPGA cached data and perform display or subsequent analysis and processing.

[0056] In one possible implementation, the FPGA module transmits the processed image data from the buffer unit to the host computer via a high-speed data interface. The host computer software parses the data and displays the image in real time. During data transmission, the FPGA can use a double buffer or pipeline mechanism to ensure transmission continuity and avoid frame loss. At the same time, the data can be simply compressed or packaged to match the transmission bandwidth.

[0057] It should be noted that the interface rate and transmission protocol should be reasonably selected according to the image frame rate and resolution to ensure real-time performance, achieve high-speed, continuous and stable data transmission and real-time display, and meet the real-time image requirements of industrial inspection, scientific research experiments and intelligent vision systems.

[0058] This application embodiment receives optical signals and converts them into electrical signals, configuring the image sensor register, including setting the working mode, exposure time, and readout parameters, to achieve flexible control of the sensor's working state, improving the adaptability and accuracy of data acquisition. By generating parameterized timing control signals in the FPGA module, including line clock, field clock, reset signal, and data validity signal, the precise control of image acquisition timing is improved, ensuring data integrity at high frame rates. A state machine and counter are used to control the output of timing signals, reducing acquisition latency and timing errors while performing orderly acquisition and synchronous processing of each frame of image. The acquired image data undergoes pixel alignment and channel stitching, enabling real-time image data processing, storage, and instant visualization monitoring, meeting the real-time application requirements of high frame rate and high resolution scenarios.

[0059] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, the above S204 can be specifically implemented through the following S301 to S306, which are explained in detail below:

[0060] S301. Set up a state machine in the FPGA module and define the idle state, running state, and end state.

[0061] A state machine is a logical control structure used to perform ordered switching within a finite set of states. It is often implemented using registers and combinational logic circuits and can achieve precise scheduling of complex timing in FPGAs.

[0062] In one possible implementation, the FPGA device establishes a three-state state machine using Verilog code, drives the state register using a clock signal, enters the idle state upon reset, enters the running state upon receiving an enable signal, and enters the end state after completing the acquisition of one frame of image data.

[0063] It should be noted that this state machine structure is scalable and can add intermediate states according to different image sensor interface requirements, thereby enhancing its adaptability to different application scenarios.

[0064] As an example, in the embodiment of this application, the state machine in the guide camera system adopts a synchronous reset design to ensure that it can quickly return to the idle state in abnormal situations.

[0065] S302. In the idle state, detect the module enable signal. When the enable signal is valid, the state machine is switched from the idle state to the running state.

[0066] In one possible implementation, the FPGA module receives an enable signal through an input port. When the signal is high, the logic circuit determines that it is valid and switches the state machine from the idle state to the running state, preparing to enter the image acquisition stage.

[0067] It should be noted that the module enable signal can come not only from the host computer command, but also from an external trigger device, so as to achieve flexible control of image acquisition.

[0068] As an example, in this embodiment of the application, the module enable signal is issued by the task scheduling unit of the satellite platform to ensure that the timing of camera acquisition matches the attitude control requirements.

[0069] S303. In operation, the counter is started to periodically count the timing control signals, and the output of the horizontal clock, vertical clock, reset signal and data valid signal is controlled according to the value of the counter.

[0070] In one possible implementation, the FPGA device uses an internal counter to accumulate counts for each clock cycle. When the count reaches a preset threshold, it outputs control signals such as line synchronization, field synchronization, pixel reset, and data valid, thereby driving the CMOS image sensor to complete line-by-line and field-by-field pixel reading. The bit width and maximum value of the counter should match the resolution and line / field timing parameters of the selected sensor; otherwise, timing deviations may occur.

[0071] As an example, in an embodiment of this application, Figure 4 This is a schematic diagram of the sensor control timing signals. In this embodiment, the ROM resources within the FPGA module are used for implementation. The ROM is generated by configuring the block RAM (BRAM) embedded in the FPGA hardware. The depth of the ROM is set to the number of clock cycles (560), and the bit width is the number of control signals. That is, the resource size of the ROM is 560*18. The state of different control signals at each moment is written into the initialization configuration of the ROM. When the FPGA is running, the clock is directly used as the address line input, and the data read from each address is the control timing at the current moment.

[0072] S304. Based on the counter value and the preset threshold, determine whether the acquisition of one frame of image has been completed. When the condition is met, the state machine enters the end state.

[0073] In one possible implementation, the FPGA module continuously compares the counter values ​​during operation. When both the line count and the field count reach the set values, it determines that the image acquisition of that frame is complete and switches the state machine to the end state.

[0074] It should be noted that the preset threshold is a judgment standard set based on the resolution and frame structure of the image sensor. The threshold setting should be flexible and adjustable in order to adapt to image sensors of different specifications.

[0075] As an example, in this embodiment, the threshold setting is consistent with the resolution of the GSENSE2020BSI CMOS sensor, thereby ensuring accurate and reliable frame synchronization.

[0076] S305. In the end state, based on the completion signal and the counter value, the state machine is reset to the idle state to prepare for the acquisition of the next frame of image.

[0077] In one possible implementation, upon completion, a completion signal is output to notify the upper-layer logic that the current frame acquisition is complete. Simultaneously, the internal counter is cleared and the state machine is reset, returning it to an idle state to respond to the next enable signal. The completion signal can also serve as a trigger condition to synchronize the backend image buffer and data processing modules, ensuring the integrity of the image data.

[0078] S306: Drives the image sensor to output pixel data through timing control signals, acquires image data and writes it into the cache unit in the FPGA module.

[0079] In one possible implementation, the FPGA module receives the data stream from the CMOS sensor while outputting timing control signals. After data alignment and channel concatenation, the data is written into a buffer to achieve real-time acquisition and storage of the entire frame image. The size and bandwidth of the buffer need to be configured appropriately according to the sensor output rate to avoid data loss or delay.

[0080] As an example, this application embodiment provides a complete implementation process example based on the above S301-S306. Figure 5 A schematic diagram of the FPGA-based camera timing control flow provided in the embodiments of this application is shown below. Figure 5As shown, the process first enters the initialization phase and performs a reset detection, where the reset signal rst_n serves as the initial control signal for the state machine. If the reset signal is invalid, the system remains in the reset state; if the reset signal is valid, the initial register parameters of the image sensor are set, and the system enters the idle state, waiting for external instructions. In the idle state, the system detects the external module enable signal, which is a trigger signal issued by the host computer or external task control unit. When the signal is detected as valid, the state machine transitions from the idle state to the running state; if the signal is invalid, the system remains in the idle state. After entering the running state, the counter begins to increment, and corresponding timing control signals are generated based on the count value. The timing counter is used to control the counting of the horizontal and vertical clocks. When its value reaches the set threshold of 559, the system automatically enters the end counter determination phase. In the end counter determination phase, the counter continues to accumulate. When its value reaches 2047, it is determined that the image acquisition for that frame is complete, and the state machine enters the end state. The end state is used to generate an acquisition completion signal and trigger data writing to the buffer module. In the final state, the state machine notifies the outside world that frame acquisition has ended by sending a completion signal, while clearing the counter to zero and automatically resetting to the idle state, ready to respond to the next image acquisition command.

[0081] Preferred, Figure 6 This is a schematic diagram of the FPGA module simulation waveform in an embodiment of this application, as shown below. Figure 6 As shown, the simulation waveform obtained by importing the Verilog code of the designed timing control method into Vivado is shown. This waveform is similar to... Figure 4 Timing of sensor control signals plotted Figure 1 This indicates that the timing of the FPGA program signals designed in the embodiments of this application is correct.

[0082] It should be noted that the process examples in this application are only for illustrating the principle. In actual applications, the threshold of the counter, the switching conditions of the state machine, and the buffer configuration can be adjusted according to the parameters of different image sensors. As an example, in the implementation of the guiding camera, the thresholds of the timing counter and the end counter are set to 559 and 2047, respectively, to adapt to the resolution requirements of the CMOS image sensor, thereby ensuring the integrity and real-time performance of frame data acquisition.

[0083] This application employs a state machine combined with a counter to achieve precise control of the line clock, field clock, reset signal, and data validity signal, avoiding the resource waste and lack of flexibility associated with traditional lookup table methods or fixed logic approaches. The state machine, through orderly switching between idle, running, and terminated states, ensures self-consistency and traceability in the acquisition, storage, and reset of each image frame, thereby improving system stability and real-time performance. Simultaneously, the dynamic threshold determination mechanism of the counter allows for flexible adjustment based on different resolutions and sensor parameters, enhancing the system's versatility and scalability. Furthermore, the acquired image data is directly written to the FPGA cache unit, ensuring high-speed image processing and data transmission, reducing the load on external processors, and further improving the overall system efficiency and reliability. The entire timing control of the image sensor is completed within the FPGA.

[0084] The foregoing mainly describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as an FPGA-based camera timing control device, includes at least one of the hardware structures and software modules corresponding to each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0085] This application embodiment can divide the FPGA-based camera timing control device into functional units according to the above method example. For example, each function can be divided into its own functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0086] When using integrated units, Figure 7 A possible structural schematic diagram of the FPGA-based camera timing control device (hereinafter referred to as FPGA-based camera timing control device 70) involved in the above embodiments is shown. The FPGA-based camera timing control device 70 includes an image sensor module 701 and an FPGA module 702, and may also include a storage unit 703. Figure 7 The schematic diagram shown can be used to illustrate the structure of the FPGA-based camera timing control device involved in the above embodiments.

[0087] when Figure 7 The schematic diagram shown illustrates the structure of the FPGA-based camera timing control device in the above embodiments. The image sensor module 701 receives light signals and converts them into electrical signals. The FPGA module 702, connected to the image sensor, is used for: configuring the image sensor's registers, including setting the operating mode, exposure time, and readout parameters; generating parameterized timing control signals based on the configuration and electrical signals, including a line clock, a field clock, a reset signal, and a data valid signal; controlling the output of the timing control signals through a state machine and a counter; acquiring image data; parsing and processing the image data to obtain processed image data, and writing the processed image data into a buffer unit within the FPGA module; and transmitting the image data from the buffer to a host computer for real-time image display.

[0088] In one possible implementation, FPGA module 702 includes:

[0089] The system comprises the following components: a register configuration unit for configuring the image sensor's operating mode, exposure time, and readout parameters; a timing control unit for generating parameterized timing control signals based on the configuration, including a line clock, a field clock, a reset signal, and a data valid signal; a state machine and counter control unit for controlling the output of timing control signals, acquiring image data, and switching states based on counter values ​​and state machine states; an image processing unit for parsing, pixel alignment, channel stitching, region of interest extraction, and pixel merging of the acquired image data, and writing the processed image data into the buffer unit within the FPGA module; and a data transmission unit for transmitting the processed image data from the buffer unit within the FPGA module to the host computer for real-time display.

[0090] In one possible implementation, the FPGA module 702 also includes a synchronization constraint unit for constraining clock, logic grouping, and output delay to ensure the timing stability and signal transmission accuracy of each functional module.

[0091] In one possible implementation, the FPGA module 702 further includes: an external interface unit for data communication with the image sensor, host computer, and other peripheral modules; a trigger control unit for receiving external trigger signals and controlling the state machine and counter to start or stop image acquisition; and a power management unit for controlling the power-on sequence, power supply voltage, and power stability of the FPGA module and the image sensor.

[0092] In one possible implementation, the FPGA module 702 is used to receive an optical signal and convert it into an electrical signal, including: receiving the optical signal; converting the optical signal into a charge in the image sensor pixel through the photoelectric effect; amplifying the charge through an amplifier; and converting the amplified charge into a digital electrical signal through an analog-to-digital converter.

[0093] In one possible implementation, the FPGA module 702 is further used to control the output of timing control signals through a state machine and a counter to acquire image data, including: setting up a state machine in the FPGA module and defining an idle state, a running state, and an end state; in the idle state, detecting a module enable signal, and when the enable signal is valid, causing the state machine to transition from the idle state to the running state; in the running state, starting a counter to periodically count the timing control signals, and controlling the output of the line clock, field clock, reset signal, and data valid signal according to the counter value; determining whether the acquisition of one frame of image has been completed based on the counter value and a preset threshold, and when the condition is met, the state machine transitions to the end state; in the end state, based on the completion signal and the counter value, resetting the state machine to the idle state to prepare for the acquisition of the next frame of image; driving the image sensor to output pixel data through timing control signals, acquiring image data, and writing it into the buffer unit within the FPGA module.

[0094] In one possible implementation, the FPGA module 702 is further used to parse and process the image data to obtain processed image data, including: reading the acquired image data from the cache unit within the FPGA module; performing pixel alignment and channel stitching on the image data to obtain first image data; performing region of interest extraction and pixel merging operations on the first image data according to preset processing parameters to obtain processed image data; formatting and packaging the processed image data, and writing it into the cache unit within the FPGA module.

[0095] In one possible implementation, FPGA module 702 is also used to constrain clock, logic grouping, and output delay within the FPGA module using a synchronization constraint file.

[0096] Figure 7If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0097] Figure 8 This is a physical diagram of the FPGA-based camera timing control device developed according to an embodiment of this application.

[0098] This application also provides a hardware structure diagram of an electronic device (denoted as electronic device 90), see [link to diagram]. Figure 9 The electronic device 90 includes a processor 901, and optionally, a memory 902 connected to the processor 901.

[0099] In the first possible implementation, see Figure 9 The electronic device 90 also includes a transceiver 903. The processor 901, memory 902, and transceiver 903 are connected via a bus. The transceiver 903 is used to communicate with other devices or communication networks. Optionally, the transceiver 903 may include a transmitter and a receiver. The device in the transceiver 903 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 903 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0100] Based on the first possible implementation method Figure 9 The structural diagram shown can be used to illustrate the structure of the electronic device involved in the above embodiments.

[0101] in, Figure 9 This can also be illustrated by a system chip in an electronic device. In this case, the actions performed by the aforementioned electronic device can be implemented by this system chip; the specific actions performed can be found above and will not be repeated here.

[0102] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0103] The processor in this application may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., and other computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a standalone semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may form a System-on-a-Chip (SoC) with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), or logic circuits that implement dedicated logic operations.

[0104] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0105] This application also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0106] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0107] This application also provides a chip including a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0108] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0109] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0110] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A camera timing control method based on FPGA, characterized in that, include: It receives optical signals and converts them into electrical signals; Configure the image sensor's registers, including setting the operating mode, exposure time, and readout parameters; Based on the configuration and the electrical signals, parameterized timing control signals are generated in the FPGA module. The timing control signals include: line clock, field clock, reset signal and data valid signal. Image data is acquired by controlling the output of the timing control signal through a state machine and a counter. The image data is parsed and processed to obtain processed image data, and the processed image data is written into the cache unit in the FPGA module; The cached image data is transmitted to the host computer for real-time image display.

2. The method according to claim 1, characterized in that, The process of receiving optical signals and converting them into electrical signals includes: Receive optical signals; The optical signal is converted into electrical charge in the pixels of the image sensor through the photoelectric effect; The charge is amplified by an amplifier; The amplified charge is converted into a digital electrical signal using an analog-to-digital converter.

3. The method according to claim 1, characterized in that, The process of controlling the output of the timing control signal through a state machine and a counter to acquire image data includes: Set up a state machine in the FPGA module and define the idle state, running state, and end state; In the idle state, the detection module enable signal is used. When the enable signal is valid, the state machine is switched from the idle state to the running state. In operation, the counter is started to periodically count the timing control signals, and the output of the horizontal clock, vertical clock, reset signal and data valid signal is controlled according to the value of the counter; Based on the counter value and the preset threshold, it is determined whether the acquisition of one frame of image has been completed. When the condition is met, the state machine enters the end state. In the end state, based on the completion signal and the counter value, the state machine is reset to the idle state to prepare for the acquisition of the next frame of image; The image sensor is driven by timing control signals to output pixel data, and the image data is acquired and written into the cache unit in the FPGA module.

4. The method according to claim 1, characterized in that, The step of parsing and processing the image data to obtain processed image data includes: Read the acquired image data from the cache unit within the FPGA module; The image data is pixel-aligned and channel-stitched to obtain the first image data; Based on preset processing parameters, the first image data is subjected to region of interest extraction and pixel merging operations to obtain processed image data; The processed image data is formatted, packaged, and written into the cache unit within the FPGA module.

5. The method according to claim 1, characterized in that, The method further includes constraining clock, logic grouping, and output delay using a synchronization constraint file in the FPGA module.

6. A camera timing control device based on FPGA, comprising: Image sensor module and FPGA module; The image sensor module is used to receive light signals and convert them into electrical signals. The FPGA module, connected to the image sensor, is used for: Configure the image sensor registers, including setting the operating mode, exposure time, and readout parameters; Based on the configuration and the electrical signal, a parameterized timing control signal is generated, the timing control signal including a line clock, a field clock, a reset signal and a data valid signal; Image data is acquired by controlling the output of the timing control signal through a state machine and a counter. The image data is parsed and processed to obtain processed image data, and the processed image data is written into the cache unit in the FPGA module; The cached image data is transmitted to the host computer for real-time image display.

7. The apparatus according to claim 6, characterized in that, The FPGA module includes: The register configuration unit is used to configure the image sensor's operating mode, exposure time, and readout parameters. A timing control unit is used to generate parameterized timing control signals according to the configuration, including a line clock, a field clock, a reset signal, and a data valid signal; The state machine and counter control unit is used to control the output of the timing control signal, acquire image data, and switch states according to the counter value and the state machine state. The image processing unit is used to parse, align pixels, stitch channels, extract regions of interest and merge pixels of the acquired image data, and write the processed image data into the cache unit in the FPGA module. The data transmission unit is used to transmit the processed image data from the cache unit within the FPGA module to the host computer for real-time display.

8. The apparatus according to claim 7, characterized in that, The FPGA module also includes a synchronization constraint unit, which is used to constrain the clock, logic grouping and output delay to ensure the timing stability and signal transmission accuracy of each functional module.

9. The apparatus according to claim 6, characterized in that, The FPGA module also includes: External interface unit, used for data communication with image sensor, host computer and other peripheral modules; The trigger control unit is used to receive external trigger signals and control the state machine and counter to start or stop image acquisition; The power management unit is used to control the power-on sequence, power supply voltage, and power stability of the FPGA module and image sensor.

10. A computer-readable storage medium storing instructions that, when executed on an FPGA-based camera timing control device, cause the FPGA-based camera timing control device to perform any one of the methods of claims 1-5.

Images