Camera supporting simultaneous output of digital and analog signals
By combining the design of CMOS image sensor, FPGA and encoder in the camera, the simultaneous output of digital and analog signals is achieved, solving the problem that existing cameras cannot output simultaneously, realizing equipment upgrades and cost controls, and improving image quality at the same time and flexibility.
Patent Information
- Application Number
- CN202420776913.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-15
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2034-04-15
AI Technical Summary
Existing cameras are unable to output digital and analog signals at the same time, resulting in difficulties in equipment upgrades and cost control.
A camera including a CMOS image sensor, FPGA, microprocessor, encoder and digital/analog interface was designed. The image data was received and processed through the FPGA, and divided it into two copies, which were output as digital signals and analog signals respectively. Supports NTSC and PAL standards.
It realizes the output of digital and analog signals simultaneously on a CMOS image sensor, which is suitable for equipment upgrades and reduces costs, while providing flexible exposure time and gain configuration, improving image resolution and frame rate.
Smart Images
Figure CN222169908U_ABST
Abstract
Description
Technical Field
[0001] The utility model relates to the technical field of image acquisition and output, and in particular to a camera supporting simultaneous output of digital and analog signals. Background Art
[0002] Cameras can be divided into analog cameras and digital cameras according to the output image signal format. Analog cameras mostly use PAL / NTSC / and other standard analog outputs. With the development and popularization of digital interface technologies such as GigE / USB 2.0 / USB 3.0 / Camera Link / CoaXPress, more and more digital cameras are replacing traditional analog cameras in various machine vision systems.
[0003] Analog cameras usually use CCD image sensors. Although CCD was developed earlier, with the continuous maturity of CMOS, the current output and models of CCD are becoming less and less, and the cost is rising accordingly. Some devices using analog cameras are in urgent need of upgrading to digital cameras or looking for lower-cost analog cameras. For example, patent publication number CN 116208722 A discloses a synchronous analog camera based on CMOS image sensor, including image sensor, FPGA control chip, communication interface, digital-to-analog converter, external trigger interface and power chip; and publication number CN204810416U discloses a multifunctional high-definition digital camera supporting multiple communication protocols. The camera includes a CMOS image sensor system, an FPGA control system and an ARM control system, wherein the CMOS image sensor system is connected to the FPGA control system through an SPI interface, so that the optical signal is converted into multiple LVDS digital signals through the CMOS image sensor system and transmitted to the FPGA control system; the FPGA control system collects and arranges the LVDS digital signals, outputs them as CSI interface timing, and transmits them to the ARM control system connected to the FPGA control system for processing. The ARM control system can transmit them to the host computer or directly process the data according to the communication protocol. However, there is currently no camera that can output both digital and analog signals simultaneously. Utility Model Content
[0004] The purpose of the utility model is to provide a camera that supports the simultaneous output of digital and analog signals, and can realize the simultaneous output of digital signals and analog signals.
[0005] In order to achieve the above purpose, the utility model provides the following technical solutions:
[0006] A camera supporting simultaneous output of digital and analog signals, comprising a CMOS image sensor, an FPGA and an analog interface, the camera also comprising a second memory, a microprocessor, an encoder and a digital interface;
[0007] The FPGA is connected to the CMOS image sensor, the microprocessor and the encoder respectively, and is used for receiving, processing, caching and transmitting image data;
[0008] The microprocessor is connected to the FPGA, the second memory and the digital interface respectively, and is used to output digital signals;
[0009] The encoder is connected to the FPGA and the analog interface respectively, and is used for outputting analog signals.
[0010] The working process of the camera is as follows: the FPGA is used to receive and process image data from the CMOS image sensor, and the processed image data is cached and divided into two parts: one part of the image data is transmitted to the microprocessor, and the microprocessor outputs the image data to the digital interface to output a digital signal; the other part of the image data is transmitted to the encoder, and the encoder converts the image data into an analog signal and outputs it to the analog interface to output an analog signal.
[0011] Furthermore, the FPGA includes:
[0012] An acquisition module, used for acquiring image data of a CMOS image sensor;
[0013] A data sorting module is used to separate the image data collected by the collection module into two sets of image data, which are used for digital signal output and analog signal output respectively;
[0014] An image signal processing module is used to process two sets of image data, including CCM color correction, linearity correction, sharpening and denoising;
[0015] A cache module, used for sequentially caching the two processed image data into the cache module;
[0016] A digital output module is used to read a copy of the image data in the cache module, output it to the microprocessor, and output a digital signal through a digital interface;
[0017] The image cropping module is used to read another image data in the cache module and crop it to a standard window size;
[0018] The analog output module is used to output the cropped image data to the encoder to convert it into an analog signal, and output the analog signal through the analog interface.
[0019] The CMOS image sensor can alternately output two frames of image data with different exposure times and gains, and the data sorting module in the FPGA sorts the image data into two sets of image data according to the output mode of the CMOS image sensor.
[0020] The image signal processing module is an ISP module, which has CCM color correction, gamma correction, sharpening and denoising functions to improve the image quality. It also has a linearity correction function to compensate for the linearity loss caused by the analog signal output.
[0021] The image cropping module is used to crop the analog signal into a window size that complies with the NTSC or PAL standard.
[0022] Furthermore, the second memory stores initialization configuration information of the CMOS image sensor, and is used for the microprocessor to control the FPGA to configure the initialization configuration information into the CMOS image sensor after power-on.
[0023] Further preferably, the FPGA includes a CMOS configuration module connected to the microprocessor, the microprocessor reads the initialization configuration information of the CMOS image sensor stored in the second memory and outputs the information to the CMOS configuration module, and the CMOS configuration module performs initialization configuration on the CMOS image sensor.
[0024] Furthermore, the camera includes a GPIO, and the GPIO is connected to the FPGA and is used to control exposure time switching and whether to turn on automatic exposure.
[0025] Further preferably, the FPGA has a built-in exposure control module connected to the GPIO, and the exposure control module controls exposure time switching and whether to turn on automatic exposure according to the state of the GPIO.
[0026] Further preferably, the GPIO includes GPIO0, GPIO1, GPIO2 and GPIO3; when the digital interface is not used, the exposure module controls the CMOS image sensor to switch between 8 pre-stored exposure times according to the current states of GPIO0, GPIO1 and GPIO2; when the input of GPIO3 is a high level, the automatic exposure function is preferentially turned on; when the digital interface is used, the exposure time of the CMOS image sensor is controlled by the host computer.
[0027] Furthermore, the camera includes a first memory connected to the FPGA for caching image data.
[0028] Further preferably, the first memory is connected to a cache module in the FPGA.
[0029] Further preferably, the first memory is DDR3 for image data caching, the second memory is EEROM, and the FPGA automatically allocates storage addresses according to the current image data size; the analog interface is an AV interface, the digital interface is a USB interface, and the encoder is an AV encoder.
[0030] The beneficial effects of the utility model are as follows: the camera provided by the utility model can output digital signals and analog signals at the same time based on the use of only one CMOS image sensor, combined with FPGA technology; it can be used to replace the analog camera of the original equipment and increase the digital output of the original equipment to achieve equipment upgrading. On this basis, the utility model can also configure different exposure times and gains for the digital and analog images respectively. In addition, the camera provided by the utility model can be used as a low-cost analog camera to meet some equipment requirements, and can also be used as a digital camera. The digital image has a higher resolution, a faster frame rate, and the window size can be flexibly configured. The range of exposure time adjustment is also wider, the cost is low and the application range is wide. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the utility model or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the utility model. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 A schematic structural diagram of a CMOS camera provided by the utility model;
[0033] Figure 2 A schematic diagram of an internal module of an FPGA provided by the utility model;
[0034] Figure 3 Schematic diagram of CMOS image sensor outputting two frames of images alternately and FPGA sorting;
[0035] Figure 4 Schematic diagram of linearity adjustment of image data output as an analog signal. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the utility model to clearly and completely describe the technical solutions in the embodiments of the utility model. Obviously, the described embodiments are only part of the embodiments of the utility model, not all of the embodiments. Based on the embodiments in the utility model, all other embodiments obtained by ordinary technicians in this field through simple deformation should fall within the scope of protection of the utility model.
[0037] like Figure 1As shown, an embodiment of the utility model provides a camera that supports simultaneous output of digital and analog signals, including a CMOS image sensor, an FPGA, a microprocessor, an encoder, a digital interface connected to the microprocessor, an analog interface connected to the encoder, a first memory connected to the FPGA, a second memory connected to the FPGA and the microprocessor, and a GPIO connected to the FPGA.
[0038] The FPGA is connected to the CMOS image sensor, microprocessor and encoder respectively; the FPGA receives and processes the image data from the CMOS image sensor, outputs part of the processed image data to the microprocessor and then to the digital interface, and outputs the other part of the processed image data to the encoder to be converted into an analog signal (the output format supports NTSC and PAL standards) and then output to the analog interface.
[0039] The first memory caches the image data received by the FPGA and the processed image data.
[0040] The second memory stores initialization configuration information of the CMOS image sensor, and is used for the microprocessor to control the FPGA to configure the initialization configuration information into the CMOS image sensor after power-on, thereby completing the initialization configuration.
[0041] GPIO is connected to FPGA to control exposure time switching and whether to enable automatic exposure.
[0042] In this embodiment, the first memory is DDR3, which is used for image data caching, the second memory is EEROM, and the FPGA automatically allocates storage addresses according to the current image data size; the analog interface is an AV interface, the digital interface is a USB3.0 interface, and the encoder is an AV encoder.
[0043] In another implementation, the analog interface may be a VGA interface, an RCA interface, an S-terminal interface, a BNC interface, etc.; the first memory may be other types of memory chips; the digital interface may be a digital interface such as GigE, USB 2.0, CameraLink, CoaXPress, etc.
[0044] The EEROM stores the initialization configuration information of the CMOS image sensor, which is used for the microprocessor to control the FPGA to configure the initialization configuration information into the CMOS image sensor after power-on, thereby completing the initialization configuration.
[0045] The FPGA is used to receive and process the image data from the CMOS image sensor (the processing includes but is not limited to analysis, linearity correction and cropping), and then divide the processed image data into two parts, one of which is transmitted to the microprocessor, and the microprocessor outputs one image data to the USB3.0 interface; the other is transmitted to the AV encoder, and the AV encoder converts the other image data into an analog signal and outputs it to the AV interface.
[0046] In this embodiment, if Figure 2 As shown, the FPGA includes a CMOS configuration module, an exposure control module, an acquisition module, a data sorting module, an image signal processing module, a DDR3 cache module, a digital output module, an image cropping module and an analog output module.
[0047] CMOS configuration module: used to receive the initialization configuration information of the microprocessor and configure the CMOS image sensor through the SPI or I2C protocol;
[0048] Exposure control module: used to configure the exposure time of the CMOS image sensor in real time according to the external GPIO status, or whether to automatically expose;
[0049] Acquisition module: used to parse LVDS or MIPI image data and convert it into parallel image data;
[0050] Data sorting module: used to separate the image data into two parts, which are output through digital signal and analog signal respectively;
[0051] Image signal processing module, i.e., ISP module: has CCM color correction, gamma correction, sharpening and denoising functions to improve image quality; the ISP module also has a linearity correction function to compensate for the linearity loss caused by the analog signal output.
[0052] DDR3 cache module: used to integrate the DDR3 controller, cache each image data in sequence and store it in the first memory at the same time;
[0053] Digital output module: used to transmit a copy of image data to the microprocessor in parallel;
[0054] Image cropping module: used to crop another image data to a window size that meets the standard according to the NTSC or PAL standard;
[0055] Analog output module: used to transmit the cropped image data to the AV encoder and convert it into an analog signal.
[0056] In one implementation, the CMOS image sensor has the function of outputting two frames of images in alternating rows, and two frames of images with different exposure times and gains can be output by configuring a register once. Figure 3 As shown, after the FPGA collects the image data from the CMOS sensor, the data sorting module separates the two frames of images in the manner of odd and even rows.
[0057] In another implementation, the CMOS image sensor outputs images with different exposure times and gains in two frames. Different exposure times and gains need to be configured before each frame starts to be exposed. The data sorting module of the FPGA separates the two frames of images according to the odd and even frames.
[0058] In this embodiment, there are 4 GPIOs, namely GPIO0, GPIO1, GPIO2 and GPIO3. When the digital interface is not used, the exposure control module controls the CMOS image sensor to switch between the 8 pre-stored exposure times according to the current status of GPIO0, GPIO1 and GPIO2; when the input of GPIO3 is high, the automatic exposure function is enabled first; when the digital interface is used, the GPIO control of the exposure time is ignored, and the exposure time of the CMOS image sensor is controlled by the software of the host computer. It is known to those skilled in the art that in the prior art, the camera is generally connected to the host computer through a data interface, preferably a computer and a USB interface.
[0059] In one implementation, the image cropping module of the FPGA crops the image data used to output the analog signal into a window size that complies with the NTSC or PAL standard.
[0060] The working process of the camera supporting simultaneous output of digital and analog signals provided in this embodiment is as follows:
[0061] S1. After the camera is powered on, the microprocessor reads the sensor initialization configuration information in the EEPROM, the CMOS configuration module receives the initialization configuration information of the microprocessor, and then initializes the CMOS image sensor through the SPI or I2C protocol;
[0062] S2. When the digital interface is not used, the exposure control module controls the CMOS image sensor to switch among the 8 pre-stored exposure times according to the current status of GPIO0, GPIO1 and GPIO2; when the input of GPIO3 is high, the automatic exposure function is enabled first; when the digital interface is used, the exposure time of the CMOS image sensor is controlled by the host computer;
[0063] S3, the acquisition module acquires image data of the CMOS image sensor, and the data sorting module separates the image data into two sets of image data, including image data for digital signal output and image data for analog signal output;
[0064] S4, the image signal processing module, i.e., the ISP module, performs CCM color correction, gamma correction, sharpening, and denoising on the image to improve the image quality; the ISP module also has a linearity correction function to compensate for the linearity loss caused by the analog signal output;
[0065] like Figure 4 As shown, after the digital signal undergoes analog conversion in step S2, the linearity of the input value and the output value will be lost to a certain extent, and P(out)=P(in) is not satisfied. If the current relationship satisfies P(out)=F(in), F(in) is mapped to P(in), and a correction table is obtained. The table is stored in FPGA, and the linearity of the image data output by the analog signal is corrected by looking up the table.
[0066] S5, the DDR3 cache module caches the image data for digital signal output and the image data for analog signal output into DDR3 in sequence;
[0067] S6, the digital output module reads the image data for digital signal output from the DDR3 cache, and sends it to the microprocessor, which outputs the digital signal through the USB3.0 interface;
[0068] S7, the image cropping module reads the image data for analog signal output from the DDR3 buffer, and crops it to a window size that complies with the NTSC or PAL standard;
[0069] S8. The analog output module sends the cropped image data for analog signal output to the AV encoder, and the AV encoder outputs the analog signal through the AV interface.
[0070] In one implementation, the AV encoder can be configured to support NTSC-M, NTSC-J, NTSC-433, PAL-B / D / G / H / I, PAL-M, PAL-N and PAL-60, and can be applied to a variety of playback devices. The field frequency of NTSC-M, NTSC-J, NTSC-433 and PAL-60 is 60HZ, and the other formats are 50HZ. Since the analog channel only uses half of the image data of the CMOS image sensor, the output frame rate of the CMOS sensor needs to be configured at 120HZ or 100HZ.
[0071] In one implementation, when the analog interface and the digital interface are used together, the configurable range of the exposure time needs to be less than 1 / 120s or 1 / 100s due to the frame rate requirements of the analog output format; however, when the digital interface is used alone, the exposure time is no longer limited.
[0072] In summary, the utility model is based on the use of only one CMOS image sensor, combined with FPGA technology, can simultaneously output digital signals and analog signals, and the digital and analog images can be configured with different exposure times and gains. In addition, digital images have a higher resolution, a faster frame rate, can flexibly configure the window size, the exposure time can be adjusted in a wider range, the cost is low and the application range is wide.
[0073] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0074] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a preferred embodiment of the utility model and is not intended to limit the scope of protection of the utility model. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the utility model are included in the scope of protection of the utility model.
Claims
1. A camera supporting simultaneous output of digital and analog signals, comprising a CMOS image sensor, an FPGA and an analog interface, characterized in that: The camera includes a second memory, a microprocessor, an encoder and a digital interface; The FPGA is connected to the CMOS image sensor, the microprocessor and the encoder respectively, and is used for receiving, processing, caching and transmitting image data; The microprocessor is connected to the FPGA, the second memory and the digital interface respectively, and is used to output digital signals; The encoder is connected to the FPGA and the analog interface respectively, and is used for outputting analog signals.
2. The camera supporting simultaneous output of digital and analog signals according to claim 1, characterized in that: The FPGA includes: An acquisition module, used for acquiring image data of a CMOS image sensor; A data sorting module is used to separate the image data collected by the collection module into two sets of image data, which are used for digital signal output and analog signal output respectively; An image signal processing module is used to process two sets of image data, including CCM color correction, linearity correction, sharpening and denoising; A cache module, used for sequentially caching the two processed image data into the cache module; A digital output module is used to read a copy of the image data in the cache module, output it to the microprocessor, and output a digital signal through a digital interface; The image cropping module is used to read another image data in the cache module and crop it to a standard window size; The analog output module is used to output the cropped image data to the encoder to convert it into an analog signal, and output the analog signal through the analog interface.
3. The camera supporting simultaneous output of digital and analog signals according to any one of claims 1-2, characterized in that: The second memory stores initialization configuration information of the CMOS image sensor, and is used for the microprocessor to control the FPGA to configure the initialization configuration information into the CMOS image sensor after power-on.
4. The camera supporting simultaneous output of digital and analog signals according to claim 3, characterized in that: The FPGA includes a CMOS configuration module connected to a microprocessor. The microprocessor reads the initialization configuration information of the CMOS image sensor stored in the second memory and outputs it to the CMOS configuration module. The CMOS configuration module performs initialization configuration on the CMOS image sensor.
5. The camera supporting simultaneous output of digital and analog signals according to any one of claims 1-2, characterized in that: The camera includes a GPIO connected to the FPGA, which is used to control the exposure time switching and whether to turn on automatic exposure.
6. The camera supporting simultaneous output of digital and analog signals according to claim 5, characterized in that: The FPGA has a built-in exposure control module connected to the GPIO, and the exposure control module controls the exposure time switching and whether to start the automatic exposure according to the state of the GPIO.
7. The camera supporting simultaneous output of digital and analog signals according to claim 6, characterized in that: The GPIO includes GPIO0, GPIO1, GPIO2 and GPIO3; when the digital interface is not used, the exposure module controls the CMOS image sensor to switch between 8 pre-stored exposure times according to the current states of GPIO0, GPIO1 and GPIO2; when the input of GPIO3 is at a high level, the automatic exposure function is enabled first; when the digital interface is used, the exposure time of the CMOS image sensor is controlled by the host computer.
8. The camera supporting simultaneous output of digital and analog signals according to any one of claims 1-2, characterized in that: The camera comprises a first memory connected to the FPGA and used for caching image data.
9. The camera supporting simultaneous output of digital and analog signals according to claim 8, characterized in that: The first memory is connected to a cache module in the FPGA.
10. The camera supporting simultaneous output of digital and analog signals according to claim 8, characterized in that: The first memory is DDR3, the second memory is EEROM, the analog interface is an AV interface, the digital interface is a USB interface, and the encoder is an AV encoder.