A camera stable acquisition device and method based on a CX3 chip

Abstract

A camera stable acquisition device and method based on a CX3 chip. The present invention provides a camera stable acquisition method based on a CX3 chip, and the method includes: monitoring an image acquisition signal, configuring the CX3 chip when the image acquisition signal is received to make the DMA channel of the CX3 chip enter the override mode, obtaining an independent buffer from the DMA channel of the CX3 chip in the override mode to send data header packets and tail packets, then resetting the DMA channel of the CX3 chip to reset it to the normal working mode, receiving image data from an image sensor, and automatically sending it out through the DMA channel. The present invention can solve the problem that the fixed state machine of the CX3 chip causes the inability to encapsulate image data according to the U3V protocol, and can achieve stable data transmission and verification while ensuring that it can follow the U3V protocol.

Description

Technical Field

[0001] The present invention relates to the field of imaging devices, and particularly to a camera stable acquisition device and method based on a CX3 chip. Background Art

[0002] Currently, most industrial cameras using the USB3.0 VISION (abbreviated as U3V) protocol on the market still adopt the architecture scheme of an FX3 chip cooperating with an FPGA. Due to the use of DDR for data caching, the abnormal handling of bus blocking performs better. However, the system design is relatively complex, the cost is relatively high, and with the power consumption and heat dissipation problems of the FPGA chip, it is difficult to achieve the design of a miniaturized low-cost camera.

[0003] Some manufacturers' FPGA chips provide an architecture scheme for converting parallel data output in Mipi. This kind of chip integrates a MIPI hard core, which reduces the development difficulty. This scheme is also a frameless camera scheme, and there will also be problems of bus blocking and frame loss. However, due to the presence of the FPGA chip, the cost is still higher than that of a CX3 camera, and the architecture is more complex.

[0004] Moreover, with the increase in chip costs and the deterioration of delivery times, more and more cameras adopt designs without FPGA chips. When designing, the example provided by the development board is a simple acquisition system based on a CX3 chip in UVC format, which can realize image data acquisition, encapsulate image data in UVC format, and send it out through a USB3.0 interface, and can support sensor acquisition with a bandwidth of 300MB, and can be applied to some applications of low-speed sensors. However, the state machine provided by this design can only support continuous acquisition of the UVC protocol. When the bus is blocked, the acquisition will be disordered, and the stability of long-term acquisition cannot be guaranteed, and it cannot support the important acquisition method of industrial cameras, trigger acquisition. When using the U3V protocol, since the state machine is fixed, in order to ensure the transmission speed, the DMA channel needs to be configured in the automatic mode. In the automatic mode, according to the state machine, only when the buffer data is full will the buffer be automatically sent to the UIB. In addition, at the end of the frame, the CPU can participate and end the buffer in advance. But at other times, the CPU cannot obtain the buffer. That is, the DMA buffer of the current state machine can only be obtained when image data arrives, and there are no extra buffer header packets and tail packets, and data information cannot be coordinated and sent.

[0005] In summary, the state machine provided by the CX3 chip has a problem. Although it is applicable to the UVC protocol and the header packet of the UVC protocol can be sent together with the image data. However, there will be problems if the U3V protocol is to be followed. The header packet and the trailer packet need to be sent in independent packets, and there is no position in the original state machine to send the header packet and the trailer packet. Moreover, the state machine of CX3 is defined by the manufacturer as a fixed and unmodifiable state machine. Due to the inability to send the header packet and the trailer packet, the existing CX3 chips are still unable to solve the problem of data acquisition from the sensor and external output under the U3V protocol. The image data without the header packet and the trailer packet will not be able to be displayed properly. Summary of the Invention

[0006] In view of the above problems, the present invention aims to provide a firmware design for an acquisition system of an industrial camera that is small-sized, low-cost, low-power, simple in architecture, and easy to develop and maintain by directly controlling a sensor with an Mipi interface through a CX3 chip (CYUSB3065) and implementing the standard USB3.0 VISION protocol. The present invention realizes a fixed state machine based on the CX3 chip and stable signal transmission under the USB3.0 VISION protocol, without the need to use an FPGA, which can not only greatly reduce costs, reduce the volume of the camera, reduce heat generation, but also ensure signal stability.

[0007] The solution of the present invention includes the implementation of the USB3.0 VISION protocol control based on the CX3 chip, the exception handling of the bus blocking situation during the acquisition process, and the general trigger acquisition solution to solve the above problems.

[0008] In view of this problem, the present invention provides a method for stable acquisition of a camera based on a CX3 chip, and the method includes:

[0009] (1) Monitoring the image acquisition signal, and configuring the CX3 chip when the image acquisition signal is received to make the DMA channel of CX3 enter the override mode;

[0010] (2) Obtaining a buffer from the DMA channel of the CX3 chip;

[0011] (3) Generating a data header packet according to the U3V protocol, storing the header packet in the obtained buffer, and sending it outwards;

[0012] (4) Resetting the DMA channel of the CX3 chip to the normal working mode, receiving the image data from the image sensor, and sending it outwards through the DMA channel;

[0013] (5) Obtaining the size of the image captured by the current sensor, calculating the data volume of each image based on the image size, and determining the number of buffers required for sending the image data according to the calculated data volume

[0014] (6) Monitor in real time whether the required number of buffers for the determined image data is reached. When the required number of buffers is reached, set the DMA channel of the CX3 chip to enter the override mode;

[0015] (7) Obtain a buffer from the DMA transmission of the CX3 chip; generate a data trailer packet according to the U3V protocol, store the trailer packet in the obtained buffer, and send it out;

[0016] (8) Reset the CX3 chip to the normal working mode.

[0017] In a preferred implementation manner, in the steps (3) and (8), data is sent through the DMA channel.

[0018] In another preferred implementation manner, the CX3 chip receives image data from an image sensor.

[0019] In another preferred implementation manner, the CX3 chip transmits the header packet data, image data, and trailer packet data to the USB interface.

[0020] In another preferred implementation manner, the method further includes determining the acquisition mode of the current camera. If it is the continuous mode, after the step (6), obtain a buffer from the DMA channel of the CX3 chip; generate the next data header packet according to the preset configuration of the camera, store the next data header packet in the obtained buffer, and send it out; reset the CX3 chip to the normal working mode.

[0021] In another preferred implementation manner, the method further includes obtaining the size of the image captured by the current image sensor from the firmware parameter storage module of the camera.

[0022] In another preferred implementation manner, the method further includes configuring the operation of the CX3 chip based on the Api interface.

[0023] The present invention also provides a camera stable acquisition device based on the CX3 chip. The camera stable acquisition device includes a CX3 chip. The input end of the CX3 chip is communicatively connected to the sensor of the camera, the output end is communicatively connected to the USB output port, and the firmware of the CX3 chip is communicatively connected to an external control device. When receiving an image acquisition instruction, the camera stable acquisition device performs image acquisition and output according to the described method.

[0024] Technical effects

[0025] 1. Without modifying the state machine of the CX3 chip, the present invention can implement the encapsulation of the U3V protocol, enabling users who originally used the U3V protocol to control cameras not to re-develop the application side and seamlessly switch cameras.

[0026] 2. The present invention can be applied to a camera architecture without frame memory. Such a camera architecture is more inclined to the application scheme of a single low-speed sensor. However, since it is a USB device and there are many USB peripherals on the PC, short-term bus congestion will occur. By designing the acquisition channels of the camera firmware through the method of the present invention, the handling of abnormal situations caused by bus congestion or bus errors can be solved, enabling the camera to stably acquire for a long time.

[0027] 3. The method of the present invention, combined with the switching of the state machine and the characteristics of different sensors, can implement the trigger acquisition scheme, a common acquisition method for industrial cameras.

[0028] 4. By using the CX3 chip, the method of the present invention solves the problem that the data output of the MIPI interface sensor must pass through the FPGA, realizes the direct access of the MIPI interface sensor data to the state machine and output, and can implement a camera acquisition system without an FPGA chip, with a simple architecture, low cost, and can effectively reduce the volume and heat dissipation of the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the architecture and connection of CX3 and the sensor;

[0030] Figure 2 It is a schematic diagram of row and column signals during the signal acquisition process;

[0031] Figure 3 It is a schematic diagram of the internal cache structure of CYUSB3065;

[0032] Figure 4 It is a schematic diagram of the acquisition management based on CX3 in the trigger mode. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The method for stable acquisition of a camera based on the CX3 chip is described in this embodiment. The method of this embodiment includes the following steps:

[0034] (1). Monitor the image acquisition signal. When the image acquisition signal is received, configure the DMA channel to make the DMA channel enter the override mode;

[0035] (2). Obtain a buffer from the DMA channel of the CX3 chip;

[0036] (3). Generate a data header packet according to the U3V protocol, store the header packet in the obtained buffer, and send it outwards;

[0037] (4) Reset the DMA channel to its normal operating mode, receive image data from the image sensor, and send it out through the DMA channel.

[0038] (5) Obtain the size of the image captured by the current sensor, calculate the data volume of each image based on the image size, and determine the number of buffers required for image data transmission according to the calculated data volume.

[0039] (6) Monitor in real time whether the determined image data transmission is completed. When the buffer count is reached, set the DMA channel to enter the override mode.

[0040] (7) Obtain a buffer from the DMA channel of the CX3 chip; generate a data trailer according to the U3V protocol format, store the trailer in the obtained buffer, and send it out. [[ID=B]]

[0041] (8) Reset the CX3 chip to its normal operating mode.

[0042] The following further describes in detail the CYUSB3065 chip (hereinafter referred to as CX3) for implementing the method of the present invention and the process of implementing the method of the present invention in conjunction with the accompanying drawings.

[0043] Figure 1 It is a simplified diagram of the CX3 state machine. As can be seen from the figure, the CX3 device has an integrated MIPI CSI-2 module. One side of this module is hard-connected to the GPIF II interface, and the other side provides an MIPI CSI-2 interface for connecting to an image sensor that supports MIPI CSI-2. The GPIF II module and its fixed-function state machine send the data of the image sensor to the USB interface through the DMA channel.

[0044] As shown in the figure, the front end of the CX3 is connected to the image sensor for receiving image data from the image sensor. A clock line and a data line are respectively arranged between the CX3 and the image sensor for unified clock and data communication. The CX3 also includes a fixed-function GPIF II state machine, a DMA channel, and a controller inside.

[0045] Since the state machine in the CX3 chip can only obtain a buffer when data arrives. The header and trailer of the UVC protocol can be sent out together with the image data, so there is no problem using this state machine. However, the header and trailer data of the U3V protocol need to be sent independently and cannot be sent together with the data. If the buffer obtained when image data arrives is used to send the header, there will be no buffer available to send the data of the current frame.

[0046] Therefore, to solve this problem, when the present invention performs signal acquisition, the controller monitors the image acquisition signal. When the image acquisition signal is received, the CX3 chip is configured so that the DMA channel of the CX3 chip enters the override mode. A buffer is obtained from the DMA channel of the CX3 chip. After the GPIF II module sends the DMA buffer, it will notify the controller through an interrupt.

[0047] Then, the controller generates a data header packet according to the camera U3V protocol, adds header information, stores the header packet in the obtained buffer, and transfers the DMA buffer to the USB interface module. On the GPIF II side, the data in the DMA buffer is automatically packaged.

[0048] After the header packet data is sent, the DMA channel of the CYUSB3065 chip is reset to the normal working mode, and a trigger signal is sent to the image sensor. The image sensor performs signal acquisition and sends it to the CYUSB3065 chip. The CYUSB3065 chip receives the image data from the image sensor frame by frame and sends it out through the DMA channel.

[0049] As Figure 2 shown, when an industrial camera acquires an image, there are concepts of frames and lines. When transmitting a frame of image, there will first be a frame valid (FV). During the frame valid period, the entire image data will be output. After an image is output, there will be a frame blanking time to switch to the output of the next frame of image data, and there is no valid data during this period. A frame of image is composed of several lines of data. When each line is output, there will be a line valid (LV) signal. After a line of image is output, there will be a line blanking time to switch to the output of the next line of data. When collecting each frame of data, when the rising edge of the field valid (FV) arrives, the line signal LV is valid, the start address is accumulated, and data is sent until the falling edge of the field signal FV arrives, and the control DMA ends the data transfer, completing the collection of a frame of data, and returning to the initial position of the state machine to wait for the arrival of the next frame of data.

[0050] The general acquisition modes of industrial cameras are divided into single-frame acquisition mode and continuous acquisition. In the continuous acquisition mode, after starting the acquisition, image output continues until a stop acquisition signal is received. After starting the acquisition in the single-frame acquisition mode, only one image is acquired and then the acquisition automatically stops. To acquire the next frame of image, the next start acquisition signal needs to be sent again.

[0051] Regardless of the acquisition mode:

[0052] When the firmware (CPU of CX3) receives the opening of the stream channel, the GPIF state machine returns to the initial state.

[0053] When the firmware receives the command to close the stream channel, it immediately stops sensor acquisition, halts the state machine, and resets the DMA channel.

[0054] When the firmware receives the command to start acquisition, it first needs to obtain a buffer from the DMA channel of the CX3 chip. According to the U3V protocol format, it generates the data header packet for the current frame respectively. After sending the header packet, it sets the sensor to start acquisition and initiates DMA transmission.

[0055] When the firmware receives the command to stop acquisition, it waits for the current frame transmission to complete, sets the sensor to stop acquisition, obtains a buffer from the DMA channel of the CX3 chip, and generates the data tail packet for the current frame respectively according to the U3V protocol format.

[0056] Continuous acquisition mode:

[0057] When transmitting image data and reaching the last buffer, the firmware (CPU) first modifies the mode of the DMA channel to the override mode, obtains a buffer from the DMA channel of the CX3 chip, generates the data tail packet for the current frame and the data header packet for the next frame respectively according to the U3V protocol format, and sends them out. After sending, it resets the mode of the DMA channel to the normal mode; repeat this process until the command to stop acquisition is received.

[0058] Single-frame acquisition mode:

[0059] When setting to start acquisition, the firmware first obtains a buffer from the DMA channel of the CX3 chip, generates a data header packet according to the U3V protocol, stores the header packet in the obtained buffer, and sends it out. It sets the sensor to start acquisition, initiates DMA, and transmits image data. When reaching the last buffer, it sets the sensor to stop acquisition, obtains a buffer from the DMA channel of the CX3 chip, and generates the data tail packet for the current frame respectively according to the U3V protocol format. When sending the start acquisition command again, repeat the above steps. In this way, the process of this acquisition is completed, and it waits for the next image acquisition command.

[0060] It should be noted that for the DMA channel of the CX3 chip to enter the override mode, the DMA channel needs to be reset to the configured state. This solution can ensure stable acquisition.

[0061] Since the camera of the present invention is based on the CX3 architecture, there is no FPGA chip and DDR as the first-level cache. In order to ensure the efficiency of transmission and the buffer transmission caused by bus congestion, the buffer transmission is blocked. In the preferred implementation, the channel is configured with two threads. By switching between the two threads alternately, the buffer switching time in a single thread can be reduced. Each thread must have at least 4 DMA buffers. By configuring multiple buffers, the cache capacity can be increased. The buffer size should be as large as possible to reduce the number of DMA transfers, such as Figure 3 shown.

[0062] Abnormal collection solution

[0063] The normal image transmission process is: send header packet -> send data packet -> send tail packet. However, during image transmission, the bus may be blocked, or the receive data buffer may not be removed in time due to CPU occupancy, resulting in data loss and subsequent acquisition failure. Therefore, the following design solutions are required to address these anomalies:

[0064] The overall principle is that frame IDs can jump when congestion occurs, but each frame must complete normally, impacting as few frames as possible. For any frame, if transmission congestion is detected, the remaining data in the currently acquired frame is replaced with a short packet (or a zero-length packet) and a tail packet. This ensures that the end normally completes a frame and begins the next one. The data size of a short packet is much smaller than that of a normal data packet, for example, set to 4 bytes.

[0065] The condition for whether a short packet is needed is: whether the amount of data sent is equal to the amount of data requested by the upper-layer driver. If it is equal, there is no need to supplement it with a short packet. If it is not enough, enough short packets must be added, otherwise the tail packet will be regarded as a data packet, resulting in data confusion.

[0066] Note: The number of short packets required is calculated based on the U3V protocol. The difference between the number of data packets required to transmit a frame of image and the actual number of data packets sent is the number of data packets that need to be resent.

[0067] If the size of a frame of data is 2M+32*1024+576, it is divided into 1M data packets. If it is less than 1M, it is divided into 1024 bytes data packets. Each data block needs to be ended with a short packet.

[0068] If the sent data is less than 1M, 35 short packets need to be sent;

[0069] If 2M sent > data > = 1M, 34 short packets need to be sent;

[0070] · If the data already sent satisfies 2M + 32 * 1024 > data >= 2M, then a short packet needs to be sent for every 1024 bytes, and a short packet needs to be sent for the last 576 bytes.

[0071] · If the data already sent satisfies 2M + 32 * 1024 + 576 > data >= 2M + 32 * 1024, then 1 short packet needs to be sent.

[0072] A key point of the exception handling method of the present invention is that the exception handling process must be entered when there are several remaining buffers. In this embodiment, a total of 8 DMA buffers are opened. If there is a blockage, that is, when the remaining buffers are less than 7, it will be processed, which will cause many short-term blockages, and situations that could have been restored by using cached buffers will be wasted. The cache should be judged as blocked when there are fewer remaining buffers to maximize the role of the cache. Since the production speed is affected by the sensor acquisition speed, and the consumption is only affected by the transmission speed, the following exception handling scheme needs to be given by comprehensive consideration:

[0073] Producer callback buffer processing flow:

[0074] When there is a blockage on the bus, the value of Productcnt (representing the inventory data volume value in CX3) will become larger and larger. Taking 8 buffers as an example, when all buffers are empty, the value of Productcnt is 1. When there is only the last buffer left, that is, when Productcnt is equal to 8, exception handling is required:

[0075] Case 1: If the first packet arrives at the last buffer, it is directly discarded and the acquisition is stopped. The stop of acquisition here is an immediate stop, without ensuring a complete frame. When Productcnt becomes 1, the acquisition is restarted. By starting and stopping the acquisition, the exception can be quickly restored, so there is no need to wait for a complete frame of data to be sent before recovery.

[0076] Case 2: If the last packet or short packet arrives at the last buffer, it is sent normally.

[0077] Case 3: If a data packet arrives at the last buffer, it is necessary to calculate how many short packets are needed for the current frame, and start sending short packets from the current buffer to ensure the normal end of the current frame. After the short packets are sent, the last packet is directly sent.

[0078] Whether to immediately stop and restart the acquisition for recovery needs to be judged according to the number of remaining buffers. The specific number of buffers needs to be measured actually, and there will be differences in this time for different sensors.

[0079] t1: Time for stopping and restarting the acquisition for recovery

[0080] t2: The time interval between two entries into the producer callback × the remaining number of buffers

[0081] When t1 is greater than t2, immediately stop sampling and restart sampling to resume; otherwise, directly discard the data packet and wait for the current frame to complete. When the last buffer arrives, the header packet of the next frame can be sent, and the subsequent buffer sending resumes normally.

[0082] When trigger sampling is required, for sensors that do not support restart, keep the sensor in continuous sampling state. Trigger sampling can be implemented as follows:

[0083] When the trigger mode is turned on (from off to on), if the sensor is in the sampling state, after the last image data buffer of the current frame is sent out, all the data of this frame has been delivered to the DMA channel and sent. Set the sensor to stop sampling, configure the DMA channel to enter the override mode, send the tail packet, restore the DMA channel to the normal mode, then change the trigger mode recorded in the firmware to on, return the command success, and then wait for the trigger signal or trigger command to arrive; if it is in the stop sampling state, directly change the trigger mode recorded in the firmware to on and return the command success.

[0084] When the trigger mode is turned off (from on to off), if the sensor is in the start sampling state, after the last image data buffer of the current frame is sent out, all the data of this frame has been sent through the DMA channel. Do not stop the sensor sampling, configure the DMA channel to enter the override mode, send the tail packet, restore the DMA channel to the normal mode, then change the trigger mode recorded in the firmware to off, and return the command success. If it is in the stop sampling state, directly change the trigger mode recorded in the firmware to off and return the command success.

[0085] When in the trigger mode of on and the sampling state, when the firmware receives the trigger signal or trigger command, immediately set the sensor to sample, start the DMA transmission. After the last image data buffer of the current frame is sent out, set the sensor to stop sampling, configure the DMA channel to enter the override mode, send the tail packet, restore the DMA channel to the normal mode, and then wait for the next trigger signal to arrive. Before the tail packet is sent out, the received trigger signal will be blocked. This design makes the acquisition time of the obtained image relatively stable, is not limited by the sensor model, and the solution is more general.

[0086] When in the stop sampling state, when the firmware receives the trigger signal or trigger command, the received trigger signal will be directly blocked.

[0087] For the used sensor, if it supports the restart setting, the sensor can immediately stop the current frame acquisition and directly start a new frame acquisition. When the sensor is in the continuous acquisition state, the restart can be directly set after receiving the trigger signal, and at the same time, the state machine can be jumped to the original state to trigger the acquisition. In this way, after a trigger image transmission is completed, do not stop the acquisition again. Just directly restart the next time, which can simplify the logic of the above trigger acquisition scheme.

Claims

1. A camera stable acquisition method based on a CX3 chip, characterized in that, The method includes: (1) Monitoring the image acquisition signal, and configuring the CX3 chip when the image acquisition signal is received, so that the DMA of the CX3 chip enters the override mode; (2) Obtaining a buffer from the DMA channel of the CX3 chip; (3) Generating a data header packet according to the USB3.0 VISION protocol, storing the header packet in the obtained buffer, and sending it outwards; (4) Resetting the DMA channel of the CX3 chip to the normal working mode, receiving image data from the image sensor, and sending it outwards through the DMA channel; (5) Obtaining the size of the image captured by the current sensor, calculating the data volume of each corresponding image based on the image size, and determining the number of buffers required for the image data according to the calculated data volume; (6) Real-time monitoring whether the determined number of buffers required for the image data is reached. When the required number of buffers is reached, the CX3 chip is set to enter the override mode; (7) Obtaining a buffer from the DMA channel of the CX3 chip; generating a data tail packet according to the USB3.0 VISION protocol, storing the tail packet in the obtained buffer, and sending it outwards; (8) Resetting the CX3 chip to the normal working mode.

2. The camera stable acquisition method based on the CX3 chip according to claim 1, characterized in that In the steps (3) and (8), data is sent through the DMA channel.

3. The camera stable acquisition method based on the CX3 chip according to claim 1, wherein The CX3 chip receives image data from the image sensor.

4. The camera stable acquisition method based on the CX3 chip according to claim 1, wherein The CX3 chip transfers the header packet data, image data, and tail packet data to the USB interface.

5. The method for stable camera acquisition based on the CX3 chip according to claim 1, wherein The method further includes judging the acquisition mode of the current camera. If it is the continuous mode, after the step (6), obtaining a buffer from the DMA module of the CX3 chip; Generating the next data header packet according to the USB3.0 VISION protocol, storing the next data header packet in the obtained buffer, and sending it outwards; (8) Resetting the CX3 chip to the normal working mode.

6. The method for stable camera acquisition based on the CX3 chip according to claim 1, wherein The method further includes obtaining the size of the image captured by the current image sensor from the parameter storage module of the camera.

7. The camera stable acquisition method based on the CX3 chip according to claim 1, wherein, The method further includes configuring the operation of the CX3 chip based on the Api interface.

8. The camera stable acquisition method based on the CX3 chip according to claim 1, characterized in that The method further includes performing anomaly detection. For any frame of image, when an anomaly is detected during the acquisition of the frame image, for the remaining data in the currently acquired frame image data, the remaining data is replaced with a short packet or a zero-length packet and a tail packet.

9. A camera stable acquisition device based on a CX3 chip, characterized in that, The camera stable acquisition device includes a CX3 chip. The input end of the CX3 chip is communicatively connected to the sensor of the camera, the output end is communicatively connected to the USB output port, and the firmware of the CX3 chip is communicatively connected to an external control device. When the camera stable acquisition device receives an image acquisition instruction, it performs image acquisition and output according to the method described in any one of claims 1-8.

Images