Embedded image-assisted optical fiber color detection system and method
By integrating the FPGA and ARM dual-core processor of ZYNQ chip in the fiber color detection system, combining internal bus communication and hardware accelerator, the communication instability and high cost caused by separate design is solved, and low-cost and high-performance embedded fiber color detection is achieved.
Patent Information
- Application Number
- CN202510675898.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-23
- Publication Date
- 2025-08-08
AI Technical Summary
The existing fiber color detection system adopts a separate design, which leads to unstable communication and high cost, making it difficult to meet the needs of low cost and high performance.
The ZYNQ chip integrates FPGA and ARM dual-core processors, and combines the FPGA hardware accelerator and ARM's powerful process processing capabilities to realize embedded image-assisted detection of the fiber color detection system, integrates fiber acquisition and visual assistance units, and uses CMOS chips for image acquisition and HLS image algorithm acceleration.
It improves the performance and stability of the system, reduces hardware costs, realizes efficient data interaction and accurate LCD screen color detection, and is suitable for automated testing and embedded systems.
Smart Images
Figure CN120448322A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of embedded automated testing, electronic measurement and embedded systems, and specifically relates to an embedded image-assisted optical fiber color detection system and method. Background Art
[0002] Measurement and control technology underpins advanced manufacturing. In the field of automated testing, LCD color detection and storage of historical test data are crucial. This embedded image-assisted fiber optic color detection system and method can control 20 IIC color sensors for automated testing. The system integrates a ZYNQ main control chip, an FPGA, and an ARM chip. The FPGA uses the xc7z010clg400-1 chip, while the embedded system utilizes the ARM CONTEX A9 dual-core chip. This system offers strong real-time communication, stable performance, and low cost.
[0003] Currently, most products on the market use a single-chip microcomputer as a color detection tool, and the single-chip microcomputer as the user operation end. This separate design uses a network cable, SDIO, or serial port to communicate between the two boards. This separate design does not conform to the development direction of low cost and high performance. The FPGA and ARM in the ZYNQ chip use an internal bus for communication, fully leveraging the advantages of the hardware accelerator on the FPGA side and the powerful process processing advantages of the ARM side. The dual-core ARM side runs VITIS bare metal, making data interaction more convenient and stable.
[0004] Leverage the advantages of CMOS high-speed acquisition and HLS image algorithm acceleration on the FPGA side for embedded automated testing. This fusion automated detection system and method, supplemented by visual assistance and primarily based on fiber optic detection, makes fiber optic color detection products more competitive. Summary of the Invention
[0005] To optimize the functionality of existing fiber color detection systems, this paper proposes an embedded image-assisted fiber color detection system and method. This system and method not only detects fiber color but also assists with other visual functions. Furthermore, the ARM and FPGA are integrated into a single chip, improving development efficiency and reducing costs.
[0006] The present invention provides an embedded image-assisted optical fiber color detection system and method. The color detection system includes an FPGA data acquisition system and an ARM data processing system.
[0007] In an embedded image-assisted fiber color detection system and method of the present invention, the FPGA data acquisition system includes a fiber acquisition unit and a visual assistance unit; the ARM processor is responsible for running the operating system, user interface and data processing algorithm, while the FPGA logic unit is used for high-speed parallel processing of data transmitted by the color sensor module.
[0008] The FPGA data acquisition system includes a fiber optic acquisition unit and a visual assistance unit;
[0009] The fiber optic acquisition unit in the FPGA data acquisition system includes a 20-channel color acquisition sensor, a 20-channel input IO module, an HDMI display module, a TTL IO output module, a uartlittle serial port module, and a uart16550 module.
[0010] In the FPGA data acquisition system, the fiber optic acquisition unit includes the uart16550 module connected through the AXI_GP1 bus; the uartlittle serial port connected through the AXI_GP2 bus; the TTL IO output module connected through the AXI_GP3 bus; the HDMI module connected through the DMA+AXI_ACP internal bus; and the 20-channel color acquisition sensor connected through the internal bus AXI_IIC.
[0011] The visual assistance unit in the FPGA data acquisition system includes a CMOS chip for data image acquisition. The data interface between the FPGA data acquisition system and the CMOS chip uses LVDS parallel automatic black level calibration and supports programmable control: gain, horizontal and vertical blanking, frame size / rate, exposure, left and right and up and down image inversion, window size and translation.
[0012] In the visual assistance unit of the FPGA data acquisition system, the FPGA acquires the CMOS image sensor through the Verilog method, and then converts the image format through LVDS to RGBA and transmits it to AXIDMA through the AXI STREAM internal bus. AXIDMA uses the internal bus AXI STREAM to move it to HLS for color image algorithm processing, and realizes hardware acceleration through the hls::cv library. The HLS output result data is sent to the CPU1 image processing unit through the AXI HP bus.
[0013] In the visual assistance unit of the FPGA data acquisition system, the camera control command is transmitted to the visual assistance unit via the AXI_GP bus.
[0014] In an embedded image-assisted optical fiber color detection system and method of the present invention, the ARM data processing system includes a CPU0 data processing unit and a CPU1 image processing unit;
[0015] The CPU0 data processing unit runs the VITIS bare metal and is mainly responsible for processing internal and external interface data and processing and outputting color sensor data;
[0016] CPU0 data processing unit, including color sensor module, EEPROM storage module, data fusion module and interface communication module;
[0017] The color sensor module in the CPU0 data processing unit is used to capture the color information on the LCD display module and convert the captured color information into electrical signals. It converts and processes the sensor data transmitted from the internal interface and outputs the color sensor data through the interface communication module.
[0018] The EEPROM storage module in the CPU0 data processing unit is used to store color calibration data, detection algorithms, and historical detection results to ensure system stability and traceability.
[0019] The interface communication module in the CPU0 data processing unit enables data transmission between the system and external devices, facilitating further data analysis and processing. It includes USB storage device input, TTL output, 232 output, 485 output, Ethernet output, and serial debug output. TTL output, 232 output, 485 output, and Ethernet output are user-selectable output interfaces.
[0020] The CPU0 data processing unit uses the AXI_IIC interface to control the 20-channel color sensor inputs in the fiber optic acquisition unit. AXI_GP1, AXI_GP2, and AXI_GP3 connect to the UART16550 IP core, the UARTlittle IP core, and TTL I / O, respectively. The AXI_GPIO interface controls communication with the 20-channel I / O input module. The AXI_ACP and DMA interfaces control the HDMI display in the fiber optic acquisition unit.
[0021] The data fusion module in the CPU0 data processing unit generates color detection results from the color sensor and image data, respectively. These detection results are then mapped to coordinates to form pseudo-image data. The pseudo-image's channels correspond to different object detection outputs, such as object center, size, and category. After comparing the pseudo-image's original RGB image with the color sensor values, at each location in the corresponding feature map, if detections from both the color sensor and image data are present, the color at that location is considered a perfect match. Otherwise, if there is no match at that location, the network generates a feature description vector for each detection result from the color sensor and image data. This feature vector is used to calculate the similarity between the detection results and, as a basis for matching.
[0022] The communication between the CPU0 data processing unit and the CPU1 image processing unit is carried out by means of the chip's internal OCM plus soft interrupt notification. The data that the CPU0 data processing unit needs to transmit to the CPU1 image processing unit includes the control command parameters for controlling the CMOS chip frame rate, pixels, zoom, and focus.
[0023] CPU1 is an image processing unit that runs the VITIS bare metal system, including the image colorimetry detection and calculation module and the calculation result data handling module.
[0024] The CPU1 image processing unit is mainly responsible for the color data transmitted from the visual assistance unit in the FPGA data acquisition system. The color data passes through the image chromaticity detection calculation module and the calculation result data transfer module respectively. Specifically, the image data is transferred to the CPU0 data processing unit through the inter-core soft interrupt method and OCM method inside the chip.
[0025] The image chromaticity detection and calculation module in the CPU1 image processing unit first detects the color value of each pixel corresponding to the LCD resolution, and then identifies the coordinates of the corresponding pixel based on the LCD position where the optical fiber contacts it. It then generates the color values of these 20 points individually and packages them into 20 coordinates in the (X, Y, Z) format, where X and Y are the coordinates of the pixels corresponding to the 20 optical fiber detection positions, and Z is the color value corresponding to this coordinate.
[0026] The calculation result data handling module in the CPU1 image processing unit packages the image detection results and transmits them to the CPU0 data processing unit through OCM and inter-core soft interrupts.
[0027] There are no special requirements for HDMI output. It can be connected to the user's monitor with an HDMI interface through an external HDMI monitor, and the HDMI interface hardware circuit pin constraints are added to the FPGA data acquisition system.
[0028] The ARM data processing system includes the CPU0 data processing unit and the CPU1 image processing unit; the CPU0 data processing unit is mainly responsible for the output of internal and external interface data; the CPU1 image processing unit is mainly responsible for assisting in the processing of the LCD pixel value processing algorithm transmitted from the visual assistance unit in the FPGA data acquisition system.
[0029] The advantages of the present invention are
[0030] This dual-core processor architecture combines the xc7z010clg400-1 FPGA chip with an ARM dual-core chip, lowering hardware costs and reducing chip circuit design size. By combining the flexibility of the ARM processor with the high-speed parallel processing capabilities of the FPGA logic unit, it improves overall system performance and stability. By integrating the ARM data processing system and the FPGA data acquisition system, this system leverages the advantages of both system process processing and FPGA high-speed data acquisition, avoiding inaccurate system data exchange caused by external assembly of multiple boards or communication transmission interference. This system has broad market prospects in automated testing and embedded systems technology.
[0031] By integrating the color sensor module and the visual assistance unit with the data fusion module, accurate detection of LCD screen color is achieved, providing strong support for color quality control. The system integrates a storage module and an interface communication module to facilitate data storage and transmission, improving the practicality and convenience of the system.
[0032] The user interaction module provides an intuitive and easy-to-use user interface, reducing operational difficulty and improving user experience. In summary, the embedded image-assisted color detection system and method of the present invention has the advantages of high performance, good stability, and simple operation, and is suitable for the field of LCD screen color quality detection and control. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 Shown is a principle block diagram of an embedded image-assisted optical fiber detection system and method.
[0034] Figure 2 The embedded image-assisted fiber optic inspection system and method architecture is shown.
[0035] Figure 3 A schematic diagram of an embedded image-assisted fiber optic interruption detection system and method is shown.
[0036] Figure 4 Shown is an embedded image-assisted optical fiber detection system and method flow chart. DETAILED DESCRIPTION
[0037] Figure 1The figure shows a functional block diagram of an embedded image-assisted fiber optic inspection system and method. The ARM data processing system includes a CPU0 data processing unit and a CPU1 image processing unit. The CPU0 data processing unit is primarily responsible for data processing from the color sensor module, EEPROM storage module, and data fusion functions. The CPU1 image processing unit is responsible for transferring high-definition images transmitted from the visual assistance unit in the FPGA data acquisition system to CPU0. The FPGA data acquisition system includes a fiber acquisition unit and a visual assistance unit. The fiber acquisition unit is responsible for collecting data from 20 image sensors. The visual assistance unit is responsible for image acquisition from the CMOS image sensor and processing the HLS IP core algorithm. These images are then transferred to the CPU1 image processing unit in the ARM data processing system via DMA.
[0038] Figure 2 As shown, the architecture of the embedded image-assisted fiber optic inspection system and method is shown. The hardware driver layer includes an embedded low-level driver and a logic low-level driver. The embedded low-level driver is responsible for internal bus drivers, network port drivers, AXI_IIC drivers, HLS IP drivers, DMA IP drivers, and EEPROM storage media drivers. The logic low-level driver is responsible for driving the 20-channel AXI_IIC color sensor, TTL drivers, CMOS chip drivers, RGBA display drivers, 485 drivers, and 232 drivers. At the system level, the CPU0 data processing unit and the CPU1 image processing unit both run the VITIS bare-metal system, using inter-core soft interrupts and OCM for image transmission and CMOS chip control. The user-level application is responsible for interface response, color acquisition and acquisition channel selection, data fusion, data output speed switching and data interface switching output, EEPROM storage and data playback functions, CMOS parameter settings, and HLS image color calculation.
[0039] Figure 3 As shown in the interrupt design diagram of the embedded image-assisted fiber optic inspection system and method, the axi_uartlittle, axi_uart16550, and TTL_IO interfaces in the communication module use the IP core's built-in interrupts to notify the CPU0 data processing unit. The fiber acquisition unit uses the AXI_IIC built-in IP core interrupt to notify. The UIO module notifies the CPU1 image unit via a shared interrupt. The visual assistance unit uses the DMA IP core's built-in interrupt to notify the CPU1 image unit.
[0040] Figure 4As shown in the figure, system initialization includes initializing the color sensor and image sensor. Data is transmitted via optical fiber to the color sensor, which then converts the three color values into analog-to-digital values via the IIC. The color detection results are then transmitted to the color sensor module in the CPU0 data processing unit via the user AXI_IIC. The CMOS image acquisition system then processes the HLS::CV image pixel data, and the color image chromaticity detection and calculation module transmits the calculation result data to the CPU0 data processing unit via the data handling module. The color sensor module captures the color information from the three color sensors and the CMOS image calculation data for data fusion. Simultaneously, the ARM processor runs the detection algorithm to further analyze the processed data and obtain the color detection results. The color detection data is output via various interfaces, and the host computer displays the color detection results.
[0041] Professionals in the field of automated testing technology and embedded systems can understand that various aspects of the present disclosure can be specifically implemented in the following forms: complete hardware implementation methods, complete software implementation methods, which can be collectively referred to as "units" or "systems" here.
[0042] The above description is only an exclusive implementation mode of the present invention and is not intended to limit the present invention. Any modifications, replacements, improvements, etc. made within the spirit and principles of the present invention should be considered as the scope of protection of the present invention by technicians in this development field without departing from the technical principles of the present invention.
Claims
1. An embedded image-assisted fiber color detection system and method, wherein the ARM data processing system is responsible for running the bare metal system, and the FPGA logic unit is used for high-speed parallel processing of data transmitted by the color sensor module. It is characterized by: The system consists of an FPGA data acquisition system and an ARM data processing system. The FPGA data acquisition system includes a fiber acquisition unit and a visual assistance unit. The ARM data processing system includes a CPU0 data processing unit and a CPU1 image processing unit. The CPU0 data processing unit runs the VITIS bare metal and is primarily responsible for processing internal and external interface data and processing and outputting color sensor data.
2. The embedded image-assisted optical fiber color detection system and method according to claim 1, characterized in that: The fiber optic acquisition unit in the FPGA data acquisition system includes a 20-channel color acquisition sensor, a 20-channel input IO module, an HDMI display module, a TTL IO output module, a uartlittle serial port module, and a uart16550 module.
3. The embedded image-assisted optical fiber color detection system and method according to claim 1, characterized in that: The visual assistance unit in the FPGA data acquisition system uses a CMOS chip for data and image acquisition. It employs LVDS parallel automatic black level calibration. Communication between the CPU0 data processing unit and the CPU1 image processing unit is accomplished through the chip's internal OCM and soft interrupt notifications. The data that the CPU0 data processing unit needs to transmit to the CPU1 image processing unit includes control command parameters for the CMOS chip's frame rate, pixel count, zoom, and focus.
4. The embedded image-assisted optical fiber color detection system and method according to claim 1, characterized in that: CPU0 data processing unit, including color sensor module, EEPROM storage module, data fusion module and interface communication module.
5. The embedded image-assisted optical fiber color detection system and method according to claim 1, characterized in that: The data fusion module in the CPU0 data processing unit generates color detection results from the color sensor and image data, respectively. These detection results are then mapped to coordinates to form pseudo-image data. The pseudo-image's channels correspond to different object detection outputs, such as object center, size, and category. After comparing the pseudo-image's original RGB image with the color sensor values, at each location in the corresponding feature map, if detections from both the color sensor and image data are present, the color at that location is considered a perfect match. Otherwise, if there is no match at that location, the network generates a feature description vector for each detection result from the color sensor and image data. This feature vector is used to calculate the similarity between the detection results and, as a basis for matching.
6. The embedded image-assisted optical fiber color detection system and method according to claim 1, characterized in that: The image chromaticity detection and calculation module in the CPU1 image processing unit first detects the color value of each pixel corresponding to the LCD resolution, and then identifies the coordinates of the corresponding pixel based on the LCD position where the optical fiber contacts it. It then generates the color values of these 20 points individually and packages them into 20 coordinates in the (X, Y, Z) format, where X and Y are the coordinates of the pixels corresponding to the 20 optical fiber detection positions, and Z is the color value corresponding to this coordinate.
7. The embedded image-assisted optical fiber color detection system and method according to claim 1, characterized in that: The CPU1 image processing unit is mainly responsible for the color data transmitted from the visual assistance unit in the FPGA data acquisition system. The color data passes through the image chromaticity detection calculation module and the calculation result data transfer module respectively. Specifically, the image data is transferred to the CPU0 data processing unit through the inter-core soft interrupt method and OCM method inside the chip.
8. The embedded image-assisted optical fiber color detection system and method architecture according to claim 1, characterized in that: The hardware driver layer includes embedded low-level drivers and logic low-level drivers. The embedded low-level drivers are responsible for internal bus drivers, network port drivers, AXI_IIC drivers, HLS IP drivers, DMA IP drivers, and EEPROM storage media drivers. The logic low-level drivers are responsible for driving the 20-channel AXI_IIC color sensor, TTL drivers, CMOS chip drivers, RGBA display drivers, 485 drivers, and 232 drivers. At the system level, the CPU0 data processing unit and CPU1 image processing unit both run the VITIS bare-metal system, using inter-core soft interrupts and OCM for image transmission and CMOS chip control. The user-level application is responsible for interface response, color acquisition and acquisition channel selection, data fusion, data output speed switching and data interface switching output, EEPROM storage and data playback functions, CMOS parameter settings, and HLS image color calculation.