A method for distinguishing and labeling multiple foreground objects in image data streams

Abstract

This invention discloses a method for distinguishing and marking multiple foreground objects in image data streams. The steps include: 1) connecting a camera to an FPGA, whereby the camera transmits the acquired fiber optic unit array image to the FPGA line-by-line in real-time in a streaming manner; 2) the FPGA detects the received image pixels, identifies and marks the light spot pixels; 3) the FPGA calculates the optical centroid coordinates of the corresponding light spot pixels based on the marked light spot pixels. This invention can process the image data stream in real-time at the image acquisition end, distinguishing and marking each fiber optic light spot pixel. This allows for real-time calculation of the light spot centroid at the image acquisition end while maintaining accuracy, thereby greatly reducing data transmission volume and enabling the host computer to obtain the optical centroid coordinates of the fiber optic unit in real-time. This provides an accurate and real-time labeled image data stream for subsequent optical centroid calculation, thus improving overall positioning efficiency.

Description

Technical Field

[0001] This invention belongs to the field of multi-object fiber optic spectroscopic telescope technology, and relates to real-time fiber optic unit localization of multi-object fiber optic spectroscopic telescopes (represented by LAMOST telescope). Specifically, it relates to a method for distinguishing and marking multiple foreground objects in image data streams, which is used to achieve rapid distinction and marking of each fiber spot during fiber optic unit localization. Background Technology

[0002] Multi-object fiber optic spectroscopic telescopes (with LAMOST as a typical example) can simultaneously acquire massive amounts of astronomical spectral data by deploying thousands of fiber optic units on the focal plane, making them core equipment for astronomical sky surveys. The positioning accuracy (requiring sub-pixel level) and response speed (requiring millisecond-level updates) of the fiber optic units directly determine the quality of the spectral data.

[0003] Currently, the positioning of fiber optic units in the LAMOST telescope primarily relies on a direct positioning method based on visual recognition. The typical process involves: capturing images of the fiber optic unit array using a high-resolution camera; transmitting the complete image data to a host computer via a transmission link; preprocessing the received image on the host computer; traversing the image pixels; searching for light spot pixels using an eight-neighbor detection method; and calculating the fiber optic centroid coordinates using the centroid method. However, this method has significant limitations: firstly, the processing flow cannot begin until the complete image data is transmitted to the host computer, and the large amount of data during transmission (up to hundreds of megabytes per frame) and link latency directly lead to a positioning response delay (typically on the order of seconds); secondly, the host computer needs to process image data from a large-scale fiber optic array, and the pixel traversal, noise balance, global threshold segmentation, and light spot pixel data extraction and differentiation operations involved in calculating the fiber optic centroid coordinates require multiple image traversals, further increasing processing time and making it difficult to meet the millisecond-level real-time update requirements. Furthermore, factors such as signal attenuation in the transmission link and multi-task scheduling conflicts on the host computer can also lead to decreased stability of the positioning results and prevent real-time positioning of the fiber optic units, easily resulting in positioning errors due to positioning delays.

[0004] Therefore, existing positioning methods have significant limitations, and there is an urgent need for a new method that can break through the limitations of the "complete image transmission - centralized processing by host computer" mode to achieve high-precision, low-latency positioning of fiber optic units, thereby improving the observation efficiency and data reliability of multi-object fiber optic spectroscopic telescopes. Summary of the Invention

[0005] This invention aims to address the problems of high positioning delay, insufficient real-time performance, and heavy data transmission pressure caused by the existing LAMOST telescope fiber optic unit positioning method, which adopts the mode of "transmitting the complete image to the host computer before searching for fiber optic unit spots and calculating optical centroids". It provides a multi-foreground object differentiation and labeling method oriented towards image data streams. This method can process image data streams in real time at the image acquisition end, differentiating and labeling each fiber optic spot pixel. This allows for real-time calculation of the spot's optical centroid at the image acquisition end while maintaining accuracy, thereby significantly reducing data transmission volume and enabling the host computer to obtain the optical centroid coordinates of the fiber optic unit in real time. This provides accurate and real-time labeled image data streams for subsequent optical centroid calculations, thus improving overall positioning efficiency.

[0006] To achieve the above objectives, the present invention provides the following technical solution: This invention integrates real-time processing logic on the FPGA / ZYNQ platform to achieve parallel processing of image data reception, spot differentiation and marking, and fiber optic centroid positioning. The specific steps of this invention's multi-foreground object differentiation and marking method for image data streams are as follows.

[0007] 1. Real-time image data acquisition and streaming reception The camera is connected to the FPGA via a Camera Link cable. The high-definition camera captures images from the fiber optic unit array, and the image data output by the camera is transmitted to the FPGA line by line in real time in a streaming manner, without waiting for the complete image to be generated, thus realizing the instant start of data reception and processing.

[0008] 2. Differentiation and marking of each fiber optic spot Based on streaming pixel data detection, a "suspect" mechanism (marking adjacent pixels within a preset range) and the cooperation of various modules are used to achieve real-time differentiation and marking of spot pixels, ensuring that densely distributed spots are not misjudged. The specific method is as follows: a) Spot pixel detection: Compare the pixel grayscale values ​​in the received image data with the previously detected pixel grayscale values. Global threshold calculated by the method ( If the grayscale value exceeds the threshold, then... If the pixel is a candidate spot pixel, then it is considered as such. The average pixel grayscale value of the image is obtained by summing and averaging the grayscale values ​​of each pixel in the previous consecutive frames. The overall standard deviation of the image is the average of the pixel gray values ​​of multiple consecutive frames preceding the current image. b) New spot detection: If a candidate spot pixel is unlabeled and not listed as a "suspect" (referring to the pixel area to be detected that may belong to the current spot), it is determined to be the first pixel of the first row of a certain spot. It is assigned a unique label that increases in the detection order. Starting from the first pixel of the first row of the new spot, the pixels within the preset range of Class A (the preset range of Class A refers to several pixels to the right of the current pixel horizontally and several pixels to the left of the first row below vertically, the specific value represented by "several" is determined according to the size and shape of the spot in the actual application) are listed as "suspects", and the coordinates and labels of each pixel of the "suspects" are recorded. c) Detection of the first pixel of a non-first row of a spot: If a candidate spot pixel has been listed as a "suspect" and the label retention logic is not enabled, it is determined to be the first pixel of a row that is not the first row of the spot. It is assigned the same label as the spot to which it belongs, the label retention logic is enabled, and the pixels within the preset range of Class A are listed as "suspects" starting from the first pixel of the non-first row of the spot detected at this time. The coordinates and labels of each pixel of the newly added "suspects" are recorded. d) Non-spot first pixel detection per row: If a candidate spot pixel has been listed as a "suspect" and the label retention logic is enabled, it is determined that the spot is not the first pixel of each row, and it is assigned the same label as the spot to which it belongs. The adjacent pixels within the B-type preset range (the B-type preset range refers to several pixels to the right of the current pixel horizontally and one row of pixels below it vertically, and the specific value represented by "several" is determined according to the size and shape of the spot in the actual application) are listed as "suspects", and the coordinates and labels of each newly added "suspect" pixel are recorded. e) Label Generation and Judgment: When a spot pixel is detected, if the spot pixel is unlabeled and not listed as a "suspect", it is assigned a unique label that increments sequentially according to the detection order. If the spot pixel has already been listed as a "suspect", it is assigned the same label as the corresponding spot. After assigning the corresponding label, the label retention logic is activated. When the label retention logic detects that the grayscale values ​​of a consecutive preset number of pixels (1-10 pixels, specifically determined based on the geometry and spatial distribution of the spot in the image) do not exceed a threshold, the system continues to maintain the pixel count. When the label preservation logic is closed, the labeling of the current row of the spot is terminated.

[0009] 3. Parallel computation of optical fiber centroid For the marked light spot pixels, the coordinates are calculated in real time according to the optical centroid formula. The result is output immediately after the calculation of a single light spot is completed, without waiting for the entire frame to be processed.

[0010] 4. Location results are output in real time. The optical centroid coordinates (byte-level data) are transmitted to the host computer via a gigabit Ethernet communication module for use in the position calibration and drive adjustment of the fiber optic unit.

[0011] A real-time fiber optic unit positioning method is characterized in that the optical centroid coordinates of each light spot are obtained according to the above method and transmitted to a host computer in real time, and the host computer calibrates or adjusts the position of the corresponding fiber optic unit according to the optical centroid coordinates of each light spot.

[0012] The core innovation of this method lies in: constructing a pipeline and parallel processing architecture using FPGA / ZYNQ to perform real-time streaming processing of the image data stream output by the camera (without waiting for the complete image to be generated), breaking through the limitations of traditional "post-processing after complete image transmission"; pioneering a collaborative strategy of "suspect object" mechanism and "label preservation logic"—the "suspect object" mechanism uses the currently detected spot pixel as the center and dynamically marks adjacent pixels in a preset range horizontally to the right and vertically downward as the detection area (avoiding missed detection or misjudgment of dense spots as the same spot), and the "label preservation logic" automatically shuts down the logic when a preset number of non-threshold pixels are continuously detected by tracking the pixels in the spot row in real time (precisely terminating the current row marking and solving the marking confusion caused by the blurry spot boundary). The two work together to achieve unambiguous differentiation and independent marking of densely distributed spots, generating an image data stream with a unique label.

[0013] The significant features of this method include: the differentiation and marking process is independent of image resolution and the number of light spots, and can be adapted to fiber arrays of any size; the processing rate is synchronized with the camera data output rate in real time, enabling instant marking of light spots at full frame rate and full resolution; the marking result (labeled image data stream) directly provides accurate input for subsequent optical centroid calculation, eliminating the time-consuming step of "complete image transmission + multiple pixel traversals" in traditional methods, indirectly reducing the optical centroid acquisition delay from the second level to the millisecond level, and reducing the data transmission volume by 99.9% (only the marked data needs to be transmitted instead of the complete image).

[0014] This invention provides core data support for real-time fiber optic positioning of multi-object fiber optic spectroscopic telescopes through efficient spot differentiation and marking. The marking results play a decisive role in accelerating the calculation of optical centroid and improving the real-time performance of the system. At the same time, it lays a key technical foundation for the development of fiber optic unit anti-collision systems.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The positioning latency is significantly reduced: By adopting the streaming processing mode of "receiving image data while distinguishing and marking light spot pixels", the waiting time of complete image transmission and centralized processing by the host computer is avoided, and the positioning response speed is improved from the second level to the millisecond level (experimental verification shows that it is within 20 milliseconds), which meets the real-time positioning requirements of the optical centroid of the fiber unit. Data transmission pressure is reduced by 99.9%: The data acquisition and processing end only outputs the optical centroid coordinates (byte-level data, 16 bytes per spot, including the optical centroid coordinates, number of spots, and maximum brightness of the spot), rather than the complete image (hundreds of megabytes, such as a 65-megapixel image of about 200MB), which significantly reduces the data load of the transmission link and reduces signal attenuation and transmission errors; Improved positioning stability: Preprocessing and calculation are completed at the image acquisition end, avoiding interference from multi-task scheduling conflicts on the host computer on positioning, and hardware-level acceleration ensures the consistency of optical centroid calculation. Adaptable to large-scale arrays: The method of this invention processes the data stream regardless of the number of fiber optic units in the image, and can simultaneously process the positioning of any number of fiber optic units, adapting to the multi-target observation scenarios of multi-target fiber optic spectroscopic telescopes. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the physical structure of the LAMOST intermediate test system.

[0017] Figure 2 This is a flowchart of an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the back illumination method.

[0019] Figure 4 This is a back-illuminated view of the LAMOST intermediate test system.

[0020] Figure 5 This is a block diagram of the internal structure of a module in ZYNQ according to an embodiment of the present invention.

[0021] Figure 6 This is a block diagram of the internal structure of the light spot differentiation and marking module of the present invention.

[0022] Figure 7 This is a back view image taken in an embodiment of the present invention.

[0023] Figure 8 This is a schematic diagram of the fiber optic spot pixel differentiation and marking method in image preprocessing according to the present invention; (a) Marking rules for the first pixel of a spot, (b) Marking rules for spots that are not the first pixel of each row, (c) Marking rules for spots that are not the first pixel of each row, (d) Rules for distinguishing and marking.

[0024] Figure 9 This is a visualization of some fiber optic spot markings in an embodiment of the present invention.

[0025] Figure 10 This is a visualization result of some fiber optic spot markings in an embodiment of the present invention.

[0026] Figure 11 This is a detailed illustration of some optical fiber spot markings before and after in an embodiment of the present invention.

[0027] Figure 12 This is a visualization of the real-time positioning results received by the host computer in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] This embodiment uses a scaled-down focal plane of LAMOST (Large Sky Area Multi-Object Fiber Spectroscopic Telescope) as an example. Figure 1 As shown, the real-time fiber optic spot differentiation and marking method of the present invention is described in detail. Existing LAMOST systems use connected component marking to locate the optical centroid of fibers on the host computer. This method requires first receiving several RAW format grayscale images (each image is approximately 100 MB in size), and then sequentially completing image overlay, mean calculation, threshold generation, and connected component analysis. This data transmission is enormous, not only time-consuming but also difficult to guarantee data stability, and it cannot obtain the optical centroid coordinates of the fiber optic spot in real time.

[0030] The architecture of this invention embodiment is as follows Figure 5 As shown, a pipelined and parallel processing integrated architecture is adopted, using multi-level FIFOs to achieve data buffering and module decoupling, ensuring parallel and collaborative operation of each module. During system operation, raw camera data is directly connected to the fiber optic unit positioning module via the CameraLink interface, establishing a high-speed data path with the internal image overlay module. After receiving the image data, the image overlay module, in conjunction with the storage device read / write module, completes image overlay according to preset requirements, generating accumulated image data and transmitting it to the image averaging module. The image averaging module performs mean calculation according to the preset number of overlay frames to recover the data characteristics of a single frame image. The processed image data then enters the adaptive threshold calculation module, which uses... Law( The grayscale threshold is calculated through two traversals. This threshold remains constant within the current instruction cycle and is used for foreground segmentation processing in subsequent frames.

[0031] Subsequently, in non-overlay mode, the image overlay module continues to receive image data through the Camera Link interface and directly sends it to the spot differentiation and marking module, as shown in the block diagram below. Figure 6As shown, this module distinguishes and marks each light spot according to the method proposed in this invention, and sends the marked image data to the light spot centroid calculation module. This module calculates the centroid coordinates of each light spot according to the centroid method, and the calculated light spot centroid coordinate sequence, its maximum brightness and number of pixels are sent to the top-level control module, packaged by the Ethernet communication subsystem and transmitted to the host computer.

[0032] Figure 2 The flowchart of an embodiment of the present invention is as follows: Step S1: The camera captures images and outputs the data stream line by line. The instruction acquisition module monitors the Ethernet receiving port in real time, analyzes and judges the data type, extracts valid instructions and performs format conversion. After verification as a valid instruction, it is transmitted to the top-level control module, forming a complete instruction interaction closed loop.

[0033] When the system starts, the host computer sends a start command to the real-time processing module via Ethernet. Upon receiving the start command, the top-level control module sends a continuous shooting command to the camera. After receiving the command, the camera continuously captures images of the intermediate test system and streams the pixel data line by line to the FPGA via the Camera Link interface, specifically as follows: Figure 5 As shown.

[0034] Step S2, calculate the global threshold The system adopts a pipelined and parallel processing integrated architecture, and uses multi-level FIFO to achieve data buffering and module decoupling, ensuring that each module runs in parallel and collaboratively.

[0035] The image overlay module monitors the Camera Link interface in real time. When the top-level control module sends a continuous shooting command to the camera, the overlay function of the image overlay module is enabled. This module receives image data and overlays it according to a preset number of frames (e.g., 5 frames). When valid data from the last frame is reached, the overlaid image data is read and accumulated to obtain the final overlay result. When receiving the last frame, the overlaid pixel data is not stored but directly sent to the image averaging module. The image averaging module averages the pixel data according to the preset number of overlaid images to restore the data characteristics of a single frame image.

[0036] The image averaging module starts outputting the averaged image data and stores it in the adaptive threshold calculation module and storage device; the adaptive threshold calculation module is enabled and first calculates the average pixel grayscale value. Then proceed to the second stage to calculate the standard deviation. Finally, the threshold was determined. This threshold serves as the image segmentation threshold during the execution of the current instruction. This threshold remains constant throughout the current instruction cycle and is used for foreground segmentation processing in subsequent frames.

[0037] After obtaining the threshold, the image overlay module stops the image overlay operation and only receives camera image data, which is then sent to the spot differentiation and marking module.

[0038] Step S3, Spot Pixel Differentiation and Marking The principle of back illumination is as follows: Figure 3 As shown, under back-illuminated conditions, after the fiber array elements are excited by the light source, the grayscale distribution of the light spot captured by the camera conforms to the characteristics of a Gaussian function. In actual astronomical observations, precise positioning of over a thousand fiber elements is required. A typical back-illuminated image is shown below. Figure 4 As shown, it contains a densely distributed array of fiber optic spots; faster differentiation and extraction of these spots is of great significance for quickly calculating the centroid coordinates of the spots, shortening the LAMOST preprocessing time, and promoting the development of LAMOST fiber unit anti-collision systems.

[0039] Simply mapping the host computer algorithm directly to an FPGA or ZYNQ for hardware acceleration is insufficient to achieve a performance breakthrough. Traditional algorithms require frequent access to global memory to record connected component labels, but FPGA on-chip storage resources are limited, and DDR read / write timing is complex, leading to compatibility issues with the FPGA's storage architecture. Therefore, this invention proposes a spot differentiation and marking method based on the characteristics of the FPGA architecture. Through deep coupling between hardware architecture and algorithm logic (such as a combination of streaming processing and a suspect mechanism), a processing speed improvement of hundreds of times compared to traditional solutions is achieved on the FPGA platform (reduced from 3 seconds / frame to 20 milliseconds / frame), enabling efficient real-time differentiation and marking of any number of fiber optic spots within the same timeframe.

[0040] The internal structure diagram of the light spot differentiation and marking module is as follows: Figure 6 As shown, the principle of marking a single light spot is as follows: Figure 8 As shown, the camera continuously outputs a data stream starting from the first pixel of the first row of the image. The spot differentiation and marking module monitors this data stream in real time. When the grayscale value of a pixel exceeds a preset threshold, that pixel may be a noise pixel or a spot pixel. Since noise and spots are marked in the same way, it does not affect the differentiation and marking logic. Subsequently, noise and spots can be determined by the number of spot pixels. The following only describes the marking process of normal spots.

[0041] Specific detection and labeling mechanisms: First pixel detection of light spot, such as Figure 8 (a) When a pixel marked by a red dot in Figure 8(a) is detected, its grayscale value is compared with a global threshold to determine whether it is a spot pixel. If the pixel's grayscale value exceeds the threshold, it is unlabeled, and it is not listed as a "suspect" (referring to a pixel area that may belong to the current spot), then it is considered the first pixel of a spot, and a unique label is assigned to it, and it is then processed according to... Figure 8 In (a), the pixels within the range shown in ① (5 pixels on the right horizontally) and ② (5 pixels on the left of the first row below vertically) are listed as "suspects". The coordinates of the "suspects" and the current spot number are stored in the FIFO. When the corresponding coordinates of the pixels appear in the subsequent data stream, they are processed as shown in Figure 8(b), Figure 8(c), and Figure 8(d).

[0042] Spot pixel detection that is not the first pixel of each row (as shown in Figure 8(b)). Finish Figure 8 After detecting and marking the first pixel of the spot shown in (a), or after detecting and marking the first pixel of each row of the spot not shown in the first row in Figure 8(c), the following will appear: Figure 8 (b) shows spot pixels that are not the first pixel of each row. If these pixels have been listed as "suspects," their grayscale values ​​are greater than the global threshold, and the label retention logic is enabled, they are determined to be spot pixels that are not the first pixel of each row. They are marked as new "suspects" according to the range rules shown in ③ (horizontally expanding by 5 pixels) and ④ (vertically lowering by 1 pixel), and are assigned the same label as the first pixel of their respective spot. The coordinates and labels of the new "suspects" are recorded in the FIFO. If their grayscale values ​​are less than the global threshold, they are marked as new "suspects" according to the range rules shown in ④, but are not assigned the label of the first pixel of their respective spot. Only the coordinates and labels of the new "suspects" are recorded in the FIFO. When pixels with the corresponding coordinates appear in subsequent data streams, they are processed in the same way as above: if the pixel grayscale value exceeds the threshold, it is assigned the same label as the first pixel of its respective spot; otherwise, it is identified as a background pixel.

[0043] Logic for detecting the first pixel in a non-first row of light spots When a pixel is identified as a "suspect" but the label preservation logic is not enabled, it indicates that the pixel is a "suspect" pixel in each row of a spot that is not in the first row. The grayscale value of this pixel is checked to see if it exceeds the global threshold: if not, it is determined to be background; if so, the label preservation logic is enabled, and the pixel is determined to be the first pixel exceeding the threshold in that row (e.g., ...). Figure 8 (c) As shown by the red dot), assign it the same label as the first pixel of its corresponding spot, and mark it as a "suspect" according to the range rules shown in ⑤ (5 pixels to the right horizontally) and ⑥ (5 pixels to the left of the first row vertically below); all subsequent spot pixels in this row are marked according to... Figure 8 Mark according to the rules shown in (b).

[0044] Tags should maintain logic, such as Figure 8 (d) When inspecting image data streams, it is impossible to predict the coordinates of the last pixel in a row of a light spot. This requires combining the shape and spatial distribution characteristics of the light spot (such as...). Figure 7As shown, this invention sets up a tag-holding logic: when a light spot pixel is detected, the tag-holding logic's count-off logic is reset to zero; when a preset number of pixels (e.g., 5) are detected consecutively without exceeding a threshold, the tag-holding logic is turned off, indicating that the pixels in that row of the light spot have been traversed. The tag-holding logic and the pixel detection logic operate in parallel and cooperate to ensure that, in actual observation scenarios, thousands of fiber optic spots with diverse shapes and spatial distributions in the grayscale images acquired by the camera can be efficiently and accurately distinguished and marked.

[0045] The algorithm processes the first pixel of each row of each spot according to... Figure 8 (a) shows the rule marking, and subsequent pixels are marked according to... Figure 8 (b) Figure 8 (c) Figure 8 (d) illustrates the rules for differentiation and marking, achieving precise positioning, differentiation, and marking of the light spot region through line-by-line scanning. Among these, Figure 8 The widths of ①, ②, ③, ⑤, and ⑥ are set according to the specific scene (as in the example of 5 pixels above). This is to avoid the situation where a single spot is mistakenly marked as two spots due to the pixel intervals of the spot. However, the width should not be too large to avoid misclassifying two spots as the same spot and incorrectly marking them with the same number. Among them, ① is 1-10 pixels on the right side horizontally, ② is 1-10 pixels on the left side of the bottom row vertically, ③ is 1-10 pixels horizontally extended, ④ is 1 pixel in the bottom row vertically, ⑤ is 1-10 pixels on the right side horizontally, and ⑥ is 1-10 pixels on the left side of the bottom row vertically.

[0046] This embodiment uses the Alinx ACZU9EGB development board (equipped with a Xilinx xczu9eg-ffvb1156-2-i ZYNQ chip), and the image acquisition uses an Eco Optoelectronics 65-megapixel industrial camera with an image resolution of 9344×7000. Figure 7 The image shown is the original image taken in practice; to facilitate comparison of the light spot images before and after marking, in Figure 7 Select a local area and zoom in to show the unlabeled spot image (e.g.) Figure 9 (As shown). Figure 9 It contains three typical fiber optic spots, whose spatial distribution and optical characteristics can be directly used for subsequent algorithm verification and performance testing.

[0047] The method of this invention was used to locate the optical centroid of the scene, and the calculated threshold was 387.8153. To demonstrate the marking results, the marked image data was transmitted to a computer for display, and the selected image was compared with... Figure 9 The same area, Figure 9 The marking effect of the three light spots in the middle is as follows Figure 10 As shown, detailed markings and details are as follows Figure 11 As shown. By Figure 11As can be seen, pixels with gray values ​​less than the threshold are not marked as spot pixels, while pixels with gray values ​​greater than the threshold are marked as spot pixels and their gray values ​​are retained.

[0048] Step S4, Calculation of the optical centroid of the fiber optic spot During the spot marking process, the tagged image data is synchronously transmitted to the spot centroid calculation module. This module calculates and statistically analyzes relevant data for each spot based on the tag, including maximum brightness, number of pixels, and centroid coordinates. The formula for calculating the centroid coordinates is as follows: in, The first spot labeled K The coordinates of a pixel. This is the grayscale value of the pixel.

[0049] Through the collaborative work of the spot differentiation and marking module and the spot centroid calculation module, the entire chain of automated processing from spot recognition to feature parameter extraction is realized, providing accurate data support for subsequent fiber unit positioning adjustment and anti-collision control. The FPGA calculation results are shown in Table 1.

[0050] Another original image was acquired within a short time and transmitted to the host computer. After calculating the threshold using the same algorithm, the optical centroid was located in Matlab using the existing LAMOST fiber unit positioning algorithm. The results are shown in Table 1. Considering that air disturbances may cause spot drift and lead to positioning errors, comparative analysis shows that the positioning results of FPGA and Matlab are highly consistent. This fully verifies that the high-speed fiber spot positioning system based on FPGA proposed in this invention has excellent positioning accuracy.

[0051] Table 1 shows the calculation results of some light spots by the real-time processing module and MATLAB in the embodiments of the present invention. Step S5: The light spot related data is sent to the host computer. The Ethernet communication module is used to realize data communication between the FPGA and the host computer. Its block diagram is as follows: Figure 5 As shown, this module integrates ARP verification functionality, which can automatically obtain the MAC address of the target host. In use, the user only needs to configure the IP address, MAC address, and port number on the FPGA side, as well as the IP address and port number of the target host, and the system can automatically establish a communication link.

[0052] The data transmission path uses a FIFO IP core to construct a transmission data buffer in an independent clock domain. This buffer supports bit width conversion and cross-clock domain data transmission. When the amount of data in the buffer meets the Ethernet frame format requirements, the system automatically encapsulates and sends a data frame. The timing and amount of data writing are dynamically adjusted by the upper-layer control logic to achieve a flexible data transmission strategy.

[0053] After the real-time processing module completes the calculation of the spot-related data, it transmits the data to the host computer via Ethernet. Figure 12 This is a visualization of the real-time positioning results received by the host computer during the operation of the unit in the embodiment.

[0054] This embodiment is only used to illustrate the technical solution of the present invention, and is not intended to limit it. Those skilled in the art can modify or make equivalent substitutions to the technical solution of the present invention without departing from the spirit and scope of the invention. The scope of protection of the present invention should be determined by the claims.

Claims

1. A method for distinguishing and labeling multiple foreground objects in an image data stream, comprising the following steps: 1) Connect the camera to the FPGA, and the camera will transmit the acquired fiber optic unit array images to the FPGA in real time in a streaming manner, line by line; 2) The FPGA detects the received image pixels, identifies the light spot pixels, and marks them. The specific method is as follows: a) Compare the pixel grayscale values ​​in the received image data with the current global grayscale threshold. A comparison is made; if the grayscale value of the current pixel exceeds the global threshold... If so, the current pixel is determined to be a candidate spot pixel; b) If a candidate spot pixel is unlabeled and not marked as a "suspect", then the candidate spot pixel is determined to be the first pixel of the first row of the new spot. The candidate spot pixel is assigned a unique label that increases in the detection order. Starting from the first pixel of the first row of the new spot, all pixels within the preset range of Class A are marked as "suspects", and the coordinates and labels of each pixel marked as a "suspect" are recorded. The preset range of Class A refers to several pixels to the right of the current pixel horizontally and several pixels to the left of the first row below vertically. c) If a candidate spot pixel has been marked as a "suspect" and the label retention logic is not enabled, then the candidate spot pixel is determined to be the first pixel of the row that is not the first row of the new spot, and it is assigned the same label as the spot to which it belongs, and the label retention logic is enabled; then, starting from the candidate spot pixel, pixels within the preset range of Class A are marked as "suspects", and the coordinates and labels of each newly marked "suspect" pixel are recorded; d) If a candidate spot pixel has been marked as a "suspect" and the label retention logic is enabled, then the candidate spot pixel is determined to be a non-row-first pixel of the new spot, and it is assigned the same label as the spot to which it belongs. Starting from the candidate spot pixel, pixels within the preset range of Class B are marked as "suspects", and the coordinates and labels of each newly marked "suspect" pixel are recorded. The preset range of Class B refers to several pixels to the right of the current pixel horizontally and one row of pixels below it vertically. e) If a detected spot pixel is unlabeled and not listed as a "suspect", it is assigned a unique label that increments sequentially according to the detection order; if the spot pixel has already been listed as a "suspect", it is assigned the same label as the spot to which it belongs, and the label holding logic is activated. When the label holding logic detects that a preset number of consecutive pixel grayscale values ​​have not exceeded the threshold, the label holding logic continues to hold. When the label retention logic is disabled, the labeling of the current row of the new spot is terminated; 3) The FPGA calculates the centroid coordinates of the corresponding light spot based on the marked light spot pixels.

2. The method according to claim 1, characterized in that, The coordinates are calculated in real time using the optical centroid formula to obtain the optical centroid coordinates of the corresponding light spot.

3. The method according to claim 1, characterized in that, use The global grayscale threshold is calculated using this method. ;in, The average pixel grayscale value of the image is obtained by summing and averaging the grayscale values ​​of each pixel in the previous consecutive frames. The overall standard deviation of the image is the average of the gray values ​​of each pixel in the previous consecutive frames.

4. The method according to claim 1, 2, or 3, characterized in that, The number of consecutive preset quantities is determined based on the geometric shape and spatial distribution of the light spots in the image.

5. The method according to claim 1, 2, or 3, characterized in that, The FPGA transmits the optical centroid coordinates to the host computer via a gigabit Ethernet communication module.

6. The method according to claim 1, 2, or 3, characterized in that, The camera is connected to the FPGA via a Camera Link cable.

7. A real-time fiber optic unit positioning method, characterized in that, The optical centroid coordinates of each light spot are obtained according to the method described in claim 1 and transmitted to the host computer in real time. The host computer calibrates or adjusts the position of the corresponding optical fiber unit according to the optical centroid coordinates of each light spot.

Images