FPGA-Based Multi-Spot Centroid Extraction Method, Device, Medium and Product

Abstract

FPGA-based multi-spot centroid extraction method, device, medium and product, which relates to the field of digital image processing. The method includes: First, the image line data is cached through a first-level FIFO and a second-level FIFO to form three consecutive groups of image line data streams, effectively solving the problem of data synchronization and processing rate matching, and improving the data processing efficiency. Then, the effective pixels of the spot are accurately determined by means of pixel thresholds, and different spot regions are accurately divided by using line continuity judgment and boundary marking, avoiding repeated recognition and misrecognition of spots, and greatly improving the accuracy of spot recognition. Finally, according to the weighted average position of the gray values of all the effective pixels of the spot, the target centroid coordinates of the target spot are calculated, fully considering the internal gray difference of the spot, and more accurately reflecting the center of the spot energy distribution compared with the conventional method, thus significantly improving the calculation accuracy of the spot centroid.

Description

Technical Field

[0001] This application relates to the field of digital image processing, and particularly to a multi-spot centroid extraction method, apparatus, medium and product based on FPGA. Background Art

[0002] In the field of digital image processing, especially in applications such as optical tracking, aerial imaging, and machine vision, the centroid extraction algorithm plays an important role in accurately positioning the center of the light spot. Therefore, quickly and accurately extracting the centroid of the light spot has become a core content of the research on image processing technology.

[0003] In the related art, the light spot centroid extraction method usually analyzes the collected image data on a processor through digital image processing software, generally including steps such as image preprocessing, light spot detection, and centroid calculation. For example, it is achieved by calculating the gray-weighted average position of all pixels in the light spot area.

[0004] Although the related art can achieve light spot detection and centroid extraction in various situations, in practical applications, especially in scenarios requiring fast response and high precision, due to relying on the computing power of the processor, high-resolution or high-frame-rate image processing will lead to an extended response time, thereby reducing the efficiency of light spot centroid extraction. Summary of the Invention

[0005] This application provides a multi-spot centroid extraction method, apparatus, medium and product based on FPGA, which is used to improve the extraction efficiency of the light spot centroid.

[0006] In a first aspect, the present application provides a multi-spot centroid extraction method based on FPGA, which is applied to a multi-spot centroid extraction device. The method includes: sequentially pushing the received image line data into a first-level FIFO and a second-level FIFO for caching, so that the input data, the output data of the first-level FIFO, and the output data of the second-level FIFO form three synchronous and continuous groups of image line data streams. The image line data is used to represent the grayscale value information of each row of pixels in the image. The size of the image is determined by the product of the pixels of a preset first number of rows and a preset second number of columns. Both the first-level FIFO and the second-level FIFO cache at least one group of image line data. The input data is used to represent the currently received image line data. The output data of the first-level FIFO is used to represent the image line data currently output by the first-level FIFO. The output data of the second-level FIFO is used to represent the image line data currently output by the second-level FIFO; constructing the three continuous groups of image line data streams into a preset grid sliding window, and the preset grid sliding window is determined by the product of the pixels of a preset third number of rows and a preset fourth number of columns; when the grayscale value of the central pixel in the preset grid sliding window is greater than a preset first pixel threshold and the sum of the grayscale values of all pixels in the preset grid sliding window is greater than a preset second pixel threshold, determining that the central pixel is a valid spot pixel, and outputting the coordinate position of the valid spot pixel; in each group of image line data, marking the coordinate position of the first valid spot pixel among the continuous valid spot pixels as the starting pixel position, and marking the coordinate position of the last valid spot pixel among the continuous valid spot pixels as the ending pixel position; marking the starting pixel position and the ending pixel position in the first image line data as valid boundaries, and graying out the valid spot pixels in the first image line data; based on the starting pixel position and the ending pixel position in the first image line data, and the starting pixel position and the ending pixel position in the second image line data, determining whether the spot in the first image line is continuous with the spot in the second image line; if continuous, marking the starting pixel position and the ending pixel position in the second image line data as valid boundaries, and graying out the valid spot pixels in the second image line data; calculating the target centroid coordinates of the target spot according to the weighted average position of the grayscale values of all valid spot pixels.

[0007] By adopting the above technical solutions, first, the multi-spot centroid extraction device caches the image line data through a first-level FIFO and a second-level FIFO, forming three synchronous and continuous image line data streams, which facilitates the construction of a preset grid sliding window, effectively solves the problem of data synchronization and processing rate matching, and improves the data processing efficiency. Secondly, the multi-spot centroid extraction device accurately determines the effective pixels of the spots by means of pixel thresholds, and accurately divides different spot regions by using line continuity judgment and boundary marking, avoiding repeated recognition and misrecognition of spots, and greatly improving the accuracy of spot recognition. Finally, the multi-spot centroid extraction device calculates the target centroid coordinates of the target spot based on the weighted average position of the gray values of all the effective pixels of the spots, fully considering the gray difference inside the spots, and more accurately reflecting the center of the spot energy distribution compared with the conventional method, thus significantly improving the calculation accuracy of the spot centroid.

[0008] Combined with some embodiments of the first aspect, in some embodiments, the received image line data is sequentially pushed into the first-level FIFO and the second-level FIFO for caching, so that the input data, the output data of the first-level FIFO, and the output data of the second-level FIFO form three synchronous and continuous image line data streams, which specifically includes: receiving the image line data, pushing the image line data into the first-level FIFO to obtain the output data of the first-level FIFO; pushing the output data of the first-level FIFO into the second-level FIFO to obtain the output data of the second-level FIFO; receiving the input data, so that the input data, the output data of the first-level FIFO, and the output data of the second-level FIFO form three synchronous and continuous image line data streams.

[0009] By adopting the above technical solutions, the flow process of the image line data in the FIFO cache is described in detail. The multi-spot centroid extraction device realizes the orderly caching and synchronous output of data by sequentially pushing the image line data into the first-level FIFO and the second-level FIFO. This data caching mechanism ensures the timing alignment of the three lines of image data, provides a stable and reliable data source for subsequent sliding window processing, and at the same time, this method also makes full use of the parallel processing ability of the FPGA, enabling data reading, caching, and output to be carried out simultaneously, improving the data processing efficiency and reducing the processing delay.

[0010] Combined with some embodiments of the first aspect, in some embodiments, the three synchronous and continuous image line data streams are constructed into a preset grid sliding window, and the preset grid sliding window is determined by the product of the pixels of a preset third number of rows and the pixels of a preset fourth number of columns, which specifically includes: taking each pixel in the three synchronous and continuous image line data streams as the central pixel; obtaining a plurality of adjacent pixels of the central pixel; taking the central pixel as the center, and constructing the preset grid sliding window according to the central pixel and the adjacent pixels, and the preset grid sliding window is determined by the product of the pixels of a preset third number of rows and the pixels of a preset fourth number of columns.

[0011] By adopting the above technical solution, the multi-spot centroid extraction device realizes the local feature extraction of three consecutive groups of image row data streams by constructing a preset grid sliding window with the central pixel as the reference. The preset grid sliding window can simultaneously consider the gray distribution around the target pixel, improving the accuracy of spot detection. At the same time, the sliding mode of the preset grid sliding window ensures continuous scanning of the entire image without missing any possible spot areas.

[0012] Combined with some embodiments of the first aspect, in some embodiments, after the step of constructing the preset grid sliding window with the central pixel as the center according to the central pixel and the adjacent pixels, where the preset grid sliding window is determined by the product of the pixels in the preset third number of rows and the pixels in the preset fourth number of columns, the method further includes: if the preset grid sliding window cannot be constructed according to the central pixel and the adjacent pixels, then taking the central pixel as the symmetry center, and complementing the preset grid sliding window according to the central pixel and the adjacent pixels.

[0013] By adopting the above technical solution, the process of constructing a preset grid sliding window in the image edge region is described in detail. When processing pixels near the image edge, due to insufficient adjacent pixels, it may not be possible to completely construct the preset grid sliding window. The multi-spot centroid extraction device takes the central pixel as the symmetry center to complement the pixels, ensuring that a complete preset grid sliding window can also be formed in the edge region. This method of symmetric complementation maintains the continuity and consistency of the spot detection algorithm, avoiding missed detection of spots in the edge region and not introducing additional processing deviations.

[0014] Combined with some embodiments of the first aspect, in some embodiments, determining whether the spot of the first image row is continuous with the spot of the second image row based on the starting pixel position and the ending pixel position in the first image row data, and the starting pixel position and the ending pixel position in the second image row data specifically includes: determining the first left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the first image row data; determining the second left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the second image row data; if the first left and right boundary column coordinates and the second left and right boundary column coordinates meet the preset row continuity judgment condition, then determining that the spot of the first image row is continuous with the spot of the second image row; if the first left and right boundary column coordinates and the second left and right boundary column coordinates do not meet the preset row continuity judgment condition, then determining that the spot of the first image row is not continuous with the spot of the second image row.

[0015] By adopting the above technical solution, the multi-spot centroid extraction device has established a strict row continuity judgment mechanism, achieving accurate segmentation of multi-row spots. By comparing the left and right boundary column coordinates of adjacent rows and according to the preset row continuity judgment conditions, the multi-spot centroid extraction device can accurately judge whether the spots in different image rows belong to the same target spot. This judgment mechanism effectively solves the problem of separating multiple adjacent spots and avoids the incorrect merging of different spots. At the same time, this method can also effectively handle the situation where a single spot spans multiple rows, ensuring the integrity of the spot area. This continuity judgment method based on boundary coordinates improves the recognition accuracy of multiple spots in complex scenarios and provides a reliable basis for dividing the spot area for subsequent centroid calculation.

[0016] Combined with some embodiments of the first aspect, in some embodiments, calculating the target centroid coordinates of the target spot according to the weighted average position of the gray values of all effective pixels of the spots specifically includes: accumulating the gray values of all effective pixels of the spots to obtain a denominator value; calculating the product of the coordinate position and the gray value of each effective pixel of the spot; accumulating all the products to obtain a numerator value; and obtaining the target centroid coordinates of the target spot based on the numerator value and the denominator value.

[0017] By adopting the above technical solution, the multi-spot centroid extraction device accumulates the gray values of all effective pixels of the spots and the product of the coordinate position and the gray value of each effective pixel of the spot, and obtains the target centroid coordinates of the accurate target spot by division. This calculation method takes into account the contribution of the gray value of each effective pixel of the spot to the centroid position, assigns a greater weight to the pixels with high gray values, and makes the calculation result closer to the actual center position of the spot.

[0018] Combined with some embodiments of the first aspect, in some embodiments, after the step of calculating the target centroid coordinates of the target spot according to the weighted average position of the gray values of all effective pixels of the spots, the method further includes: after outputting the target centroid coordinates, graying out the target spot to avoid repeated recognition of the target spot.

[0019] By adopting the above technical solution, the multi-spot centroid extraction device establishes an effective spot processing marking mechanism by graying out the target spot after outputting the target centroid coordinates, ensuring that the processed spots will not be repeatedly recognized and calculated, effectively avoiding the problem of repeated calculation that may be caused by continuous data processing, not only improving the processing efficiency but also ensuring the result accuracy. By real-time updating the spot status in the image row data, the multi-spot centroid extraction device can correctly process multiple spots with dense distribution and accurately track the processing status of each spot, so as to be able to reliably handle the multi-spot situation in complex scenarios and improve the practicability.

[0020] In a second aspect, an embodiment of the present application provides a multi-spot centroid extraction device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions. The one or more processors call the computer instructions to cause the multi-spot centroid extraction device to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product containing instructions. When the computer program product runs on a multi-spot centroid extraction device, it causes the multi-spot centroid extraction device to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions. When the instructions run on a multi-spot centroid extraction device, it causes the multi-spot centroid extraction device to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] It can be understood that the multi-spot centroid extraction device provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1. By adopting the above technical solutions, first, the multi-spot centroid extraction device caches image line data through a first-level FIFO and a second-level FIFO to form three synchronous and continuous image line data streams, which is convenient for constructing a preset grid sliding window, effectively solves the problem of data synchronization and processing rate matching, and improves data processing efficiency. Second, the multi-spot centroid extraction device accurately determines the effective pixels of the spot by means of pixel thresholds, and accurately divides different spot regions by using line continuity judgment and boundary marking, avoiding repeated recognition and misrecognition of spots, and greatly improving the accuracy of spot recognition. Finally, the multi-spot centroid extraction device calculates the target centroid coordinates of the target spot based on the weighted average position of the gray values of all the effective pixels of the spot, fully considering the internal gray difference of the spot, and more accurately reflecting the center of the spot energy distribution compared with the conventional method, thus significantly improving the calculation accuracy of the spot centroid.

[0026] 2. By adopting the above technical solution, the multi-spot centroid extraction device has established a strict row continuity judgment mechanism, achieving accurate segmentation of multi-row spots. By comparing the left and right boundary column coordinates of adjacent rows and according to the preset row continuity judgment conditions, the multi-spot centroid extraction device can accurately determine whether the spots in different image rows belong to the same target spot. This judgment mechanism effectively solves the problem of separating multiple adjacent spots and avoids the incorrect merging of different spots. At the same time, this method can also effectively handle the situation where a single spot spans multiple rows, ensuring the integrity of the spot area. This continuity judgment method based on boundary coordinates improves the recognition accuracy of multiple spots in complex scenarios and provides a reliable basis for spot area division for subsequent centroid calculation.

[0027] 3. By adopting the above technical solution, the multi-spot centroid extraction device uses the method of graying out the target spot after outputting the target centroid coordinates to establish an effective spot processing marking mechanism, ensuring that the processed spots will not be repeatedly recognized and calculated, effectively avoiding the problem of repeated calculation that may be caused by continuous data processing, not only improving the processing efficiency but also ensuring the result accuracy. By real-time updating the spot status in the image row data, the multi-spot centroid extraction device can correctly process multiple densely distributed spots and accurately track the processing status of each spot, thus being able to reliably handle the multi-spot situation in complex scenarios and improving the practicality. Description of the Drawings

[0028] Figure 1 is a flowchart of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application;

[0029] Figure 2 is another flowchart of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application;

[0030] Figure 3 is a module diagram of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application;

[0031] Figure 4 is a scenario diagram of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application;

[0032] Figure 5 is a schematic structural diagram of an entity device of a multi-spot centroid extraction device in an embodiment of the present application. Detailed Embodiment

[0033] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular forms "a", "an", "the above", "the", and "this" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed items.

[0034] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and should not be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0035] The following describes the process of the method provided in this embodiment. Please refer to Figure 1 , which is a schematic flowchart of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application.

[0036] S101. Sequentially push the received image line data into the first-level FIFO and the second-level FIFO for caching, so that the input data, the output data of the first-level FIFO, and the output data of the second-level FIFO form three synchronous continuous groups of image line data streams. The image line data is used to represent the grayscale value information of each row of pixels in the image. The size of the image is determined by the product of the pixels of a preset first number of rows and the pixels of a preset second number of columns. Both the first-level FIFO and the second-level FIFO cache at least one group of image line data. The input data is used to represent the currently received image line data. The output data of the first-level FIFO is used to represent the image line data currently output by the first-level FIFO. The output data of the second-level FIFO is used to represent the image line data currently output by the second-level FIFO;

[0037] Among them, the image line data refers to the grayscale value information of each row of pixels in the image, and the grayscale value of each pixel is used to represent the brightness level of the pixel point; the first-level FIFO and the second-level FIFO refer to two data buffers that work according to the first-in-first-out principle and are used to temporarily store and synchronize data; the input data refers to the latest image line data currently being received; the FIFO output data refers to the cached image line data output from the FIFO data buffer; the three continuous groups of image line data streams refer to three adjacent rows of image data arranged in time sequence.

[0038] Specifically, first, the multi-spot centroid extraction device receives image line data containing grayscale value information, caches the image line data into a first-level FIFO, and the data output by the first-level FIFO is further cached into a second-level FIFO. Through this cache mechanism of cascading two FIFOs, the currently input data, the data of the previous line output by the first-level FIFO, and the data of the line before the previous line output by the second-level FIFO are completely aligned in time sequence, forming a stable continuous three-group image line data stream, providing a basis for subsequent sliding window processing.

[0039] Optionally, generally, the received image line data is sequentially cached into the first-level FIFO and the second-level FIFO to form a synchronous continuous three-group image line data stream of the input data, the data output by the first-level FIFO, and the data output by the second-level FIFO. This can be achieved in the following way (not limited here): receive the image line data, cache the image line data into the first-level FIFO to obtain the data output by the first-level FIFO; cache the data output by the first-level FIFO into the second-level FIFO to obtain the data output by the second-level FIFO; receive the input data to make the input data, the data output by the first-level FIFO, and the data output by the second-level FIFO form a synchronous continuous three-group image line data stream.

[0040] S102. Construct the continuous three-group image line data stream into a preset grid sliding window, which is determined by the product of the pixels of a preset third number of rows and a preset fourth number of columns;

[0041] Among them, the preset grid sliding window refers to a rectangular pixel area composed of a predetermined number of rows and a predetermined number of columns, used for local feature analysis; the continuous three-group image line data stream refers to three adjacent image data arranged in time sequence; the preset third number of rows represents the number of pixel rows included in the preset grid sliding window in the vertical direction, usually an odd number to ensure the uniqueness of the central pixel; the preset fourth number of columns represents the number of pixel columns included in the preset grid sliding window in the horizontal direction, usually equal to the number of rows to form a square window; the product determination represents the total number of pixels within the preset grid sliding window, used to specify the size of the local analysis area.

[0042] Specifically, first, the multi-spot centroid extraction device determines the size of the preset grid sliding window to be constructed according to the size parameters of the preset grid sliding window (the preset third number of rows and the preset fourth number of columns), such as a 3×3 or 5×5 rectangular window. Then, starting from the leftmost side of three consecutive groups of image row data streams, the multi-spot centroid extraction device organizes the pixel data at the corresponding positions into the preset grid sliding window according to a predetermined row-column structure to form a complete local analysis unit. The construction process of the preset grid sliding window needs to ensure the correct positional correspondence of the pixel data, that is, the relative positions of the upper-row data, the middle-row data, and the lower-row data in the preset grid sliding window remain unchanged. Once the preset grid sliding window is constructed, it can slide pixel by pixel in the horizontal direction, and reorganize the window data once every time it moves one position, so as to realize the continuous scanning and analysis of the entire image.

[0043] S103. When the gray value of the central pixel in the preset grid sliding window is greater than the preset first pixel threshold and the sum of the gray values of all pixels in the preset grid sliding window is greater than the preset second pixel threshold, determine that the central pixel is a valid spot pixel, and output the coordinate position of the valid spot pixel;

[0044] Among them, the central pixel refers to the pixel point at the center position of the preset grid sliding window; the preset first pixel threshold refers to the gray value standard for judging whether a single pixel point is bright enough; the preset second pixel threshold refers to the gray value sum standard for judging the overall brightness of the local area; the valid spot pixel refers to the pixel point confirmed to belong to the spot area.

[0045] Specifically, first, the multi-spot centroid extraction device checks whether the gray value of the central pixel in the preset grid sliding window exceeds the preset first pixel threshold to ensure that the pixel point has sufficient brightness. At the same time, the multi-spot centroid extraction device calculates the sum of the gray values of all pixels in the preset grid sliding window and compares it with the preset second pixel threshold to ensure that the overall brightness of the local area meets the requirements. When both of the above conditions are met, the multi-spot centroid extraction device can determine that the central pixel is a valid spot pixel and record its coordinate position.

[0046] S104. In each group of image row data, mark the coordinate position of the first valid spot pixel in the consecutive valid spot pixels as the starting pixel position, and mark the coordinate position of the last valid spot pixel in the consecutive valid spot pixels as the ending pixel position;

[0047] Among them, the consecutive valid spot pixels refer to the set of pixel points that are adjacent and confirmed as valid spot pixels in the same row; the starting pixel position refers to the starting position coordinate of each segment of consecutive valid spot pixels; the ending pixel position refers to the ending position coordinate of each segment of consecutive valid spot pixels.

[0048] Specifically, the multi-spot centroid extraction device analyzes the effective pixels of the spots in each group of image line data to find the continuity of the pixel points in the horizontal direction. When continuous effective pixels of the spots are detected, the coordinates of the leftmost effective pixel of the spot are recorded as the starting pixel position, and the coordinates of the rightmost effective pixel of the spot are recorded as the ending pixel position, so as to determine the horizontal range of each spot in this line.

[0049] S105. Mark the starting pixel position and the ending pixel position in the first image line data as valid boundaries, and gray out the effective pixels of the spots in the first image line data;

[0050] Among them, the first image line data refers to the image line being currently processed; the valid boundary refers to the start and end position marks of the pixel area that has been confirmed to belong to the same spot; graying out means modifying the gray value of the processed effective pixels of the spots to the background value to avoid repeated processing; the effective pixels of the spots refer to the pixel points that are confirmed to belong to the spot area after analysis by sliding a preset grid sliding window.

[0051] Specifically, the multi-spot centroid extraction device marks the determined starting pixel position and ending pixel position as the valid boundaries of the spots in this line for subsequent judgment of spot continuity. At the same time, in order to avoid repeated processing of the same spot, the gray values of all the confirmed effective pixels of the spots in this line are modified to the background gray value. This dual mechanism of marking and graying out not only retains the position information of the spots but also prevents repeated calculations, improving the processing efficiency and accuracy.

[0052] S106. Based on the starting pixel position and the ending pixel position in the first image line data, and the starting pixel position and the ending pixel position in the second image line data, judge whether the spots in the first image line are continuous with the spots in the second image line;

[0053] Among them, the second image line data refers to the next image line data adjacent to the first image line; continuous means whether the spots in two adjacent lines belong to the same complete spot area.

[0054] Specifically, the multi-spot centroid extraction device judges whether they belong to the same spot by comparing the starting pixel position and the ending pixel position of the effective pixels of the spots in the first image line data and the second image line data. The judgment criteria usually include: whether the horizontal positions of the spots in two lines overlap, whether the size of the overlapping area exceeds a preset threshold, whether the width ratio of the spots in two lines is within a reasonable range, etc. Through the comprehensive judgment of these conditions, the complete spot area spanning multiple lines can be accurately identified.

[0055] Optionally, generally, based on the starting pixel position and the ending pixel position in the first image line data, and the starting pixel position and the ending pixel position in the second image line data, determining whether the light spot of the first image line is continuous with the light spot of the second image line can be achieved in the following ways, which are not limited herein: According to the starting pixel position and the ending pixel position in the first image line data, determine the first left and right boundary column coordinates; According to the starting pixel position and the ending pixel position in the second image line data, determine the second left and right boundary column coordinates; If the first left and right boundary column coordinates and the second left and right boundary column coordinates meet the preset row continuity judgment condition, it is determined that the light spot of the first image line is continuous with the light spot of the second image line; If the first left and right boundary column coordinates and the second left and right boundary column coordinates do not meet the preset row continuity judgment condition, it is determined that the light spot of the first image line is not continuous with the light spot of the second image line.

[0056] The preset row continuity judgment condition is: ;

[0057] Among them, X1a and X2a respectively represent the left boundary column coordinates of the light spots of the first image line and the second image line; X1b and X2b respectively represent the right boundary column coordinates of the light spots of the first image line and the second image line. When the preset row continuity judgment condition is met, it means that there is an overlapping area between the two rows of the light spot, that is, the light spots are continuous between rows; When the preset row continuity judgment condition is not met, it means that there is no overlapping area between the two rows of the light spot, that is, the light spots are not continuous between rows.

[0058] S107. If it is continuous, mark the starting pixel position and the ending pixel position in the second image line data as valid boundaries, and gray out the valid pixels of the light spot in the second image line data;

[0059] Among them, continuous means meeting the preset light spot continuity judgment standard; Marking as a valid boundary means adding the boundary information of the valid pixels of the light spot in this row to the complete light spot area being processed; Graying out means modifying the gray value of the pixel points that have been confirmed to belong to the current light spot to the background value.

[0060] Specifically, the multi-light spot centroid extraction device adds the boundary information of the valid pixels of the light spot in the second image line to the light spot area being processed, and continues to construct a complete light spot contour. At the same time, gray out all the valid pixels of the light spot in the second image line to avoid being repeatedly calculated or being wrongly divided into other light spots, ensuring the integrity and uniqueness of the light spot area.

[0061] S108. Calculate the target centroid coordinates of the target light spot according to the weighted average position of the gray values of all the valid pixels of the light spot.

[0062] Among them, the weighted average position of the grayscale value is the result of weighted calculation considering the contribution of the grayscale value of each pixel; the target light spot refers to a single completely recognized light spot area; the target centroid coordinate represents the precise coordinate value of the center of gravity position of the target light spot; the weighted calculation refers to determining its weight in the centroid calculation according to the size of the pixel grayscale value.

[0063] Specifically, the multi-light spot centroid extraction device first calculates the sum of the grayscale values of all effective pixels of the light spots in the light spot area as the denominator, then calculates the product of the coordinate position of each effective pixel of the light spot and its grayscale value and accumulates it as the numerator, and finally divides the numerator by the denominator to obtain the weighted average position, which is the centroid coordinate of the light spot. This calculation method based on grayscale value weighting considers the distribution characteristics of pixel brightness and can more accurately reflect the actual center position of the light spot, especially suitable for light spots with irregular shapes or non-uniform brightness distributions.

[0064] Optionally, generally, according to the weighted average position of the grayscale values of all effective pixels of the light spots, the target centroid coordinate of the target light spot can be obtained through the following methods, which are not limited here: accumulate the grayscale values of all effective pixels of the light spots to obtain the denominator value; calculate the product of the coordinate position of each effective pixel of the light spot and its grayscale value; accumulate all the products to obtain the numerator value; based on the numerator value and the denominator value, obtain the target centroid coordinate of the target light spot.

[0065] By adopting the above technical solutions, first, the multi-light spot centroid extraction device caches the image line data through the first-level FIFO and the second-level FIFO to form three synchronous continuous image line data streams, which is convenient for constructing a preset grid sliding window, effectively solves the problem of data synchronization and processing rate matching, and improves the data processing efficiency. Secondly, the multi-light spot centroid extraction device accurately determines the effective pixels of the light spots with the help of pixel thresholds, and accurately divides different light spot areas by using the method of row continuity judgment and boundary marking, avoiding repeated recognition and misrecognition of light spots, and greatly improving the accuracy of light spot recognition. Finally, the multi-light spot centroid extraction device calculates the target centroid coordinate of the target light spot according to the weighted average position of the grayscale values of all effective pixels of the light spots, fully considering the internal grayscale difference of the light spots, and more accurately reflects the center of light spot energy distribution compared with the conventional method, thus significantly improving the calculation accuracy of the light spot centroid.

[0066] The following is a further and more specific process description of the method provided in this embodiment. Please refer to Figure 2 , which is another process schematic diagram of the multi-light spot centroid extraction method based on FPGA in the embodiments of the present application.

[0067] S201. Sequentially push the received image line data into the first-level FIFO and the second-level FIFO for caching, so that the input data, the first-level FIFO output data, and the second-level FIFO output data form three synchronous and continuous image line data streams. The image line data is used to represent the grayscale value information of each row of pixels in the image. The size of the image is determined by the product of the pixels of the preset first number of rows and the preset second number of columns. Both the first-level FIFO and the second-level FIFO cache at least one set of image line data. The input data is used to represent the currently received image line data. The first-level FIFO output data is used to represent the image line data currently output by the first-level FIFO. The second-level FIFO output data is used to represent the image line data currently output by the second-level FIFO;

[0068] Specifically, refer to step S101, which will not be elaborated here.

[0069] S202. Take each pixel in the three continuous image line data streams as the central pixel;

[0070] Among them, the three continuous image line data streams represent three adjacent image line data sets after being synchronized by the FIFO cache; a pixel is the smallest unit that constitutes a digital image, containing position and grayscale value information; the central pixel represents the target pixel point currently selected as the center of the preset grid sliding window; the grayscale value information is used to represent the brightness level of the pixel point.

[0071] Specifically, the multi-spot centroid extraction device sequentially sets each pixel point in the three continuous image line data streams as the central pixel in the order from left to right and from top to bottom. For each row, the multi-spot centroid extraction device starts from the first pixel and traverses to the last pixel one by one, ensuring that all pixels have the opportunity to be analyzed and processed as the center point of the preset grid sliding window, guaranteeing the integrity and continuity of image processing.

[0072] S203. Obtain multiple adjacent pixels of the central pixel;

[0073] Specifically, the multi-spot centroid extraction device determines the adjacent pixels according to the central pixel. The adjacent pixels are the surrounding pixel points directly connected to the central pixel in terms of spatial position, usually including the pixels above, below, to the left, to the right of the central pixel, and the pixels in the four diagonal directions. These adjacent pixels and the central pixel together constitute the basic unit for local feature analysis.

[0074] S204. With the central pixel as the center, construct the preset grid sliding window according to the central pixel and the adjacent pixels. The preset grid sliding window is determined by the product of the pixels of the preset third number of rows and the preset fourth number of columns;

[0075] Among them, the preset grid sliding window refers to a rectangular pixel area with a fixed size; the preset third number of rows represents the number of pixels in the vertical direction of the preset grid sliding window; the preset fourth number of columns represents the number of pixels in the horizontal direction of the preset grid sliding window.

[0076] Specifically, first, the multi-spot centroid extraction device places the central pixel at the center of the window, and then constructs a rectangular window according to the preset third number of rows and the preset fourth number of columns. Usually, the window size is an odd number such as 3×3, 5×5, etc., to ensure that the central pixel is located in the center of the window. The construction process needs to ensure the integrity and orderliness of the pixels in the preset grid sliding window, so that the window can accurately reflect the local image features.

[0077] S205: If the preset grid sliding window cannot be constructed according to the central pixel and the adjacent pixels, the preset grid sliding window is completed according to the central pixel and the adjacent pixels with the central pixel as the symmetry center;

[0078] Among them, unable to construct refers to the state where the central pixel is located at the edge of the image, resulting in insufficient adjacent pixels; the symmetry center refers to the reference point when performing pixel padding; the padding operation means filling the missing window pixels according to specific rules; pixel mirroring refers to a symmetrical filling method with the central pixel as the axis.

[0079] Specifically, the multi-spot centroid extraction device first determines the position and number of missing pixels, and then uses the central pixel as the axis of symmetry to map the existing adjacent pixels to the missing position in a mirrored manner. For example, when the central pixel is located at the left edge of the image, the existing pixels on the right can be filled in the missing position on the left in a left-right symmetrical manner. This symmetry-based filling method maintains the local features of the image, avoids the particularity of edge processing, and enables edge pixels to obtain complete window analysis.

[0080] S206, when the grayscale value of the central pixel of the preset grid sliding window is greater than the preset first pixel threshold and the sum of the grayscale values ​​of all pixels of the preset grid sliding window is greater than the preset second pixel threshold, determine that the central pixel is a light spot effective pixel, and output the coordinate position of the light spot effective pixel;

[0081] For details, please refer to step S103, which will not be described in detail here.

[0082] S207, in each group of image row data, marking the coordinate position of the first effective pixel of the continuous effective pixels of the light spots as the starting pixel position, and marking the coordinate position of the last effective pixel of the continuous effective pixels of the light spots as the ending pixel position;

[0083] For details, please refer to step S104, which will not be described in detail here.

[0084] S208. Mark the starting pixel position and the ending pixel position in the first image line data as valid boundaries, and set the light spot valid pixels in the first image line data to gray;

[0085] Specifically, refer to step S105, which will not be elaborated here.

[0086] S209. Based on the starting pixel position and the ending pixel position in the first image line data, and the starting pixel position and the ending pixel position in the second image line data, determine whether the light spot of the first image line is continuous with the light spot of the second image line;

[0087] Specifically, refer to step S106, which will not be elaborated here.

[0088] S210. If they are continuous, mark the starting pixel position and the ending pixel position in the second image line data as valid boundaries, and set the light spot valid pixels in the second image line data to gray;

[0089] Specifically, refer to step S107, which will not be elaborated here.

[0090] S211. Calculate the target centroid coordinates of the target light spot according to the weighted average position of the gray values of all light spot valid pixels;

[0091] Specifically, refer to step S108, which will not be elaborated here.

[0092] S212. After outputting the target centroid coordinates, set the target light spot to gray to avoid repeated recognition of the target light spot.

[0093] Among them, the target centroid coordinates refer to the centroid position information of the target light spot that has completed the calculation, including the accurate position values in the horizontal and vertical directions; the target light spot refers to the complete light spot area from which the target centroid coordinates have been extracted; the graying operation means modifying the gray values of all pixels in the target light spot to the background gray value; repeated recognition refers to the situation where the same light spot area is detected and calculated multiple times; the light spot area refers to the set of all continuous pixels that have been confirmed to belong to the same light spot.

[0094] Specifically, first, the multi-spot centroid extraction device determines the spot area to be grayed out, that is, all the valid pixels of the spots that have been marked as belonging to the target spots. Then, the multi-spot centroid extraction device uniformly modifies the gray values of these spot valid pixels to the same gray value as the image background, usually choosing a lower gray value to ensure a clear contrast with other unprocessed spots. This graying process not only marks the processed spot area but also fundamentally prevents the repeated detection of this spot area by changing the pixel gray values. For multiple adjacent or partially overlapping spots, this processing method is particularly important because it can ensure that each spot is only processed once, improving the processing accuracy and efficiency in the multi-spot scenario.

[0095] The following is a module description of the method provided in this embodiment. Please refer to Figure 3 , which is a schematic diagram of a module of the multi-spot centroid extraction method based on FPGA in the embodiment of the present application.

[0096] There are multiple spot screening modules in the multi-spot centroid extraction device, which process the image data stream in sequence. After being processed by the spot screening module 1, an image data stream after screening is output as the input of the spot screening module 2, and so on. The image data stream passes through the spot screening module 1, the spot screening module 2, the spot screening module 3... until the spot screening module N in sequence, so as to realize the multi-spot centroid extraction in the whole image as a whole.

[0097] The following is a scenario description of the method provided in this embodiment. Please refer to Figure 4 , which is a schematic diagram of a scenario of the multi-spot centroid extraction method based on FPGA in the embodiment of the present application.

[0098] As Figure 4 (a) is an unprocessed image, which contains multiple spots, Figure 4 (b) is a partially processed image, where the grayed-out ones represent the processed spots, and the non-grayed-out ones represent the spots that have not been processed yet.

[0099] The following describes the multi-spot centroid extraction device in the embodiment of the present invention application from the perspective of hardware processing. Please refer to Figure 5 , which is a schematic diagram of the physical device structure of the multi-spot centroid extraction device in the embodiment of the present application.

[0100] It should be noted that Figure 5 the structure of the multi-spot centroid extraction device shown is only an example and should not bring any limitations to the functions and usage scopes of the embodiments of the present invention.

[0101] As Figure 5As shown, the multi-spot centroid extraction device includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) 502 or the program loaded from the storage section 508 into the Random Access Memory (RAM) 503, such as executing the method described in the above embodiments. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0102] The following components are connected to the I / O interface 505: an input section 506 including an audio input device, a button switch, etc.; an output section 507 including a Liquid Crystal Display (LCD), an audio output device, an indicator light, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read from it can be installed into the storage section 508 as needed.

[0103] Specifically, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication section 509, and / or installed from the removable medium 511. When the computer program is executed by the Central Processing Unit (CPU) 501, various functions defined in the present invention are executed.

[0104] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings.

[0106] Specifically, the multi-spot centroid extraction device of this embodiment includes a processor and a memory. A computer program is stored on the memory. When the computer program is executed by the processor, the multi-spot centroid extraction method based on FPGA provided in the above embodiment is implemented.

[0107] On the other hand, the present invention also provides a computer-readable storage medium, which may be included in the multi-spot centroid extraction device described in the above embodiment; or it may exist separately without being assembled into the multi-spot centroid extraction device. The above storage medium carries one or more computer programs. When the above one or more computer programs are executed by a processor of the multi-spot centroid extraction device, the multi-spot centroid extraction device is enabled to implement the multi-spot centroid extraction method based on FPGA provided in the above embodiment.

[0108] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0109] As used in the foregoing embodiments, depending on the context, the term "when" may be construed to mean "if" or "after" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "upon determining" or "if (the stated condition or event) is detected" may be construed to mean "if determined" or "in response to determining" or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

[0110] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by computer programs instructing relevant hardware. The programs can be stored in a computer-readable storage medium. When the programs are executed, they can include the processes of the foregoing method embodiments. The foregoing storage media include: various media such as ROM or random access memory RAM, magnetic disks, or optical discs that can store program codes.

Claims

1. A multi-spot centroid extraction method based on FPGA, characterized in that: Applied to a multi-spot centroid extraction device, the method comprises: The received image row data are sequentially pushed into the primary FIFO and the secondary FIFO for caching, so that the input data and the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image row data streams, the image row data are used to represent the gray value information of each row of pixels of the image, the size of the image is determined by the product of a preset first number of rows of pixels and a preset second number of columns of pixels, the primary FIFO and the secondary FIFO each cache at least one set of image row data, the input data are used to represent the image row data currently being received, the primary FIFO output data are used to represent the image row data currently output by the primary FIFO, and the secondary FIFO output data are used to represent the image row data currently output by the secondary FIFO; constructing the three consecutive groups of image row data streams into a preset grid sliding window, wherein the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels; When the grayscale value of the central pixel of the preset grid sliding window is greater than a preset first pixel threshold and the sum of the grayscale values ​​of all pixels of the preset grid sliding window is greater than a preset second pixel threshold, the central pixel is determined to be a light spot effective pixel, and the coordinate position of the light spot effective pixel is output; In each group of image row data, the coordinate position of the first effective pixel of the continuous effective pixels of the light spots is marked as the starting pixel position, and the coordinate position of the last effective pixel of the continuous effective pixels of the light spots is marked as the ending pixel position; Marking the starting pixel position and the ending pixel position in the first image row data as valid boundaries, and graying out the effective pixels of the light spot in the first image row data; Based on the starting pixel position and the ending pixel position in the first image row data, and the starting pixel position and the ending pixel position in the second image row data, determining whether the light spot of the first image row is continuous with the light spot of the second image row; If they are continuous, the starting pixel position and the ending pixel position in the second image row data are marked as valid boundaries, and the effective pixels of the light spot in the second image row data are grayed out; The target centroid coordinates of the target light spot are calculated based on the weighted average position of the gray values ​​of all effective pixels of the light spot.

2. The method according to claim 1, characterized in that The received image row data is sequentially pushed into the primary FIFO and the secondary FIFO for buffering, so that the input data and the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image row data streams, specifically including: receiving the image row data, pushing the image row data into a first-level FIFO, and obtaining first-level FIFO output data; Pushing the primary FIFO output data into the secondary FIFO to obtain the secondary FIFO output data; Input data is received so that the input data, the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image line data streams.

3. The method according to claim 1, characterized in that The step of constructing the three consecutive groups of image row data streams into a preset grid sliding window, wherein the preset grid sliding window is determined by multiplying pixels of a preset third number of rows by pixels of a preset fourth number of columns, specifically includes: Taking each pixel in the three consecutive groups of image row data streams as a central pixel; Acquire a plurality of adjacent pixels of the central pixel; Taking the central pixel as the center, the preset grid sliding window is constructed according to the central pixel and the adjacent pixels, and the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels.

4. The method according to claim 3, characterized in that After the step of constructing the preset grid sliding window based on the central pixel and the adjacent pixels with the central pixel as the center, wherein the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels, the method further includes: If the preset grid sliding window cannot be constructed according to the central pixel and the adjacent pixels, the preset grid sliding window is completed according to the central pixel and the adjacent pixels with the central pixel as the symmetry center.

5. The method according to claim 1, characterized in that The determining whether the light spot of the first image row is continuous with the light spot of the second image row based on the starting pixel position and the ending pixel position in the first image row data and the starting pixel position and the ending pixel position in the second image row data specifically includes: Determine first left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the first image row data; Determine the second left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the second image row data; If the first left-right boundary column coordinates and the second left-right boundary column coordinates meet a preset row continuity judgment condition, it is determined that the light spot of the first image row is continuous with the light spot of the second image row; If the first left-right boundary column coordinates and the second left-right boundary column coordinates do not meet the preset row continuity judgment condition, it is determined that the light spot of the first image row is discontinuous with the light spot of the second image row.

6. The method according to claim 1, characterized in that The target centroid coordinates of the target light spot are calculated based on the weighted average position of the gray values ​​of all effective pixels of the light spot, specifically including: Accumulate the grayscale values ​​of all effective pixels of the light spot to obtain the denominator value; Calculate the product of the coordinate position of each effective pixel of the light spot and the gray value; Add up all the products to get the numerator value; Based on the numerator value and the denominator value, the target centroid coordinates of the target light spot are obtained.

7. The method according to claim 1, characterized in that After the step of calculating the target centroid coordinates of the target light spot according to the weighted average position of the gray values ​​of all effective pixels of the light spot, the method further includes: After the target centroid coordinates are output, the target light spot is grayed out to avoid repeated identification of the target light spot.

8. A multi-spot centroid extraction device, characterized in that: The multi-spot centroid extraction device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the multi-spot centroid extraction device to perform the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a multi-spot centroid extraction device, the multi-spot centroid extraction device is enabled to perform the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a multi-spot centroid extraction device, the multi-spot centroid extraction device is enabled to perform the method according to any one of claims 1 to 7.

Images