Multi-light-spot centroid extraction method and device based on FPGA, medium and product

Abstract

The invention discloses a multi-light-spot centroid extraction method and device based on an FPGA, a medium and a product, and relates to the field of digital image processing, and the method comprises the steps: firstly, caching image line data through a first-stage FIFO and a second-stage FIFO, forming three continuous groups of image line data streams, effectively solving the problems of data synchronization and processing rate matching, and improving the data processing efficiency; then, effective pixels of the light spots are accurately determined by means of a pixel threshold value, and different light spot areas are accurately divided by means of line continuity judgment and boundary marking, so that repeated recognition and error recognition of the light spots are avoided, and the accuracy of light spot recognition is greatly improved. And finally, according to the weighted average positions of the gray values of the effective pixels of all the light spots, the target centroid coordinates of the target light spots are calculated, the internal gray difference of the light spots is fully considered, and compared with a conventional method, the light spot energy distribution center is more accurately reflected, so that the calculation precision of the light spot centroid is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of digital image processing, and in particular to a method, device, medium and product for extracting the centroid of multiple light spots based on FPGA. Background Art

[0002] In the field of digital image processing, especially in applications such as optical tracking, aerial imaging, and machine vision, the centroid extraction algorithm plays an important role in accurately locating the center of the light spot. Therefore, quickly and accurately extracting the centroid of the light spot has become a core content of image processing technology research.

[0003] In the related art, the method of extracting the centroid of a light spot is usually to analyze the collected image data on a processor through digital image processing software, which generally includes steps such as image preprocessing, light spot detection, and centroid calculation. For example, it is achieved by calculating the gray-weighted average position of all pixels in the light spot area.

[0004] Although the relevant technology can realize light spot detection and centroid extraction in many cases, in actual applications, especially in scenarios requiring fast response and high precision, high-resolution or high-frame-rate image processing will lead to longer response time due to its reliance on the computing power of the processor, thereby reducing the efficiency of light spot centroid extraction. Summary of the invention

[0005] The present application provides a method, device, medium and product for extracting the centroid of multiple light spots based on FPGA, which are used to improve the extraction efficiency of the centroid of light spots.

[0006] In a first aspect, the present application provides a multi-spot centroid extraction method based on FPGA, which is applied to a multi-spot centroid extraction device, and the method includes: pushing the received image row data into a primary FIFO and a secondary FIFO for caching in sequence, so that the input data and the primary FIFO output data and the secondary FIFO output data form three synchronous continuous groups of image row data streams, and the image row data is used to represent the grayscale value information of each row of pixels in the image, and the size of the image is determined by the product of a preset first number of rows of pixels and a preset second number of columns of pixels, and the primary FIFO and the secondary FIFO both cache at least one group of image row data, and the input data is used to represent the image row data currently being received, and the primary FIFO output data is used to represent the image row data currently output by the primary FIFO, and the secondary FIFO output data is used to represent the image row data currently output by the secondary FIFO; constructing the three continuous groups of image row data streams into a preset grid sliding window, and the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels; the grayscale value information of the center pixel of the preset grid sliding window is calculated by calculating the grayscale value information of the center pixel of the preset grid sliding window. When the gray value is greater than a preset first pixel threshold and the sum of the gray values ​​of all pixels in the preset grid sliding window is greater than a preset second pixel threshold, the central pixel is determined to be a spot effective pixel, and the coordinate position of the spot effective pixel is output; in each group of image row data, the coordinate position of the first spot effective pixel in the continuous spot effective pixels is marked as the starting pixel position, and the coordinate position of the last spot effective pixel in the continuous spot effective pixels is marked as the ending pixel position; the starting pixel position and the ending pixel position in the first image row data are marked as effective boundaries, and the spot effective pixels in the first image row data are grayed out; based on the starting pixel position and the ending pixel position in the first image row data, and the starting pixel position and the ending pixel position in the second image row data, it is determined whether the spot of the first image row is continuous with the spot of the second image row; if continuous, the starting pixel position and the ending pixel position in the second image row data are marked as effective boundaries, and the spot effective pixels in the second image row data are grayed out; according to the weighted average position of the gray values ​​of all spot effective pixels, the target centroid coordinates of the target spot are calculated.

[0007] By adopting the above technical solution, firstly, the multi-spot centroid extraction device caches the image row data through the first-level FIFO and the second-level FIFO to form three sets of synchronous continuous image row data streams, which is convenient for constructing a preset grid sliding window, effectively solving the problem of data synchronization and processing rate matching, and improving data processing efficiency. Secondly, the multi-spot centroid extraction device accurately determines the effective pixels of the light spot with the help of pixel thresholds, and accurately divides different light spot areas by row continuity judgment and boundary marking, avoiding repeated identification and misidentification of the light spot, and greatly improving the accuracy of light spot identification. Finally, the multi-spot centroid extraction device calculates the target centroid coordinates of the target light spot based on the weighted average position of the grayscale values ​​of all effective pixels of the light spot, fully considering the grayscale differences inside the light spot, and more accurately reflects the energy distribution center of the light spot than conventional methods, thereby significantly improving the calculation accuracy of the light spot centroid.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the received image row data is pushed into the first-level FIFO and the second-level FIFO in sequence for caching, so that the input data and the first-level FIFO output data and the second-level FIFO output data form three consecutive groups of image row data streams that are synchronized, specifically including: receiving the image row data, pushing the image row data into the first-level FIFO, and obtaining the first-level FIFO output data; pushing the first-level FIFO output data into the second-level FIFO, and obtaining the second-level FIFO output data; receiving input data, so that the input data and the first-level FIFO output data and the second-level FIFO output data form three consecutive groups of image row data streams that are synchronized.

[0009] By adopting the above technical solution, the flow process of image row data in the FIFO cache is described in detail. The multi-spot centroid extraction device realizes the orderly caching and synchronous output of data by pressing the image row data into the first-level FIFO and the second-level FIFO in sequence. This data caching mechanism ensures the timing alignment of the three-row image data, providing a stable and reliable data source for subsequent sliding window processing. At the same time, this method also makes full use of the parallel processing capability of FPGA, so that data reading, caching and output can be carried out simultaneously, improving data processing efficiency and reducing processing delay.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the three consecutive groups of image row data streams are constructed into a preset grid sliding window, and the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels, specifically including: taking each pixel in the three consecutive groups of image row data streams as a central pixel; obtaining multiple adjacent pixels of the central pixel; taking the central pixel as the center, and constructing the preset grid sliding window according to the central pixel and the adjacent pixels, and the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels.

[0011] By adopting the above technical solution, the multi-spot centroid extraction device constructs a preset grid sliding window based on the central pixel, thereby realizing the local feature extraction of three consecutive sets of image row data streams. The preset grid sliding window can simultaneously consider the grayscale distribution around the target pixel, thereby improving the accuracy of spot detection. At the same time, the sliding mode of the preset grid sliding window ensures the continuous scanning of the entire image without missing any possible spot area.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of constructing the preset grid sliding window with the central pixel as the center according to the central pixel and the adjacent pixels, and the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels, the method further includes: if the preset grid sliding window cannot be constructed based on the central pixel and the adjacent pixels, then the preset grid sliding window is completed based on the central pixel and the adjacent pixels with the central pixel as the center of symmetry.

[0013] By adopting the above technical solution, the process of constructing a preset grid sliding window in the edge area of ​​the image is described in detail. When processing pixels close to the edge of the image, the preset grid sliding window may not be fully constructed due to insufficient adjacent pixels. The multi-spot centroid extraction device uses the central pixel as the symmetry center to perform pixel padding, ensuring that the edge area can also form a complete preset grid sliding window. This symmetrical padding method maintains the continuity and consistency of the spot detection algorithm, avoids missed detection of spots in the edge area, and does not introduce additional processing deviations.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the determination of whether the light spot of the first image row is continuous with the light spot of the second image row based on the starting pixel position and the ending pixel position in the first image row data, and the starting pixel position and the ending pixel position in the second image row data, specifically includes: determining the first left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the first image row data; determining the second left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the second image row data; if the first left and right boundary column coordinates and the second left and right boundary column coordinates meet the preset row continuity judgment condition, determining that the light spot of the first image row is continuous with the light spot of the second image row; if the first left and right boundary column coordinates and the second left and right boundary column coordinates do not meet the preset row continuity judgment condition, determining that the light spot of the first image row is discontinuous with the light spot of the second image row.

[0015] By adopting the above technical solution, the multi-spot centroid extraction device has established a strict row continuity judgment mechanism and realized the accurate segmentation of multiple rows of spots. By comparing the left and right boundary column coordinates of two adjacent rows, according to the preset row continuity judgment conditions, the multi-spot centroid extraction device can accurately judge whether the spots in different image rows belong to the same target spot. This judgment mechanism effectively solves the separation problem of multiple adjacent spots and avoids the erroneous merging of different spots. At the same time, this method can also effectively handle the situation where a single spot spans multiple rows, ensuring the integrity of the spot area. This continuity judgment method based on boundary coordinates improves the recognition accuracy of multiple spots in complex scenes and provides a reliable basis for spot area division for subsequent centroid calculations.

[0016] In combination with some embodiments of the first aspect, in some embodiments, the target center of mass coordinates of the target light spot are calculated based on the weighted average position of the grayscale values ​​of all effective pixels of the light spot, specifically including: adding the grayscale values ​​of all effective pixels of the light spot to obtain a denominator value; calculating the product of the coordinate position of each effective pixel of the light spot and the grayscale value; adding all products to obtain a numerator value; and obtaining the target center of mass coordinates of the target light spot based on the numerator value and the denominator value.

[0017] By adopting the above technical solution, the multi-spot centroid extraction device accumulates the grayscale values ​​of all effective pixels of the light spot and the product of the coordinate position and grayscale value of each effective pixel of the light spot, and obtains the accurate target centroid coordinates of the target light spot by dividing. This calculation method takes into account the contribution of the grayscale value of each effective pixel of the light spot to the centroid position, and gives a greater weight to pixels with high grayscale values, so that the calculation result is closer to the actual center position of the light spot.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of calculating the target center of mass coordinates of the target light spot based on the weighted average position of the grayscale values ​​of all effective pixels of the light spots, the method further includes: after outputting the target center of mass coordinates, graying out the target light spot to avoid repeated identification of the target light spot.

[0019] By adopting the above technical solution, the multi-spot centroid extraction device sets the target spot to gray after outputting the target centroid coordinates, thus establishing an effective spot processing marking mechanism, ensuring that the processed spots will not be repeatedly identified and calculated, effectively avoiding the problem of repeated calculations that may occur due to continuous data processing, and not only improving the processing efficiency, but also ensuring the accuracy of the results. By updating the spot status in the image row data in real time, the multi-spot centroid extraction device can correctly process multiple densely distributed spots and accurately track the processing status of each spot, so that it can reliably process multiple spots in complex scenes and improve practicality.

[0020] In a second aspect, an embodiment of the present application provides a multi-spot centroid extraction device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the multi-spot centroid extraction device to perform the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions. When the above-mentioned computer program product is run on a multi-spot centroid extraction device, the above-mentioned multi-spot centroid extraction device performs the method described in the first aspect and any possible implementation method of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions. When the instructions are executed on a multi-spot centroid extraction device, the multi-spot centroid extraction device performs the method described in the first aspect and any possible implementation method of the first aspect.

[0023] It can be understood that the multi-spot centroid extraction device provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By adopting the above technical solution, firstly, the multi-spot centroid extraction device caches the image row data through the first-level FIFO and the second-level FIFO to form three sets of synchronous continuous image row data streams, which is convenient for constructing a preset grid sliding window, effectively solving the problem of data synchronization and processing rate matching, and improving data processing efficiency. Secondly, the multi-spot centroid extraction device accurately determines the effective pixels of the light spot with the help of pixel thresholds, and accurately divides different light spot areas by row continuity judgment and boundary marking, avoiding repeated identification and misidentification of the light spot, and greatly improving the accuracy of light spot identification. Finally, the multi-spot centroid extraction device calculates the target centroid coordinates of the target light spot based on the weighted average position of the grayscale values ​​of all effective pixels of the light spot, fully considering the grayscale differences inside the light spot, and more accurately reflects the energy distribution center of the light spot than conventional methods, thereby significantly improving the calculation accuracy of the light spot centroid.

[0025] 2. By adopting the above technical solution, the multi-spot centroid extraction device has established a strict row continuity judgment mechanism and realized the accurate segmentation of multiple rows of light spots. By comparing the left and right boundary column coordinates of two adjacent rows, according to the preset row continuity judgment conditions, the multi-spot centroid extraction device can accurately judge whether the light spots in different image rows belong to the same target light spot. This judgment mechanism effectively solves the problem of separating multiple adjacent light spots and avoids the erroneous merging of different light spots. At the same time, this method can also effectively handle the situation where a single light spot spans multiple rows, ensuring the integrity of the light spot area. This continuity judgment method based on boundary coordinates improves the recognition accuracy of multiple light spots in complex scenes and provides a reliable basis for the division of light spot areas for subsequent centroid calculations.

[0026] 3. By adopting the above technical solution, the multi-spot centroid extraction device sets the target spot to gray after outputting the target centroid coordinates, thus establishing an effective spot processing marking mechanism, ensuring that the processed spots will not be repeatedly identified and calculated, effectively avoiding the problem of repeated calculations due to continuous data processing, which not only improves the processing efficiency, but also ensures the accuracy of the results. By updating the spot status in the image row data in real time, the multi-spot centroid extraction device can correctly process multiple densely distributed spots and accurately track the processing status of each spot, so that it can reliably handle multiple spots in complex scenes and improve practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application; Figure 2 is another flow chart of the multi-spot centroid extraction method based on FPGA in an embodiment of the present application; Figure 3 It is a module schematic diagram of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application; Figure 4 This is a schematic diagram of a scenario of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application; Figure 5 It is a schematic diagram of the structure of a physical device of a multi-spot centroid extraction device in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification of the present application, the singular expressions "one", "a kind of", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0030] The following is a description of the process of the method provided by this implementation. Figure 1 , is a flow chart of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application.

[0031] S101, pushing the received image row data into the primary FIFO and the secondary FIFO in sequence for caching, so that the input data and the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image row data streams, the image row data is used to represent the gray value information of each row of pixels of the image, the size of the image is determined by the product of a preset first number of rows of pixels and a preset second number of columns of pixels, the primary FIFO and the secondary FIFO each cache at least one set of image row data, the input data is used to represent the image row data currently being received, the primary FIFO output data is used to represent the image row data currently output by the primary FIFO, and the secondary FIFO output data is used to represent the image row data currently output by the secondary FIFO; Among them, image row data refers to the gray value information of each row of pixels in the image, and the gray value of each pixel is used to represent the brightness level of the pixel; the first-level FIFO and the second-level FIFO refer to two data buffers that work on the first-in-first-out principle and are used to temporarily store and synchronize data; input data refers to the latest image row data currently being received; FIFO output data refers to the cached image row data output from the FIFO data buffer; three consecutive groups of image row data streams refer to three adjacent rows of image data arranged in time sequence.

[0032] Specifically, first, the multi-spot centroid extraction device receives image row data containing gray value information, pushes the image row data into the first-level FIFO for caching, and the data output by the first-level FIFO is further pushed into the second-level FIFO for caching. Through this dual-FIFO cascade caching mechanism, the current input data, the previous row of data output by the first-level FIFO, and the previous row of data output by the second-level FIFO are completely aligned in time sequence, forming a stable continuous three-group image row data stream, which provides a basis for subsequent sliding window processing.

[0033] Optionally, under normal circumstances, the received image row data is sequentially pushed into the primary FIFO and the secondary FIFO for caching, so that the input data and the primary FIFO output data and the secondary FIFO output data form three consecutive groups of image row data streams that are synchronized. This can be achieved in the following manner, which is not limited here: receiving the image row data, pushing the image row data into the primary FIFO, and obtaining the primary FIFO output data; pushing the primary FIFO output data into the secondary FIFO, and obtaining the secondary FIFO output data; receiving the input data, so that the input data and the primary FIFO output data and the secondary FIFO output data form three consecutive groups of image row data streams that are synchronized.

[0034] S102, constructing the three consecutive groups of image row data streams into a preset grid sliding window, where the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels; Among them, the preset grid sliding window refers to a rectangular pixel area composed of a predetermined number of rows and a predetermined number of columns, which is used for local feature analysis; three consecutive groups of image row data streams refer to three adjacent rows of image data arranged in time sequence; the preset third number of rows represents the number of pixel rows contained in the preset grid sliding window in the vertical direction, which is usually an odd number to ensure the uniqueness of the center pixel; the preset fourth number of columns represents the number of pixel columns contained in the preset grid sliding window in the horizontal direction, which is usually equal to the number of rows to form a square window; the product determines the total number of pixels in the preset grid sliding window, which is used to specify the size of the local analysis area.

[0035] Specifically, first, the multi-spot centroid extraction device determines the size of the preset grid sliding window to be constructed according to the size parameters of the preset grid sliding window (the preset third number of rows and the preset fourth number of columns), such as a 3×3 or 5×5 rectangular window. Then, the multi-spot centroid extraction device starts from the leftmost of the three consecutive groups of image row data streams, and organizes the pixel data at the corresponding position into the preset grid sliding window according to the predetermined row and column structure to form a complete local analysis unit. The construction process of the preset grid sliding window needs to ensure that the position correspondence of the pixel data is correct, that is, the relative positions of the upper row data, the middle row data and the lower row data in the preset grid sliding window remain unchanged. Once the preset grid sliding window is constructed, it can slide pixel by pixel in the horizontal direction, and the window data is reorganized every time it moves a position, thereby realizing continuous scanning and analysis of the entire image.

[0036] S103, when the grayscale value of the central pixel of the preset grid sliding window is greater than the preset first pixel threshold and the sum of the grayscale values ​​of all pixels of the preset grid sliding window is greater than the preset second pixel threshold, determine that the central pixel is a light spot effective pixel, and output the coordinate position of the light spot effective pixel; Among them, the center pixel refers to the pixel point at the center position of the preset grid sliding window; the preset first pixel threshold refers to the gray value standard used to judge whether a single pixel point is bright enough; the preset second pixel threshold refers to the gray value sum standard used to judge the overall brightness of the local area; the light spot effective pixel refers to the pixel point confirmed to belong to the light spot area.

[0037] Specifically, first, the multi-spot centroid extraction device checks whether the grayscale value of the central pixel of the preset grid sliding window exceeds the preset first pixel threshold to ensure that the pixel has sufficient brightness. At the same time, the multi-spot centroid extraction device calculates the sum of the grayscale values ​​of all pixels in the preset grid sliding window and compares it with the preset second pixel threshold to ensure that the overall brightness of the local area meets the requirements. When the above two conditions are met at the same time, the multi-spot centroid extraction device can identify the central pixel as a valid pixel of the light spot and record its coordinate position.

[0038] S104, in each group of image row data, marking the coordinate position of the first effective pixel of the continuous effective pixels of the light spots as the starting pixel position, and marking the coordinate position of the last effective pixel of the continuous effective pixels of the light spots as the ending pixel position; Among them, continuous spot effective pixels refer to the set of adjacent pixel points in the same row that are confirmed as spot effective pixels; the starting pixel position refers to the starting position coordinates of each segment of continuous spot effective pixels; and the ending pixel position refers to the ending position coordinates of each segment of continuous spot effective pixels.

[0039] Specifically, the multi-spot centroid extraction device analyzes the effective pixels of the spots in each group of image row data to find the continuity of the pixels in the horizontal direction. When continuous effective pixels of the spots are detected, the coordinates of the effective pixels of the leftmost spot are recorded as the starting pixel position, and the coordinates of the effective pixels of the rightmost spot are recorded as the ending pixel position, so as to determine the horizontal range of each spot in the row.

[0040] S105, marking the starting pixel position and the ending pixel position in the first image row data as valid boundaries, and graying out the effective pixels of the light spot in the first image row data; Among them, the first image row data refers to the image row currently being processed; the effective boundary refers to the start and end position marks of the pixel area that has been confirmed to belong to the same light spot; graying refers to modifying the grayscale value of the processed effective pixels of the light spot to the background value to avoid repeated processing; the effective pixels of the light spot refer to the pixel points confirmed to belong to the light spot area after analysis through the sliding preset grid sliding window.

[0041] Specifically, the multi-spot centroid extraction device marks the determined starting pixel position and ending pixel position as the effective boundary of the row of spots for subsequent spot continuity judgment. At the same time, in order to avoid repeated processing of the same spot, the grayscale values ​​of all confirmed effective pixels of the spots in the row are modified to the background grayscale value. This dual mechanism of marking and graying not only retains the location information of the spots, but also prevents repeated calculations, thereby improving processing efficiency and accuracy.

[0042] S106, judging whether the light spot of the first image row is continuous with the light spot of the second image row based on the starting pixel position and the ending pixel position in the first image row data and the starting pixel position and the ending pixel position in the second image row data; The second image row data refers to the next image row data adjacent to the first image row; and continuity refers to whether the light spots of two adjacent rows belong to the same complete light spot area.

[0043] Specifically, the multi-spot centroid extraction device determines whether they belong to the same spot by comparing the starting pixel position and the ending pixel position of the effective pixels of the spots in the first image row data and the second image row data. The judgment criteria usually include: whether the horizontal positions of the two rows of spots overlap, whether the size of the overlapping area exceeds a preset threshold, whether the width ratio of the two rows of spots is within a reasonable range, etc. Through the comprehensive judgment of these conditions, the complete spot area spanning multiple rows can be accurately identified.

[0044] Optionally, in general, based on the starting pixel position and the ending pixel position in the first image row data, and the starting pixel position and the ending pixel position in the second image row data, determining whether the light spot of the first image row is continuous with the light spot of the second image row can be achieved in the following manner, which is not limited here: determining the first left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the first image row data; determining the second left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the second image row data; if the first left and right boundary column coordinates and the second left and right boundary column coordinates meet the preset row continuity judgment condition, then determining that the light spot of the first image row is continuous with the light spot of the second image row; if the first left and right boundary column coordinates and the second left and right boundary column coordinates do not meet the preset row continuity judgment condition, then determining that the light spot of the first image row is discontinuous with the light spot of the second image row.

[0045] The preset row continuity judgment conditions are: ; Wherein, X1a and X2a represent the left boundary column coordinates of the light spot of the first image row and the second image row respectively; X1b and X2b represent the right boundary column coordinates of the light spot of the first image row and the second image row respectively. When the preset row continuity judgment condition is met, it means that there is an overlapping area between the two rows of light spots, that is, the light spot rows are continuous; when the preset row continuity judgment condition is not met, it means that there is no overlapping area between the two rows of light spots, that is, the light spot rows are discontinuous.

[0046] S107, if they are continuous, marking the starting pixel position and the ending pixel position in the second image row data as valid boundaries, and graying out the valid pixels of the light spot in the second image row data; Among them, continuity means meeting the preset light spot continuity judgment standard; marking as a valid boundary means adding the boundary information of the valid pixels of the row of light spots to the complete light spot area currently being processed; graying means modifying the grayscale value of the pixel points confirmed to belong to the current light spot to the background value.

[0047] Specifically, the multi-spot centroid extraction device adds the boundary information of the spot effective pixels of the second image row to the spot area currently being processed, and continues to construct a complete spot contour. At the same time, all the spot effective pixels in the second image row are grayed out to avoid repeated calculation or being wrongly divided into other spots, thereby ensuring the integrity and uniqueness of the spot area.

[0048] S108. Calculate the target centroid coordinates of the target light spot according to the weighted average position of the gray values ​​of all effective pixels of the light spot.

[0049] Among them, the weighted average position of gray value is the weighted calculation result that takes into account the contribution of the gray value of each pixel; the target light spot refers to a single light spot area that has been completely identified; the target centroid coordinates represent the precise coordinate values ​​of the center of gravity of the target light spot; and the weighted calculation refers to determining the weight of the pixel in the centroid calculation according to the size of the pixel gray value.

[0050] Specifically, the multi-spot centroid extraction device first calculates the sum of the grayscale values ​​of all effective pixels of the spot in the spot area as the denominator, then calculates the product of the coordinate position of each effective pixel of the spot and its grayscale value and adds them up as the numerator, and finally divides the numerator by the denominator to obtain the weighted average position, which is the centroid coordinate of the spot. This calculation method based on grayscale value weighting takes into account the distribution characteristics of pixel brightness and can more accurately reflect the actual center position of the spot, which is particularly suitable for spots with irregular shapes or non-uniform brightness distribution.

[0051] Optionally, in general, the target center of mass coordinates of the target light spot can be calculated based on the weighted average position of the grayscale values ​​of all effective pixels of the light spot in the following manner, which is not limited here: the grayscale values ​​of all effective pixels of the light spot are added to obtain a denominator value; the product of the coordinate position of each effective pixel of the light spot and the grayscale value is calculated; all products are added to obtain a numerator value; based on the numerator value and the denominator value, the target center of mass coordinates of the target light spot are obtained.

[0052] By adopting the above technical solution, firstly, the multi-spot centroid extraction device caches the image row data through the first-level FIFO and the second-level FIFO to form three sets of synchronous continuous image row data streams, which is convenient for constructing a preset grid sliding window, effectively solving the problem of data synchronization and processing rate matching, and improving data processing efficiency. Secondly, the multi-spot centroid extraction device accurately determines the effective pixels of the light spot with the help of pixel thresholds, and accurately divides different light spot areas by row continuity judgment and boundary marking, avoiding repeated identification and misidentification of the light spot, and greatly improving the accuracy of light spot identification. Finally, the multi-spot centroid extraction device calculates the target centroid coordinates of the target light spot based on the weighted average position of the grayscale values ​​of all effective pixels of the light spot, fully considering the grayscale differences inside the light spot, and more accurately reflects the energy distribution center of the light spot than conventional methods, thereby significantly improving the calculation accuracy of the light spot centroid.

[0053] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the multi-spot centroid extraction method based on FPGA in an embodiment of the present application.

[0054] S201, pushing the received image row data into the primary FIFO and the secondary FIFO in sequence for caching, so that the input data and the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image row data streams, the image row data is used to represent the gray value information of each row of pixels of the image, the size of the image is determined by the product of a preset first number of rows of pixels and a preset second number of columns of pixels, the primary FIFO and the secondary FIFO each cache at least one set of image row data, the input data is used to represent the image row data currently being received, the primary FIFO output data is used to represent the image row data currently output by the primary FIFO, and the secondary FIFO output data is used to represent the image row data currently output by the secondary FIFO; For details, please refer to step S101, which will not be described in detail here.

[0055] S202, taking each pixel in the three consecutive groups of image row data streams as a central pixel; Among them, three consecutive groups of image row data streams represent three groups of adjacent image row data after FIFO cache synchronization; pixel refers to the smallest unit that constitutes a digital image, including position and grayscale value information; the center pixel represents the target pixel point currently selected as the center of the preset grid sliding window; the grayscale value information is used to represent the brightness level of the pixel point.

[0056] Specifically, the multi-spot centroid extraction device sets each pixel point in three consecutive groups of image row data streams as the center pixel in order from left to right and from top to bottom. For each row, the multi-spot centroid extraction device starts from the first pixel and traverses to the last pixel one by one, ensuring that all pixels have the opportunity to be analyzed and processed as the center point of the preset grid sliding window, thereby ensuring the integrity and continuity of image processing.

[0057] S203, acquiring a plurality of adjacent pixels of the central pixel; Specifically, the multi-spot centroid extraction device determines the adjacent pixels based on the central pixel. The adjacent pixels refer to the surrounding pixels directly connected to the central pixel in space, usually including the pixels above, below, left, right and four diagonal directions of the central pixel. These adjacent pixels together with the central pixel constitute the basic unit of local feature analysis.

[0058] S204, taking the central pixel as the center, constructing the preset grid sliding window according to the central pixel and the adjacent pixels, where the preset grid sliding window is determined by multiplying pixels of a preset third number of rows by pixels of a preset fourth number of columns; Among them, the preset grid sliding window refers to a rectangular pixel area with a fixed size; the preset third number of rows represents the number of pixels in the vertical direction of the preset grid sliding window; the preset fourth number of columns represents the number of pixels in the horizontal direction of the preset grid sliding window.

[0059] Specifically, first, the multi-spot centroid extraction device places the central pixel at the center of the window, and then constructs a rectangular window according to the preset third number of rows and the preset fourth number of columns. Usually, the window size is an odd number such as 3×3, 5×5, etc., to ensure that the central pixel is located in the center of the window. The construction process needs to ensure the integrity and orderliness of the pixels in the preset grid sliding window, so that the window can accurately reflect the local image features.

[0060] S205: If the preset grid sliding window cannot be constructed according to the central pixel and the adjacent pixels, the preset grid sliding window is completed according to the central pixel and the adjacent pixels with the central pixel as the symmetry center; Among them, unable to construct refers to the state where the central pixel is located at the edge of the image, resulting in insufficient adjacent pixels; the symmetry center refers to the reference point when performing pixel padding; the padding operation means filling the missing window pixels according to specific rules; pixel mirroring refers to a symmetrical filling method with the central pixel as the axis.

[0061] Specifically, the multi-spot centroid extraction device first determines the position and number of missing pixels, and then uses the central pixel as the axis of symmetry to map the existing adjacent pixels to the missing position in a mirrored manner. For example, when the central pixel is located at the left edge of the image, the existing pixels on the right can be filled in the missing position on the left in a left-right symmetrical manner. This symmetry-based filling method maintains the local features of the image, avoids the particularity of edge processing, and enables edge pixels to obtain complete window analysis.

[0062] S206, when the grayscale value of the central pixel of the preset grid sliding window is greater than the preset first pixel threshold and the sum of the grayscale values ​​of all pixels of the preset grid sliding window is greater than the preset second pixel threshold, determine that the central pixel is a light spot effective pixel, and output the coordinate position of the light spot effective pixel; For details, please refer to step S103, which will not be described in detail here.

[0063] S207, in each group of image row data, marking the coordinate position of the first effective pixel of the continuous effective pixels of the light spots as the starting pixel position, and marking the coordinate position of the last effective pixel of the continuous effective pixels of the light spots as the ending pixel position; For details, please refer to step S104, which will not be described in detail here.

[0064] S208, marking the starting pixel position and the ending pixel position in the first image row data as valid boundaries, and graying out the effective pixels of the light spot in the first image row data; For details, please refer to step S105, which will not be described in detail here.

[0065] S209, judging whether the light spot of the first image row is continuous with the light spot of the second image row based on the starting pixel position and the ending pixel position in the first image row data and the starting pixel position and the ending pixel position in the second image row data; For details, please refer to step S106, which will not be described in detail here.

[0066] S210, if they are continuous, marking the starting pixel position and the ending pixel position in the second image row data as valid boundaries, and graying out the valid pixels of the light spot in the second image row data; For details, please refer to step S107, which will not be described in detail here.

[0067] S211, calculating the target centroid coordinates of the target light spot according to the weighted average position of the gray values ​​of all effective pixels of the light spot; For details, please refer to step S108, which will not be described in detail here.

[0068] S212, after outputting the target centroid coordinates, graying out the target light spot to avoid repeated recognition of the target light spot.

[0069] Among them, the target centroid coordinates refer to the centroid position information of the target light spot that has been calculated, including the precise position values ​​in the horizontal and vertical directions; the target light spot refers to the complete light spot area from which the target centroid coordinates have been extracted; the graying operation means changing the grayscale values ​​of all pixels in the target light spot to the background grayscale value; repeated identification refers to the situation where the same light spot area is detected and calculated multiple times; the light spot area refers to the set of all continuous pixels that have been confirmed to belong to the same light spot.

[0070] Specifically, first, the multi-spot centroid extraction device determines the spot area that needs to be grayed out, that is, all the effective pixels of the spots that have been marked as belonging to the target spot. Then, the multi-spot centroid extraction device uniformly modifies the grayscale values ​​of these effective pixels of the spots to the same grayscale value as the image background, and usually selects a lower grayscale value to ensure a clear contrast with other unprocessed spots. This graying process not only marks the processed spot area, but also fundamentally prevents the spot area from being repeatedly detected by changing the pixel grayscale value. This processing method is particularly important for multiple adjacent or partially overlapping spots, because it can ensure that each spot is processed only once, thereby improving the processing accuracy and efficiency in multi-spot scenes.

[0071] The following is a module description of the method provided by this implementation. Figure 3 , which is a module diagram of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application.

[0072] There are multiple spot screening modules in the multi-spot centroid extraction device, which process the image data stream in turn. After processing by the spot screening module 1, a filtered image data stream is output as the input of the spot screening module 2. And so on, the image data stream passes through the spot screening module 1, the spot screening module 2, the spot screening module 3... until the spot screening module N, thereby realizing the multi-spot centroid extraction in the entire image as a whole.

[0073] The following is a scenario description of the method provided by this implementation. Figure 4 , is a scene diagram of a multi-spot centroid extraction method based on FPGA in an embodiment of the present application.

[0074] like Figure 4 (a) is an unprocessed image, which contains multiple light spots. Figure 4 (b) is a partially processed image, in which the grayed-out areas represent processed light spots, and the non-grayed-out areas represent unprocessed light spots.

[0075] The following describes the multi-spot centroid extraction device in the embodiment of the present invention from the perspective of hardware processing. Figure 5 , is a schematic diagram of a physical device structure of a multi-spot centroid extraction device in an embodiment of the present application.

[0076] It should be noted that Figure 5 The structure of the multi-spot centroid extraction device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0077] like Figure 5 As shown, the multi-spot centroid extraction device includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method described in the above embodiment. In RAM 503, various programs and data required for system operation are also stored. CPU 501, ROM 502 and RAM 503 are connected to each other through bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0078] The following components are connected to the I / O interface 505: an input section 506 including an audio input device, a button switch, etc.; an output section 507 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0079] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the present invention are performed.

[0080] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0082] Specifically, the multi-spot centroid extraction device of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the multi-spot centroid extraction method based on FPGA provided in the above embodiment is implemented.

[0083] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the multi-spot centroid extraction device described in the above embodiment; or may exist independently without being assembled into the multi-spot centroid extraction device. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the multi-spot centroid extraction device, the multi-spot centroid extraction device implements the FPGA-based multi-spot centroid extraction method provided in the above embodiment.

[0084] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0085] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0086] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A multi-spot centroid extraction method based on FPGA, characterized in that: Applied to a multi-spot centroid extraction device, the method comprises: The received image row data are sequentially pushed into the primary FIFO and the secondary FIFO for caching, so that the input data and the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image row data streams, the image row data are used to represent the gray value information of each row of pixels of the image, the size of the image is determined by the product of a preset first number of rows of pixels and a preset second number of columns of pixels, the primary FIFO and the secondary FIFO each cache at least one set of image row data, the input data are used to represent the image row data currently being received, the primary FIFO output data are used to represent the image row data currently output by the primary FIFO, and the secondary FIFO output data are used to represent the image row data currently output by the secondary FIFO; constructing the three consecutive groups of image row data streams into a preset grid sliding window, wherein the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels; When the grayscale value of the central pixel of the preset grid sliding window is greater than a preset first pixel threshold and the sum of the grayscale values ​​of all pixels of the preset grid sliding window is greater than a preset second pixel threshold, the central pixel is determined to be a light spot effective pixel, and the coordinate position of the light spot effective pixel is output; In each group of image row data, the coordinate position of the first effective pixel of the continuous effective pixels of the light spots is marked as the starting pixel position, and the coordinate position of the last effective pixel of the continuous effective pixels of the light spots is marked as the ending pixel position; Marking the starting pixel position and the ending pixel position in the first image row data as valid boundaries, and graying out the effective pixels of the light spot in the first image row data; Based on the starting pixel position and the ending pixel position in the first image row data, and the starting pixel position and the ending pixel position in the second image row data, determining whether the light spot of the first image row is continuous with the light spot of the second image row; If they are continuous, the starting pixel position and the ending pixel position in the second image row data are marked as valid boundaries, and the effective pixels of the light spot in the second image row data are grayed out; The target centroid coordinates of the target light spot are calculated based on the weighted average position of the gray values ​​of all effective pixels of the light spot.

2. The method according to claim 1, characterized in that The received image row data is sequentially pushed into the primary FIFO and the secondary FIFO for buffering, so that the input data and the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image row data streams, specifically including: receiving the image row data, pushing the image row data into a first-level FIFO, and obtaining first-level FIFO output data; Pushing the primary FIFO output data into the secondary FIFO to obtain the secondary FIFO output data; Input data is received so that the input data, the primary FIFO output data and the secondary FIFO output data form three sets of synchronous continuous image line data streams.

3. The method according to claim 1, characterized in that The step of constructing the three consecutive groups of image row data streams into a preset grid sliding window, wherein the preset grid sliding window is determined by multiplying pixels of a preset third number of rows by pixels of a preset fourth number of columns, specifically includes: Taking each pixel in the three consecutive groups of image row data streams as a central pixel; Acquire a plurality of adjacent pixels of the central pixel; Taking the central pixel as the center, the preset grid sliding window is constructed according to the central pixel and the adjacent pixels, and the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels.

4. The method according to claim 3, characterized in that After the step of constructing the preset grid sliding window based on the central pixel and the adjacent pixels with the central pixel as the center, wherein the preset grid sliding window is determined by the product of a preset third number of rows of pixels and a preset fourth number of columns of pixels, the method further includes: If the preset grid sliding window cannot be constructed according to the central pixel and the adjacent pixels, the preset grid sliding window is completed according to the central pixel and the adjacent pixels with the central pixel as the symmetry center.

5. The method according to claim 1, characterized in that The determining whether the light spot of the first image row is continuous with the light spot of the second image row based on the starting pixel position and the ending pixel position in the first image row data and the starting pixel position and the ending pixel position in the second image row data specifically includes: Determine first left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the first image row data; Determine the second left and right boundary column coordinates according to the starting pixel position and the ending pixel position in the second image row data; If the first left-right boundary column coordinates and the second left-right boundary column coordinates meet a preset row continuity judgment condition, it is determined that the light spot of the first image row is continuous with the light spot of the second image row; If the first left-right boundary column coordinates and the second left-right boundary column coordinates do not meet the preset row continuity judgment condition, it is determined that the light spot of the first image row is discontinuous with the light spot of the second image row.

6. The method according to claim 1, characterized in that The target centroid coordinates of the target light spot are calculated based on the weighted average position of the gray values ​​of all effective pixels of the light spot, specifically including: Accumulate the grayscale values ​​of all effective pixels of the light spot to obtain the denominator value; Calculate the product of the coordinate position of each effective pixel of the light spot and the gray value; Add up all the products to get the numerator value; Based on the numerator value and the denominator value, the target centroid coordinates of the target light spot are obtained.

7. The method according to claim 1, characterized in that After the step of calculating the target centroid coordinates of the target light spot according to the weighted average position of the gray values ​​of all effective pixels of the light spot, the method further includes: After the target centroid coordinates are output, the target light spot is grayed out to avoid repeated identification of the target light spot.

8. A multi-spot centroid extraction device, characterized in that: The multi-spot centroid extraction device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the multi-spot centroid extraction device to perform the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a multi-spot centroid extraction device, the multi-spot centroid extraction device is enabled to perform the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a multi-spot centroid extraction device, the multi-spot centroid extraction device is enabled to perform the method according to any one of claims 1 to 7.

Images