FPGA (Field Programmable Gate Array)-based weighted centroid extraction method and device for grayscale image

Abstract

The invention discloses a weighted centroid extraction method and device for a gray level image based on an FPGA, and belongs to the technical field of image processing, and the method comprises the steps: calculating a brightness threshold value of a target in an image through a CPU or a GPU, receiving image data needing to be subjected to weighted centroid calculation and a line field signal of the image through an optical fiber, and carrying out the weighted centroid extraction through the optical fiber; and reading a target brightness threshold in the register by the FPGA to extract a feature number and remove isolated points, and taking a pixel value gray value of an effective target pixel as a weight to be substituted into a weighted centroid coordinate formula for calculation to obtain a weighted centroid coordinate of the target. According to the invention, the characteristics of rich logic resources, high speed and high performance of the FPGA are fully exerted, and the real-time performance and reliability of image data processing are guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a weighted centroid extraction method and apparatus for grayscale images based on FPGA. Background Technology

[0002] Centroid extraction is a key technology for accurately locating the geometric center of an object by calculating the weighted average position of pixels in the target region. It is widely used in optical tracking (such as laser guidance and stellar positioning), industrial inspection (robot grasping and defect analysis), and medical imaging (cell tracking). Centroid extraction transforms complex image or signal data into high-precision position coordinates, providing fundamental data support for subsequent target recognition, motion analysis, and closed-loop control, directly impacting the system's measurement accuracy and response speed. However, the real-time hardware implementation of centroid extraction faces numerous challenges, and existing technologies suffer from inherent defects and performance bottlenecks to varying degrees.

[0003] Firstly, there are real-time bottlenecks and response lag issues. The processing of large amounts of image data and complex weighted operations (involving numerous multiply-accumulate operations) introduce significant processing latency. In general-purpose processor (CPU)-based systems, image data requires multiple memory transfers and complex instruction scheduling, severely limiting data throughput. In graphics processing unit (GPU) solutions, the inherent pipeline latency and data migration latency between the host and video memory are equally significant. These latency issues directly lead to system response lag, making it difficult to meet the stringent real-time requirements of high-speed dynamic scenes, ultimately affecting the accuracy and stability of target tracking.

[0004] At the same time, the performance and flexibility of a single computing platform are also limited, and any single type of computing platform faces irreconcilable contradictions when handling this task.

[0005] CPUs are limited by their inherent serial execution mode, resulting in extremely low parallelism when processing massive amounts of pixel data. Their performance is severely constrained by memory bandwidth and they have poor energy efficiency.

[0006] While GPUs possess powerful parallel computing capabilities, their architecture is optimized for large-scale parallel computing. When processing large-scale image algorithms of this type, they suffer from excessive power consumption and significant latency issues.

[0007] Field Programmable Gate Arrays (FPGAs) are programmable signal processing devices with abundant logic resources. They are reprogrammable, allowing users to customize configuration information to define functions according to design requirements.

[0008] Image processing is the technology of analyzing and manipulating images to improve their quality or extract useful information. It is widely used in fields such as medical imaging, autonomous driving, surveillance systems, industrial inspection, and facial recognition. Real-time processing and efficient computation are crucial in these applications, which is precisely where FPGAs excel.

[0009] FPGAs possess high parallel processing capabilities and customizability, enabling the implementation of complex image processing algorithms at the hardware level, such as edge detection, image filtering, and feature extraction. This makes FPGAs excel at processing large-scale data, significantly reducing processing latency and increasing data throughput. Simultaneously, the parallel architecture of FPGAs allows multiple processing units to operate concurrently, adapting to various image processing tasks and thus improving the overall system efficiency. Furthermore, FPGAs can be programmed and configured to meet specific needs, allowing developers to optimize performance in different application scenarios without redesigning the hardware.

[0010] Compared to traditional processors, FPGAs offer advantages in power consumption and cost, especially in applications requiring long-term operation and high real-time performance, where they provide a higher cost-performance ratio. Therefore, FPGAs offer a flexible, efficient, and economical solution for image processing applications, making them an ideal choice for implementing image processing algorithms.

[0011] While FPGA solutions can achieve extremely high computing performance through hardware parallelism, once the algorithm logic is fixed, real-time parameter updates are difficult (e.g., dynamically adjusting weight models or algorithm flows requires resynthesis, placement, and routing, which is extremely time-consuming). Furthermore, large systems also face the challenge of re-fixing FPGA programs, resulting in limited adaptability to different application scenarios and a heavy reliance on frequent manual intervention by engineers. In complex field environments (such as changes in lighting, noise types, and target characteristics), rapid online adaptive adjustments are difficult to achieve, reducing the system's intelligence level and availability.

[0012] In summary, developing a hardware implementation scheme for weighted centroid extraction that combines ultra-high real-time performance, high precision, low power consumption, and good adaptability has become an urgent need to drive technological development in related fields. FPGA-based hardware acceleration schemes, with their powerful parallel processing capabilities, flexible hardware programmability, and potential to form heterogeneous computing with the CPU as coprocessors, provide an ideal technical path to simultaneously solve the aforementioned technical challenges. This research aims to explore an FPGA-based hardware architecture specifically optimized for grayscale image weighted centroid algorithms to overcome many bottlenecks in existing technologies. Summary of the Invention

[0013] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0014] A weighted centroid extraction method for grayscale images based on FPGA, comprising:

[0015] Step 1: Use the CPU or GPU to calculate the target brightness threshold in real time and write it into the register, and then use the FPGA to read the target brightness threshold from the register.

[0016] Step 2: The line and field valid signals of the image and the image data are input into the FPGA;

[0017] Step 3: The FPGA receives image data, segments the image data by pixels, splits the image data into pixel grayscale values, and caches the pixel grayscale values ​​of each pixel into a register.

[0018] Step 4: Calculate the image resolution using the effective line and field signals of the image, under the tick of the clock signal;

[0019] Step 5: Define the top-left corner of the image as the origin (0, 0), extend the x-axis to the right, and extend the y-axis downwards, marking the coordinates of each point as follows. ;

[0020] Step 6: Identify the pixels whose grayscale values ​​fall between the target brightness and darkness thresholds as the corresponding pixels of the target.

[0021] Step 7: Mark each pixel;

[0022] Step 8: Use feature number A to remove certain pixel grayscale values ​​that fall within the target brightness threshold but are actually isolated pixels, thus obtaining the final effective target pixel grayscale values. ;

[0023] Step 9: Convert the final effective target pixel grayscale value respectively with its x-axis coordinate y-axis coordinate Multiply them, then calculate their cumulative sum to obtain the weighted sum in the x and y directions;

[0024] Step 10: Calculate the sum of gray values ​​for the entire image;

[0025] Step 11: Perform a division operation using the sum of the grayscale values ​​of the entire image as the divisor and the weighted sum along the x and y axes as the dividend, to obtain ( () represents the final centroid coordinates.

[0026] A weighted centroid extraction device for grayscale images based on FPGA, comprising:

[0027] The target brightness threshold acquisition module uses CPU or GPU to calculate the target brightness threshold in real time and write it into a register, and then the FPGA reads the target brightness threshold from the register.

[0028] The image data acquisition module transmits the effective line and field signals and image data of the image to the FPGA;

[0029] The segmentation module receives image data from the FPGA, segments the image data by pixels, breaks down the image data into pixel grayscale values, and caches the pixel grayscale values ​​of each pixel in a register.

[0030] The image resolution acquisition module calculates the image resolution by using the effective line and field signals of the image in the clock cycle.

[0031] The first labeling module defines the top-left corner of the image as the origin (0,0), extends the x-axis to the right, and the y-axis to the bottom, labeling the coordinates of each point as follows: ;

[0032] The pixel identification module identifies pixels whose grayscale values ​​fall between the target brightness and darkness thresholds as the corresponding pixels of the target.

[0033] The second marking module marks each pixel.

[0034] The target pixel grayscale value acquisition module uses feature number A to eliminate certain pixels whose grayscale values ​​fall within the target brightness threshold but are actually isolated pixels, thus obtaining the final effective target pixel grayscale values. ;

[0035] The weighted sum calculation module calculates the final effective target pixel grayscale value. respectively with its x-axis coordinate y-axis coordinate Multiply them, then calculate their cumulative sum to obtain the weighted sum in the x and y directions;

[0036] The summation module calculates the sum of gray values ​​across the entire image.

[0037] The centroid extraction module uses the sum of the grayscale values ​​of the entire image as the divisor and the weighted sum of the values ​​along the x and y axes as the dividend to perform a division operation, resulting in (…). () represents the final centroid coordinates.

[0038] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the FPGA-based weighted centroid extraction method for grayscale images.

[0039] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the FPGA-based weighted centroid extraction method for grayscale images.

[0040] The present invention has the following beneficial effects:

[0041] This invention utilizes CPU or GPU calculation results, camera format (taking a 4-tap 64-bit camera as an example), and camera acquisition data (line and field effective signals of the image, image data) to automatically generate brightness and darkness thresholds using CPU. This achieves high autonomy and flexibility without manual parameter tuning, and the parameters can be updated in real time according to the target status. It solves the problem in the prior art that parameters such as target brightness and darkness thresholds, image resolution, and target size rely on manual settings and are not updated in a timely manner.

[0042] This invention significantly improves the real-time response capability of the system by using FPGA for real-time data processing at the image input end, providing an efficient data foundation for subsequent target extraction and image processing; it also solves the problem of large image preprocessing delays and the impact on overall processing speed in existing technologies.

[0043] This invention achieves seamless integration of data flow and control flow by closely combining parameter generation with image acquisition, enabling dynamic adaptation under different target states and environmental conditions; it solves the problems of poor system adaptability and frequent manual intervention in the prior art.

[0044] This invention adopts a modular hardware architecture to enable collaborative processing between FPGA and CPU or GPU, achieving a combination of hardware parallel acceleration and flexible software scheduling; it solves the bottleneck problem of performance or flexibility of a single computing platform in the centroid calculation process in the prior art.

[0045] This invention introduces a real-time monitoring and feedback mechanism during parameter calculation and updating: when the target shape changes, the CPU calculates the target brightness threshold in real time and transmits it to the FPGA, and the target centroid can also change accordingly, ensuring the accuracy of the target centroid. This enables the system to adjust the processing strategy in real time according to the target changes during operation; and solves the problems of system response lag and insufficient target tracking accuracy in the prior art. Attached Figure Description

[0046] Figure 1 This is a flowchart of the FPGA-based weighted centroid extraction method for grayscale images according to the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0048] This invention provides a weighted centroid extraction method for grayscale images based on FPGA, comprising:

[0049] Step 1: Use the CPU or GPU to calculate the target brightness threshold in real time and write it into the register, and then use the FPGA to read the target brightness threshold from the register.

[0050] Step 2: The line and field valid signals of the image and the image data are input into the FPGA;

[0051] Step 3: The FPGA receives image data, segments the image data by pixels, splits the image data into pixel grayscale values, and caches the pixel grayscale values ​​of each pixel into a register.

[0052] Step 4: Calculate the image resolution using the effective line and field signals of the image, under the tick of the clock signal;

[0053] Step 5: Define the top-left corner of the image as the origin (0,0), extend the x-axis to the right, and extend the y-axis downwards, marking the coordinates of each point as follows. ;

[0054] Step 6: Identify the pixels whose grayscale values ​​fall between the target brightness and darkness thresholds as the corresponding pixels of the target.

[0055] Step 7: Mark each pixel;

[0056] Step 8: Use feature number A to remove certain pixel grayscale values ​​that fall within the target brightness threshold but are actually isolated pixels, thus obtaining the final effective target pixel grayscale values. ;

[0057] Step 9: Convert the final effective target pixel grayscale value respectively with its x-axis coordinate y-axis coordinate Multiply them, then calculate their cumulative sum to obtain the weighted sum in the x and y directions;

[0058] Step 10: Calculate the sum of gray values ​​for the entire image;

[0059] Step 11: Perform a division operation using the sum of the grayscale values ​​of the entire image as the divisor and the weighted sum along the x and y axes as the dividend, to obtain ( () represents the final centroid coordinates.

[0060] The technical solution of the present invention will be further described below with reference to specific embodiments. For example... Figure 1 As shown, this invention provides a weighted centroid extraction method for grayscale images based on FPGA, which fully leverages the abundant logic resources, high speed and high performance of FPGA, ensures the real-time performance and reliability of image data processing, and provides greater flexibility for subsequent image processing.

[0061] Step 1: Use the CPU or GPU to calculate the target brightness threshold in real time and write it into the register. Then, the FPGA reads the target brightness threshold from the register to ensure that the target brightness threshold can be updated in real time.

[0062] Step 2: The effective line and field signals of the image and the image data are transmitted to the FPGA via optical fiber in a 4-tap (a data readout method for camera sensors, which divides the sensor's pixel array into 1, 2, 4, or 8 parallel output channels, each channel simultaneously reading the pixel data it is responsible for; the parallel output channels are called "tap") data format (other formats can also be applied; step 3 requires splitting according to different formats). A specific implementation uses a 4-tap 64-bit camera as an example, but 1-tap, 2-tap, and 8-tap cameras are also possible.

[0063] Step 3: The FPGA receives the image data transmitted via the optical fiber, segments the image data by pixel, and splits the incoming 4-tap 64-bit image data into four 16-bit pixel grayscale values. The grayscale value of each pixel is then cached in four registers. , , , If using 1-tap, 2-tap, or 8-tap cameras, the image data is split into other corresponding numbers of pixels.

[0064] Step 4: Using the effective line and field signals of the image, calculate the image resolution (including image width and height) under the clock signal's beat, where the image width is... Image height is .

[0065] Step 5: Define the top-left corner of the image as the origin (0,0), extend the x-axis to the right (column index), and extend the y-axis downwards (row index), marking the coordinates of each point as... .

[0066] Step 6: Compare the pixel grayscale value of each pixel with the target brightness threshold transmitted in real time by the CPU or GPU, and determine the pixel whose pixel grayscale value is between the target brightness threshold as the pixel corresponding to the target.

[0067] Step 7: Use the feature number to label each pixel. The pixel corresponding to the target is labeled as 1, otherwise it is labeled as 0.

[0068] Step 8: The feature array of several consecutively input pixels (8 in this specific implementation) is converted into an 8-bit binary number, which is then used as the feature number A. Feature number A can be used to eliminate some isolated pixels whose grayscale values ​​fall within the target brightness threshold. By judging the grayscale values ​​of pixels using feature number A, these isolated pixels can be removed to avoid interfering with the centroid determination, thus obtaining the final valid target pixel grayscale values. .

[0069] Step 9: Convert the final effective target pixel grayscale value respectively with its x-axis coordinate y-axis coordinate Multiply them, then calculate their cumulative sums to obtain the weighted sums in the x and y directions. , ,as follows:

[0070] ;

[0071] ;

[0072] in, , For the first The coordinates of the points.

[0073] Step 10: Under the influence of line and field signals, traverse the entire image and calculate the cumulative sum of gray values ​​for the entire image. :

[0074] ;

[0075] in, Let x be the x-coordinate of a pixel in the image. Let be the y-coordinate of that pixel. This is the grayscale value of the pixel.

[0076] Step 11: Call the division operation IP core (a functional module in VIVADO software, which is a software that turns code into a "circuit diagram" that can run on an FPGA) to sum the grayscale values ​​of the entire image. As a divisor, , The dividends are added to two separate dividers for division operations to obtain the results. The final centroid coordinates are shown below. The result of the divider is output, and the remainder is discarded to obtain the final value:

[0077] ;

[0078] .

[0079] The solution of the present invention may further include: step 12, outputting the finally obtained centroid.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0081] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0085] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0086] The above description is merely an embodiment of the present invention and does not limit the scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related system fields, are similarly included within the protection scope of the present invention.

[0087] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A weighted centroid extraction method for grayscale images based on FPGA, characterized in that, include: Step 1: Use the CPU or GPU to calculate the target brightness threshold in real time and write it into the register, and then use the FPGA to read the target brightness threshold from the register. Step 2: The line and field valid signals of the image and the image data are input into the FPGA; Step 3: The FPGA receives image data, segments the image data by pixels, splits the image data into pixel grayscale values, and caches the pixel grayscale values ​​of each pixel into a register. Step 4: Calculate the image resolution using the effective line and field signals of the image, under the tick of the clock signal; Step 5: Define the top-left corner of the image as the origin (0,0), extend the x-axis to the right, and extend the y-axis downwards, marking the coordinates of each point as follows. ; Step 6: Identify the pixels whose grayscale values ​​fall between the target brightness and darkness thresholds as the corresponding pixels of the target. Step 7: Mark each pixel; Step 8: Use feature number A to remove certain pixel grayscale values ​​that fall within the target brightness threshold but are actually isolated pixels, thus obtaining the final effective target pixel grayscale values. ; Step 9: Convert the final effective target pixel grayscale value respectively with its x-axis coordinate y-axis coordinate Multiply them, then calculate their cumulative sum to obtain the weighted sum in the x and y directions; Step 10: Calculate the sum of gray values ​​for the entire image; Step 11: Perform a division operation using the sum of the grayscale values ​​of the entire image as the divisor and the weighted sum along the x and y axes as the dividend, to obtain ( () represents the final centroid coordinates.

2. The FPGA-based weighted centroid extraction method for grayscale images according to claim 1, characterized in that, In step 2, the effective horizontal and vertical signals of the image and the image data are transmitted to the FPGA via optical fiber in 1-tap, 2-tap, 4-tap, or 8-tap formats.

3. The FPGA-based weighted centroid extraction method for grayscale images according to claim 1, characterized in that, In step 4, the image resolution includes the image width and height, where the image width is... Image height is .

4. The FPGA-based weighted centroid extraction method for grayscale images according to claim 1, characterized in that, In step 6, the pixel grayscale value of each pixel is compared with the target brightness threshold transmitted in real time by the CPU or GPU, and the pixels whose pixel grayscale value is between the target brightness threshold are determined to be the pixels corresponding to the target.

5. The FPGA-based weighted centroid extraction method for grayscale images according to claim 1, characterized in that, In step 7, each pixel is marked using the feature number. The pixel corresponding to the target is marked as 1, otherwise it is marked as 0. In step 8, the feature array of several consecutively passed pixels is converted into a binary number, which is used as the feature number A.

6. The FPGA-based weighted centroid extraction method for grayscale images according to claim 3, characterized in that, In step 9, the weighted sum in the x and y directions , for: ； ； in, , For the first The coordinates of the points; In step 10, the grayscale values ​​of the entire image are summed. for: ； in, Let x be the x-coordinate of a pixel in the image. Let be the y-coordinate of that pixel. This is the grayscale value of the pixel.

7. The FPGA-based weighted centroid extraction method for grayscale images according to claim 6, characterized in that, The coordinates of the centroid are: ； 。 8. A weighted centroid extraction device for grayscale images based on FPGA, characterized in that, include: The target brightness threshold acquisition module uses CPU or GPU to calculate the target brightness threshold in real time and write it into a register, and then the FPGA reads the target brightness threshold from the register. The image data acquisition module transmits the effective line and field signals and image data of the image to the FPGA; The segmentation module receives image data from the FPGA, segments the image data by pixels, breaks down the image data into pixel grayscale values, and caches the pixel grayscale values ​​of each pixel in a register. The image resolution acquisition module calculates the image resolution by using the effective line and field signals of the image in the clock cycle. The first labeling module defines the top-left corner of the image as the origin (0,0), extends the x-axis to the right, and the y-axis to the bottom, labeling the coordinates of each point as follows: ; The pixel identification module identifies pixels whose grayscale values ​​fall between the target brightness and darkness thresholds as the corresponding pixels of the target. The second marking module marks each pixel. The target pixel grayscale value acquisition module uses feature number A to eliminate certain pixels whose grayscale values ​​fall within the target brightness threshold but are actually isolated pixels, thus obtaining the final effective target pixel grayscale values. ; The weighted sum calculation module calculates the final effective target pixel grayscale value. respectively with its x-axis coordinate y-axis coordinate Multiply them, then calculate their cumulative sum to obtain the weighted sum in the x and y directions; The summation module calculates the sum of gray values ​​across the entire image. The centroid extraction module uses the sum of the grayscale values ​​of the entire image as the divisor and the weighted sum of the values ​​along the x and y axes as the dividend to perform a division operation, resulting in (…). () represents the final centroid coordinates.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the FPGA-based weighted centroid extraction method for grayscale images as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the FPGA-based weighted centroid extraction method for grayscale images as described in any one of claims 1 to 7.

Images