Image measurement system

Abstract

The invention discloses an image measurement system, which comprises an input device for inputting an image to be measured, and a processing device integrated with a preprocessing unit, a morphology fitting unit and an angle calculation unit, the preprocessing unit is used for performing two-dimensional convolution filtering and sliding window median filtering on the to-be-detected image through the parallel filtering architecture to obtain a first image; the morphology fitting unit is used for obtaining morphology fitting parameters of the target structure in the first image through least square fitting; the angle calculation unit is used for comparing the morphology fitting parameter with a preset reference model parameter to obtain an angle measurement result of the target structure and a specific direction; the system further comprises an output device for outputting and displaying angle measurement results. According to the scheme, under the condition that influence factors such as noise, gray fluctuation and local interference exist in the image quality, parameters reflecting the real morphology of the target structure can still be stably obtained, more reliable angle measurement is achieved accordingly, and the accuracy of an angle measurement result is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image measurement system. Background Technology

[0002] In applications such as industrial manufacturing, precision machining, and machine vision inspection, it is necessary to accurately measure the angle of a target structure relative to a specific direction, such as the deflection angle of the target structure relative to the horizontal or vertical direction. This angle measurement result typically serves as a crucial basis for dimensional inspection, posture calibration, assembly accuracy control, and defect identification, thus requiring high accuracy and stability.

[0003] In related technologies, image angle measurement techniques based on general-purpose hardware (such as central processing units (CPUs) and graphics processing units (GPUs)) are typically used to measure the angle of a target structure relative to a specific direction.

[0004] However, in practical applications, measurement methods based on general-purpose hardware often fail to reliably capture the true shape of the target structure when there are factors such as noise, grayscale fluctuations, and local interference in the image quality, resulting in low accuracy of angle measurement results. Summary of the Invention

[0005] To address the aforementioned issues, this application provides an image measurement system for improving the accuracy of angle measurement results under conditions of multiple influencing factors.

[0006] Based on this, the following technical solution is disclosed in this application:

[0007] In a first aspect, embodiments of this application provide an image measurement system, including:

[0008] An input device is used to input an image to be tested, wherein the image to be tested is an image formed for a target structure;

[0009] The processing device integrates a preprocessing unit, a shape fitting unit, and an angle calculation unit. The preprocessing unit performs two-dimensional convolution filtering and sliding window value filtering on the image to be tested using a parallel filtering architecture to obtain a first image. The shape fitting unit obtains the shape fitting parameters of the target structure in the first image using the least squares method. The angle calculation unit compares the shape fitting parameters with preset benchmark model parameters to obtain the angle measurement result of the target structure with respect to a specific direction.

[0010] An output device is used to output and display the angle measurement results.

[0011] As can be seen from the above technical solutions, this application has at least the following beneficial effects:

[0012] First, the preprocessing unit employs a parallel filtering architecture to perform two-dimensional convolutional filtering and sliding window value filtering on the image under test, which can suppress random noise and impulse noise within the same processing unit. Two-dimensional convolutional filtering can smooth the overall image, reducing the impact of grayscale fluctuations on the image structure. Sliding window value filtering can weaken the interference of local abnormal pixels on the image. Therefore, the first image can more realistically reflect the overall morphology of the target structure, providing a stable and more accurate input basis for subsequent morphology fitting.

[0013] Based on this, the morphology fitting unit fits the morphology fitting parameters of the target structure using the least squares method based on the first image. This enables the overall modeling of the target structure's morphology with the participation of multiple pixels. It reduces the impact of local noise or residual interference on the fitting results, thus obtaining more stable and continuous morphology fitting parameters.

[0014] Furthermore, the angle calculation unit calculates the angle measurement results of the target structure in a specific direction by comparing the shape fitting parameters with the preset benchmark model parameters. Since the angle measurement results are based on stable and accurate shape fitting parameters, the angle calculation process is less sensitive to image noise, gray-level unevenness, and local interference, and is less affected by interference.

[0015] Therefore, by integrating a preprocessing unit, a shape fitting unit, and an angle calculation unit into the processing device, parameters reflecting the true shape of the target structure can still be stably obtained even when there are influencing factors such as noise, grayscale fluctuations, and local interference in the image quality. Based on this, more reliable angle measurements can be achieved, thus improving the accuracy of the angle measurement results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the structure of an image measurement system provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of the structure of a Gaussian filter unit provided in an embodiment of this application;

[0019] Figure 3This is a schematic diagram of the structure of a median filtering unit provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the structure of an image reconstruction unit provided in an embodiment of this application. Detailed Implementation

[0021] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0022] In related technologies, image processing algorithms are typically executed on general-purpose hardware to analyze images of target structures and perform angle measurements accordingly. These technologies generally implement angle measurement functions by running software algorithms for feature extraction and angle calculation on general-purpose computing platforms, offering advantages such as flexible implementation and strong versatility.

[0023] However, in real-world industrial applications, the images under test are often affected by the imaging environment and acquisition conditions, easily leading to problems such as noise superposition, uneven grayscale, local occlusion, or background interference. Since general-purpose hardware primarily relies on software algorithms to execute the above processing steps sequentially or in parallel, when image quality fluctuates significantly, the algorithms are highly sensitive to noise and local anomalies, making it difficult to reliably extract effective information that truly reflects the overall shape. This can easily cause deviations in subsequent angle measurement results based on this shape information, making it difficult to guarantee measurement accuracy and stability.

[0024] Based on this, the present application provides an image measurement system that integrates a preprocessing unit, a shape fitting unit, and an angle calculation unit through a processing device. This enables the system to stably acquire parameters that reflect the true shape of the target structure even when there are influencing factors such as noise, grayscale fluctuations, and local interference in the image quality. As a result, more reliable angle measurements are achieved, and the accuracy of the angle measurement results is improved.

[0025] All data collected in this application (such as images to be tested) was collected with the consent and authorization of the data subject (such as users, institutions or enterprises), and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0026] See Figure 1 The figure is a schematic diagram of the structure of an image measurement system provided in an embodiment of this application.

[0027] This image measurement system includes an input device, a processing device, and an output device. These will be described in detail below.

[0028] (1) Input device, used to input the image to be tested.

[0029] The input device is a data input component in an image measurement system, used to input externally acquired images to be measured into the processing unit of the image measurement system, so that the processing unit can process the images. The target structure is the structure where angle measurement is required. For example, the target structure can be the edge contour, groove structure, extension direction of processing texture, or other structural areas in the image that exhibit overall directional characteristics of a workpiece (such as a wafer, assembly substrate, support base, display panel, etc.).

[0030] The image to be tested is an image formed with respect to the target structure. That is to say, in the imaging process of the image to be tested, the target structure is used as the main imaging object, so that the target structure has identifiable morphological features and spatial distribution in the image.

[0031] It should be noted that the image to be tested is not limited to including only the target structure; it may also include background regions or other structures. The target structure in the image to be tested should have sufficient image representation for subsequent measurements. The image to be tested may be a single image formed for the target structure, multiple images formed for the target structure from the same angle, or multiple image sequences formed for the target structure from multiple angles. This application does not limit this.

[0032] In one possible implementation, if there are multiple images to be tested, the input device is specifically used to receive the sequence of images to be tested serially via a dedicated input / output interface.

[0033] In this system, the sequence of images to be measured is transmitted sequentially into the image measurement system in a data stream manner. Dedicated input / output interfaces are dedicated data transmission channels or hardware interfaces, such as dedicated IOBanks, used solely for inputting and outputting images to be measured without participating in other data processing. This targeted approach improves data throughput efficiency.

[0034] In addition, the input unit also includes a first storage buffer and a second storage buffer, which are used to alternately write the image to be tested and output the image to be tested to the preprocessing unit, so as to realize continuous acquisition and processing of the sequence of images to be tested.

[0035] The storage buffer is a storage area used to temporarily store the image under test. The first storage buffer and the second storage buffer are different storage buffers. Alternating write means that while writing the image under test is performed in one storage buffer, the stored image under test is output to the preprocessing unit in another storage buffer, so that writing and output are alternated between different buffers.

[0036] As one possible implementation, the storage buffer can be configured to have more than two.

[0037] For example, in scenarios where images to be measured are continuously input at a high frequency, multiple storage buffers can be used sequentially to temporarily store images to be measured arriving at different times, enabling the image measurement system to receive high-density image data without data loss.

[0038] This allows for greater scheduling flexibility in the input, buffering, and output of the sequence of images under test, thus maintaining continuous buffering and stable output of the images under test even when the input rate fluctuates or the processing rate changes.

[0039] For example, the input unit receives the sequence Src of the image under test, the frame number FrameId of each image under test in Src, and the original image position Pos of each image under test in Src. The input unit can support the acquisition of single or multiple frames of low-resolution image under test sequences at the same location, with sequences at the same location distinguished by the frame number FrameId. Data enters the processing device through a dedicated input / output interface IO Bank and is processed by the preprocessing unit. The sequence of images under test can be cached in off-chip Double Data Rate (DDR) memory or large-scale on-chip Block Random Access Memory (BRAM). To achieve high-speed continuous acquisition, the input unit can employ a ping-pong caching mechanism: using two storage buffers, such as a first storage buffer and a second storage buffer, so that while one storage buffer is being written with a new image under test, the image under test stored in the other storage buffer can be read and processed simultaneously by subsequent processing modules, thereby greatly improving data throughput and system real-time performance.

[0040] Therefore, by using serial input, dedicated input / output interfaces, and dual storage buffers to alternately operate multiple images under test, the image measurement system can maintain stable input, continuous processing, and efficient output when continuously acquiring multiple frames of images under test, thereby improving the processing efficiency and performance of the image measurement system in multiple image measurement scenarios.

[0041] (2) The processing device integrates a preprocessing unit, a shape fitting unit and an angle calculation unit.

[0042] The processing unit is used to process the image under test to obtain the corresponding angle measurement results. The preprocessing unit, the shape fitting unit, and the angle calculation unit are configured in the same processing unit, and the processing unit manages and coordinates the operation of each functional unit.

[0043] In one possible implementation, the processing device is equipped with a field-programmable gate array (FPGA), which implements the functions of a preprocessing unit, a shape fitting unit, and an angle calculation unit through multiple logic operation regions.

[0044] It's worth noting that Field-Programmable Gate Arrays (FPGAs), as highly parallel and flexibly programmable integrated circuits, exhibit unique advantages in the field of image processing. An FPGA consists of a large number of configurable logic blocks (CLBs), input / output blocks (IOBs), and routing resources. Users can program it using hardware description languages ​​(such as Verilog or VHDL) to achieve customized hardware designs. Compared to Application-Specific Integrated Circuits (ASICs), FPGAs feature shorter development cycles and higher reconfigurability; compared to general-purpose processors, FPGAs offer advantages in image processing such as strong parallel processing capabilities, low latency, and high power efficiency.

[0045] It is understandable that current image processing, especially image angle measurement, is mostly implemented using general-purpose hardware (such as CPUs and GPUs). Although widely used, it exposes a series of inherent technical bottlenecks and limitations in high-precision, high-efficiency, and highly adaptable industrial and real-time processing scenarios: First, the measurement accuracy is insufficient. Traditional image matching algorithms, such as template matching, are easily affected by changes in image lighting and noise interference, resulting in large matching errors, which in turn affect the accuracy of angle calculation. This error is even more significant in the measurement of images with small structures or complex backgrounds. Second, the adaptability is poor. For target structures of different sizes and shapes, existing methods often need to be customized for specific scenarios. The first problem is the lack of standardization and versatility. When the position and size of the target structure in the image change, it is difficult to guarantee the stability and accuracy of the measurement. The second problem is the low degree of automation. Some image angle measurement methods rely on manual intervention, such as manually selecting the measurement area and adjusting parameters, which cannot meet the needs of large-scale automated production, resulting in low measurement efficiency and high labor costs. The third problem is the balance between computing efficiency and resource consumption. When processing high-resolution images or performing intensive calculations, solutions based on general-purpose processors (CPU / GPU) are difficult to customize and optimize according to specific image processing algorithms, putting them at a disadvantage in scenarios that pursue the ultimate energy efficiency and low cost.

[0046] In one feasible embodiment, the processing device of this application is based on the characteristics of FPGA devices, integrating a large number of logic units to implement corresponding functions. First, by utilizing the FPGA's fully pipelined architecture and hardware optimization algorithms, compared with existing general-purpose hardware architecture-oriented solutions, this application's solution can implement core computational steps such as image preprocessing (grayscale conversion, denoising, enhancement), least-squares shape fitting, and even steps such as super-resolution reconstruction and high-precision registration, all through the highly parallel pipelined structure and customized digital logic within the FPGA. This eliminates time-consuming software iterations and frequent memory access operations on general-purpose processors, greatly improving data throughput and computational efficiency through pipelined parallel processing. Second, the process is automated. The entire image processing process of this application can be seamlessly integrated on the FPGA. By performing real-time hardware analysis on target features of different sizes and shapes, there is no need for software redevelopment or manual intervention for different scenarios, solving the problems of poor adaptability and low automation in existing technologies. It is understandable that the technical solution of this application can significantly improve measurement accuracy. With the high-precision computing and super-resolution reconstruction technology on FPGA, it achieves sub-pixel level image registration and detail recovery capabilities, effectively overcoming the matching errors caused by changes in lighting and noise interference in traditional methods. Even under conditions of small structures or complex backgrounds, it can achieve high-accuracy angle calculation. Moreover, the technical solution of this application also greatly improves processing speed and efficiency. The highly parallel architecture and deep pipeline design of FPGA allow image preprocessing, parameter fitting and other steps to be performed simultaneously. Its processing latency is much lower than that of traditional solutions that rely on CPU / GPU for serial or batch processing, which can meet the requirements of industrial inspection scenarios with high frame rates and extremely high real-time requirements. In addition, the technical solution of this application also optimizes power consumption and integration. Compared with CPU and GPU solutions with high power consumption and ASIC solutions with poor flexibility, the FPGA-based implementation achieves a better balance in power consumption, performance and cost. Its hardware architecture can be optimized and configured according to actual needs to avoid resource waste, and is more suitable for application scenarios with strict power consumption constraints, such as embedded systems, mobile devices, and data centers.

[0047] The preprocessing unit, the morphology fitting unit, and the angle calculation unit are described below.

[0048] 1. The preprocessing unit is used to perform two-dimensional convolution filtering and sliding window value filtering on the image to be tested through a parallel filtering architecture to obtain the first image.

[0049] Specifically, the preprocessing unit is a functional unit within the processing device. It performs preprocessing operations on the input image to be measured before angle measurement, thereby reducing the impact of noise, grayscale fluctuations, local anomalies, and other influencing factors on subsequent morphology analysis. The preprocessing unit uses the image to be measured as input and performs filtering processing on it to make the overall morphology of the target structure in the image more stable, providing a suitable image basis for subsequent morphology fitting analysis. The first image is the image obtained after filtering processing by the preprocessing unit.

[0050] In this embodiment, the preprocessing unit processes the image under test using a parallel filtering architecture. A parallel filtering architecture refers to performing multiple different types of filtering operations on the same image under test, allowing each filtering method to target different types of image interference. This parallel filtering architecture improves the preprocessing's adaptability to complex imaging conditions without increasing the complexity of the processing flow.

[0051] In the parallel filtering architecture, the preprocessing unit performs two-dimensional convolution filtering and sliding window value filtering on the image to be tested.

[0052] Two-dimensional convolutional filtering is an image filtering method based on weighted calculations of pixel neighborhoods. By comprehensively considering the relationship between a pixel and its surrounding pixels, it performs overall image smoothing, thereby reducing image instability caused by grayscale fluctuations or overall noise. Two-dimensional convolutional filtering helps maintain the overall continuity of the target structure, making its shape clearer.

[0053] Sliding window midpoint filtering is a filtering method based on local pixel statistical characteristics. It involves setting a sliding window in the image under test that moves with the pixel position, sorting the pixel values ​​within the sliding window, and selecting the median value as the output. This suppresses the influence of local abnormal pixels or sudden interference on the image under test, maintaining a stable grayscale representation within local regions.

[0054] By combining two-dimensional convolutional filtering and sliding window value filtering in the same parallel filtering architecture, the preprocessing unit can simultaneously perform overall smoothing and local anomaly suppression on the image under test, thereby weakening different types of noise and interference. This results in a preprocessed image with high stability at both the overall and local levels.

[0055] In one possible embodiment, the preprocessing unit can perform two-dimensional convolution filtering and sliding window value filtering on the input image to be tested, respectively, to obtain the filtering results of the two filtering methods, and form a first image for subsequent processing based on the two filtering results through image synthesis technology.

[0056] In another possible embodiment, the preprocessing unit may perform two-dimensional convolutional filtering and sliding window midpoint filtering sequentially, and further input the filtering result of the two-dimensional convolutional filtering into the processing of the sliding window midpoint filtering to obtain the first image.

[0057] 2. The morphology fitting unit is used to obtain the morphology fitting parameters of the target structure in the first image by fitting using the least squares method.

[0058] Specifically, the morphology fitting unit is a functional unit within the processing device. After image preprocessing, it analyzes and processes image information related to the target structure in the first image to obtain morphology fitting parameters that characterize the overall morphological features of the target structure. This transforms the scattered pixel information in the first image into a geometrically meaningful parameter expression, providing a data foundation for subsequent angle calculations.

[0059] The morphology fitting unit uses the least squares method to fit the target structure in the first image. The least squares method is a mathematical approach that determines the optimal fitting result by minimizing the sum of squared overall errors. Its core principle is to comprehensively consider the combined influence of multiple data points on the fitting result, rather than relying on individual data points. By using the least squares method, fitting results that reflect the overall trend of the target structure can be obtained even in the presence of local noise or residual interference.

[0060] During the fitting process, the morphology fitting unit can model the overall morphology of the target structure based on multiple pixels or image positions related to the target structure in the first image, and determine the fitting result that minimizes the overall error using the least squares method. This fitting result is output in the form of morphology fitting parameters, which describe the morphological features of the target structure in the image. The morphology fitting parameters are a set of parameters used to characterize the morphology of the target structure.

[0061] In one possible embodiment, when the target structure exhibits an approximately linear morphology in the first image, the shape fitting parameters can be used to describe the overall extension direction of the target structure. In another possible embodiment, the shape fitting parameters can also be used to characterize the overall geometric relationship of the target structure in the image.

[0062] By employing the least squares method to obtain morphology fitting parameters, the overall morphology of the target structure represented in the first image is stably modeled, reducing the influence of local noise, individual abnormal pixels, or local interference on the fitting results, and improving the stability and consistency of morphology parameters.

[0063] 3. The angle calculation unit is used to compare the shape fitting parameters with the preset benchmark model parameters to obtain the angle measurement results of the target structure and a specific direction.

[0064] Specifically, the angle calculation unit is a functional unit in the processing device, used to further analyze the shape fitting parameters of the target structure after obtaining them, thereby calculating the angle measurement results of the target structure relative to a specific direction.

[0065] The reference model parameters are a set of parameters used to represent a specific direction, serving as a comparison benchmark during angle calculations. The specific direction is a pre-defined reference direction, such as the horizontal, vertical, or other reference directions used for angle measurement.

[0066] The angle calculation unit determines the geometric relationship between the shape fitting parameters and the benchmark model parameters by comparing the two, and then calculates the angle between the target structure and a specific direction based on the geometric relationship.

[0067] In one possible embodiment, when the target structure appears as a structure with a clear orientation in the first image, the angle calculation unit can compare the overall orientation reflected by the shape fitting parameters with the reference model parameters that characterize a specific direction, and then obtain the angle measurement result of the target structure relative to that specific direction.

[0068] In one possible embodiment, the baseline model parameters are determined as follows:

[0069] (1) The angle calculation unit obtains the set of feature points of the target structure.

[0070] The feature point set includes multiple feature points. Each feature point corresponds to a representative spatial location of the target structure in the image.

[0071] (2) The angle calculation unit samples the feature point set to obtain multiple sample point sets and generates candidate models corresponding to the multiple sample point sets respectively.

[0072] Specifically, a subset of feature points is selected from the feature point set to form a sampling point set for model construction. Multiple sampling point sets can be obtained by sampling the feature point set multiple times. For each sampling point set, the angle calculation unit generates a corresponding candidate model. A candidate model refers to a model representation based on the sampling point set that describes the morphological features of the target structure. Candidate models generated from different sampling point sets may differ, resulting in multiple different candidate models.

[0073] (3) The angle calculation unit determines the baseline model parameters of the target structure based on the candidate model with the most interior points.

[0074] In this context, inliers are feature points other than the candidate model whose distance to the candidate model is less than a distance threshold. By using the number of inliers as the evaluation criterion for the quality of candidate models, and selecting the candidate model with the most inliers as the model used to generate the baseline model parameters, the selected baseline model can maintain consistency with more feature points overall, thus more realistically reflecting the overall morphological characteristics of the target structure.

[0075] Therefore, by sampling the feature point set of the target structure and generating multiple candidate models by the angle calculation unit, and determining the baseline model parameters by the candidate model with the most interior points, the baseline model parameters can be selected from multiple candidate models with the highest consistency with the overall shape of the target structure. This reduces the influence of local abnormal feature points and noise on the baseline model determination process, making the determined baseline model parameters more stable and reliable, thereby improving the accuracy and consistency of angle measurement results between the target structure and a specific direction.

[0076] In one possible embodiment, the angle calculation unit includes an included angle calculation module and a direction determination module.

[0077] The included angle calculation module is used to obtain the current slope of the line segment to be tested in the target structure based on the shape fitting parameters of the target structure, and to obtain the reference slope of the baseline line corresponding to the line segment to be tested in the target structure based on the reference model parameters of the target structure, and to calculate the angle offset based on the current slope and the reference slope.

[0078] For example, the angle between the line segment to be measured and the corresponding baseline can be expressed by the following formula:

[0079] ;

[0080] in, This is the angular offset. The current slope, The baseline slope is used.

[0081] When the baseline corresponding to the line segment to be tested in the target structure points to the vertical direction, the angle calculation module calculates the angle between the line segment to be tested in the target structure and the vertical direction based on the shape fitting parameters as the angle offset.

[0082] As one possible implementation, when the baseline corresponding to the line segment to be measured in the target structure points vertically, and the line segment to be measured also points vertically, the angle offset is 0. When the baseline corresponding to the line segment to be measured in the target structure points vertically, and the line segment to be measured does not point vertically, the angle offset can be calculated using the following formula:

[0083] ;

[0084] in, This represents the angular offset of the line segment to be measured relative to the vertical direction. This represents the current slope of the line segment to be measured.

[0085] The direction determination module is used to obtain the average coordinates of the line segment to be tested in the target structure based on the shape fitting parameters, and to obtain the angular offset direction by comparing the position with the baseline corresponding to the line segment to be tested in the target structure.

[0086] The angle measurement results between the target structure and a specific direction are formed based on the angle offset and the angle offset direction.

[0087] During the angle measurement process, in order to ensure that the angle measurement results not only include angle magnitude information but also reflect the offset orientation of the target structure relative to a specific direction, the direction determination module, after obtaining the shape fitting parameters of the target structure, further determines the average coordinates (such as the average coordinates of the x-coordinate values) of the line segment to be measured in the target structure based on the shape fitting parameters, and uses the average coordinates as the representative position of the line segment to be measured in the image coordinate system.

[0088] Subsequently, the direction determination module compares the average coordinates of the line segment to be tested with the position of the baseline corresponding to the line segment to be tested in the target structure in the image. By determining which side of the baseline the average coordinates of the line segment to be tested is located on, the offset direction of the target structure relative to the baseline is determined, that is, whether the target structure is offset to the left or to the right relative to a specific direction.

[0089] Based on the above position comparison results, the direction determination module can assign corresponding positive and negative signs to the angle offset, so that the angle offset numerically includes both angle magnitude and offset direction information, and finally outputs the angle measurement result including offset direction information.

[0090] Therefore, by dividing the angle calculation unit into an angle calculation module and a direction determination module, the angle calculation module calculates the angle offset of the target structure relative to a specific direction based on the shape fitting parameters and the reference model parameters, while the direction determination module determines the angle offset direction by comparing the average coordinates of the line segment to be measured with the position of the corresponding reference line. Thus, the angle measurement result includes both angle magnitude and offset direction information, making the angle measurement result more comprehensive.

[0091] Furthermore, when the baseline points vertically, by adopting an angle calculation method specific to the vertical direction, the instability caused by the slope approaching infinity is avoided, making the calculation of the angle offset more reliable.

[0092] (3) Output device, used to output and display angle measurement results.

[0093] Specifically, the output device is used to output and display the angle measurement results output by the angle calculation unit, so that the angle measurement results can be directly obtained and used by users or other processing systems. For example, the input device is an output driver component, a graphics card, a monitor, or other similar devices.

[0094] As can be seen from the above technical solution, firstly, the preprocessing unit adopts a parallel filtering architecture to perform two-dimensional convolutional filtering and sliding window value filtering on the image under test, which can suppress random noise and impulse noise in the same processing unit. Two-dimensional convolutional filtering can smooth the overall image and reduce the impact of grayscale fluctuations on the image structure. Sliding window value filtering can weaken the interference of local abnormal pixels on the image. Therefore, the first image can more realistically reflect the overall morphology of the target structure, providing a stable and more accurate input basis for subsequent morphology fitting.

[0095] Based on this, the morphology fitting unit, using the first image, fits the morphology fitting parameters of the target structure using the least squares method. This enables the overall modeling of the target structure's morphology with the participation of multiple pixels. It reduces the impact of local noise or residual interference on the fitting results, thus obtaining more stable and continuous morphology fitting parameters.

[0096] Furthermore, the angle calculation unit calculates the angle measurement results of the target structure in a specific direction by comparing the shape fitting parameters with the preset benchmark model parameters. Since the angle measurement results are based on stable and accurate shape fitting parameters, the angle calculation process is less sensitive to image noise, gray-level unevenness, and local interference, and is less affected by interference.

[0097] Therefore, by integrating a preprocessing unit, a shape fitting unit, and an angle calculation unit into the processing device, parameters reflecting the true shape of the target structure can still be stably obtained even when there are influencing factors such as noise, grayscale fluctuations, and local interference in the image quality. Based on this, more reliable angle measurements can be achieved, thus improving the accuracy of the angle measurement results.

[0098] In one possible embodiment, the preprocessing unit includes a one-dimensional row-direction convolution module, a row-column buffer transformation module, and a one-dimensional column-direction convolution module.

[0099] In the process of two-dimensional convolutional filtering, the one-dimensional row direction convolution module is used to perform convolution operation on the image to be tested according to the preset filter convolution kernel to obtain the initial feature map.

[0100] The preset filter convolution kernel is a weight parameter that is predetermined before the convolution filtering process. It is used to perform weighted calculations on multiple pixels during the convolution operation, thereby achieving smoothing or feature enhancement processing of the image.

[0101] The preset filtering convolution kernel can be a matrix or a vector, where each weight represents the degree of influence of different neighboring pixels in the convolution calculation. By weighted summing of the target pixel and its surrounding pixels, the changes in local pixel values ​​in the image can be smoothed out, thereby reducing the impact of noise and grayscale fluctuations on image quality. As one possible implementation, the preset filtering convolution kernel can include different kernels for the column and row directions.

[0102] The initial feature map is an intermediate result obtained after processing by a one-dimensional row-direction convolution module. In other words, the initial feature map is the image result obtained after column-direction convolution.

[0103] The row and column buffer transformation module is used to cache the initial feature map through a ping-pong mechanism, and then convert the initial feature map into a column direction before inputting it into a one-dimensional column direction convolution module.

[0104] The ping-pong mechanism, also known as the double buffering mechanism, is a data caching and processing scheduling method. By setting at least two storage buffers, data write operations and read operations are alternated between different buffers, thereby avoiding simultaneous occupation of the same storage resource by read and write operations.

[0105] In the ping-pong mechanism, multiple storage buffers can take turns working according to a predetermined switching rule: when one storage buffer is used to receive or write new data, another storage buffer is used to provide cached data to subsequent processing modules. After one storage buffer completes the write operation, it will switch roles with another storage buffer, forming a working mode similar to "ping-pong ball switching back and forth".

[0106] By adopting the ping-pong mechanism, the data input process and the data processing process can be carried out in parallel, avoiding data blocking caused by waiting for each other during read and write operations. This improves the processing efficiency and stability of the system in continuous data input scenarios, and is especially suitable for application scenarios that require continuous data acquisition and processing.

[0107] As one possible implementation, the ping-pong mechanism can include at least three storage caches.

[0108] The one-dimensional column-direction convolution module is used to perform convolution operations on the initial feature map according to the preset filter convolution kernel to obtain a two-dimensional feature map. The two-dimensional feature map is configured as the result of the two-dimensional convolution filtering performed by the preprocessing module.

[0109] The two-dimensional feature map is the image result obtained after processing by a one-dimensional row-direction convolution module and a one-dimensional column-direction convolution module.

[0110] See Figure 2 The figure is a schematic diagram of the structure of a Gaussian filter unit provided in an embodiment of this application.

[0111] As shown in the figure, this schematic diagram illustrates the implementation architecture of one-dimensional convolution processing and row-column data transformation. Input data is sequentially input to the one-dimensional convolution module (row direction), i.e., the one-dimensional row-direction convolution module. The one-dimensional convolution module (row direction) performs convolution operations on the input data in the row direction according to a preset filtered convolution kernel, generating the corresponding initial feature map.

[0112] After the one-dimensional convolution module (row direction) completes the convolution process, the convolution result is written to the row-column buffer transformation module. The row-column buffer transformation module includes multiple block random access memories (BRAMs), such as Block RAM0, Block RAM1, and Block RAM2, used to cache the initial feature data obtained from the row-direction convolution. The row-column buffer transformation module also includes an address controller, which generates write and read signals to control the writing and reading order of the initial feature map in each Block RAM, thereby achieving ordered scheduling of the cached data.

[0113] Through the caching of the Block RAM and the control of the address controller, the row-column buffer conversion module can convert the initial feature data, originally stored in row order, into data output in column order, changing the data arrangement direction from row to column. The data after the row-column direction conversion is output to the one-dimensional convolution module (column direction), i.e., the column-direction convolution module. The one-dimensional convolution module (column direction) performs convolution operations on the input data in the column direction according to a preset filtered convolution kernel, thereby combining the row-direction convolution results with the column-direction convolution results to obtain a two-dimensional feature map. The configuration information of the preset filtered convolution kernel is provided by the convolution kernel configuration module and stored in the corresponding storage unit for use by the one-dimensional convolution module (row direction) and the one-dimensional convolution module (column direction) during convolution operations.

[0114] Finally, the one-dimensional convolution module (column direction) outputs a two-dimensional feature map in a column-by-column serial manner. The two-dimensional feature map is configured as the output result of the two-dimensional convolution filtering performed by the preprocessing module and is used in subsequent image processing.

[0115] Therefore, by adopting a separable convolutional architecture, the two-dimensional convolutional filtering is split into two one-dimensional convolutions in the row and column directions. Between the two convolutions, a row and column buffer conversion module is used to achieve ping-pong mechanism caching and data direction conversion, so that the initial feature map can be reused efficiently. While ensuring the effect of two-dimensional convolutional filtering, the number of multipliers required in the convolution operation is reduced, the logical resource consumption is reduced, and the overall processing efficiency of the system and the resource utilization of the hardware implementation are improved.

[0116] In one possible embodiment, the preprocessing unit further includes a line buffer module, N deserializers, and a multi-layer comparator, where N is a positive integer.

[0117] The row buffer module is used to input the two-dimensional feature map pixel by pixel in sequence, and obtain multiple pixel values ​​within a preset sliding window through a caching mechanism.

[0118] The preset sliding window is a preset range of a pixel used to limit the pixel group participating in the median calculation. The preset sliding window moves row by row on the two-dimensional feature map to obtain multiple pixel groups, each containing multiple pixel values.

[0119] As one possible implementation, the pixel value can be the grayscale value of the pixel.

[0120] N deserializers are used to deserialize multiple pixel groups in parallel, and the deserialized pixel values ​​are input into a multi-layer comparator.

[0121] The deserializer is used to separate multiple pixel values ​​in a pixel group, and the multi-level comparator is a functional module used to compare and sort multiple pixel values. By setting multiple comparison levels, it compares the input pixel values ​​of different sizes level by level, thereby determining the size relationship between each pixel value level by level.

[0122] The input pixel values ​​of different sizes are compared and sorted by a multi-layer comparator, and the median is output pixel by pixel. The median output pixel by pixel is configured as the result of sliding window mid-value filtering performed by the preprocessing module.

[0123] The first image is formed by traversing the two-dimensional feature map using a preset sliding window and utilizing the result of sliding window mid-value filtering.

[0124] Therefore, by employing a multi-layer comparator in the preprocessing unit to compare and sort multiple pixel values ​​within a preset sliding window step by step, the median calculation process is transformed from traditional sequential comparison to hierarchical parallel comparison, thereby reducing the number of comparisons and processing cycles required to complete a single median calculation. Furthermore, the row buffer module performs pixel-by-pixel serial input of the two-dimensional feature map, and N deserializers perform parallel deserialization of multiple pixel groups, allowing multiple pixel values ​​to simultaneously enter the multi-layer comparator for comparison. This enables the median to be quickly determined and output pixel-by-pixel serially. This not only suppresses abnormal pixels and noise interference but also improves the processing efficiency of median filtering while maintaining its effectiveness.

[0125] The following explanation uses a multi-layer comparator, consisting of a first-layer comparator, a second-layer comparator, and a third-layer comparator, as an example.

[0126] The first-level comparator is used to compare the size of multiple deserialized pixel values ​​and sort them in three levels to obtain multiple first comparison results. The first low pixel value, the first middle pixel value, and the first high pixel value among the multiple first comparison results are respectively input into the corresponding second-level comparator.

[0127] The first comparison result is the comparison result output by the first layer comparator. The first low pixel value, the first medium pixel value, and the first high pixel value are the pixel values ​​in the first comparison result sorted from low to high.

[0128] The second-layer comparator is used to compare and sort the first low pixel value, first middle pixel value, and first high pixel value in multiple first comparison results in three levels to obtain multiple second comparison results. The second low pixel value, second middle pixel value, and second high pixel value in multiple second comparison results are then input into the third-layer comparator.

[0129] The second comparison result is the comparison result output by the second layer comparator. The second low pixel value, the second medium pixel value, and the second high pixel value are the pixel values ​​in the second comparison result sorted from low to high.

[0130] The third-level comparator is used to compare the size of the second comparison result and directly output the median value.

[0131] See Figure 3 This is a schematic diagram of a median filtering unit provided in an embodiment of this application. Taking a multi-layer comparator including a first-layer comparator, a second-layer comparator, and a third-layer comparator as an example, the working process of this structure will be illustrated by way of example.

[0132] like Figure 3 As shown, the input data is input serially pixel by pixel. After being buffered by the row buffer module and the storage buffer, it forms multiple pixel groups. Each pixel group is deserialized by its corresponding deserializer to obtain multiple pixel values ​​used in the median calculation. The functions of the ping-pong mechanism and the address controller are described above and will not be repeated here.

[0133] Subsequently, the deserialized pixel values ​​are input to the first-layer comparator (composed of three three-point comparators). The first-layer comparator performs size comparison and three-level sorting (i.e., sorting by high, medium, and low) on each group of deserialized pixel values, resulting in multiple first comparison results. These first comparison results include the first low pixel value, the first medium pixel value, and the first high pixel value, sorted from low to high. Next, the first low pixel value, the first medium pixel value, and the first high pixel value from the multiple first comparison results are input to the corresponding second-layer comparator (composed of three three-point comparators). The second-layer comparator performs size comparison and three-level sorting again on the input pixel values, resulting in multiple second comparison results. These second comparison results include the second low pixel value, the second medium pixel value, and the second high pixel value, sorted from low to high. Subsequently, the second lowest pixel value, second middle pixel value, and second highest pixel value from multiple second comparison results are input together into a third-layer comparator (one 3-point comparator). This third-layer comparator performs further comparisons on the second comparison results and directly outputs the pixel value at the middle position as the median. For a preset 3x3 sliding window, a total of seven 3-point comparators are used to sort the nine pixel values ​​within the preset sliding window, and the median is directly output. The entire process decomposes the complete sorting of the nine pixel values ​​into multi-level parallel local sorting, greatly improving implementation efficiency.

[0134] Therefore, through the step-by-step comparison and filtering of the above multi-layer comparators, the median can be quickly determined in the multi-level parallel comparison structure and finally output in a pixel-by-pixel serial manner as the result of sliding window median filtering. This improves the overall comparison efficiency and processing speed while ensuring the median filtering effect.

[0135] In one possible implementation, if there are multiple images to be tested, the processing device further includes a registration unit and an image reconstruction unit.

[0136] The registration unit is signal-connected to the preprocessing unit and obtains multiple first images from the preprocessing unit for subpixel-level alignment of the multiple first images to obtain multiple registration images.

[0137] Subpixel-level alignment refers to finely adjusting the relative positional relationship between images during the image alignment process, so that the corresponding target structures in different images achieve positional consistency within the subpixel accuracy range, thereby reducing positional errors caused by acquisition offset or minor jitter.

[0138] Through subpixel-level alignment processing, the registration unit can adjust corresponding target structures in multiple first images to a unified spatial position, thereby obtaining multiple registered images. A registered image refers to the image result that has been aligned in spatial position, and it is used for subsequent information fusion and reconstruction processing.

[0139] The image reconstruction unit is signal-connected to the registration unit and is used to perform information fusion and image reconstruction on multiple registered images to obtain a high-resolution reconstructed image. The high-resolution reconstructed image is input to the topography fitting unit to obtain the topography fitting parameters corresponding to the target structure in the first image through least squares fitting.

[0140] Information fusion is used to fuse image information included in multiple registered images.

[0141] By reconstructing multiple registered images, the image reconstruction unit can generate a high-resolution reconstructed image. Compared to a single first image, the high-resolution reconstructed image has higher spatial resolution or clearer detail information, thus reflecting the morphological features of the target structure more clearly.

[0142] Furthermore, the high-resolution reconstructed image is input into the topography fitting unit, which performs topography fitting on the target structure using the least squares method to obtain the topography fitting parameters corresponding to the target structure. Since the topography fitting is based on the high-resolution reconstructed image, the fitting process is built upon a more refined and stable image foundation, which helps improve the accuracy of the topography fitting parameters. The process for determining the topography fitting parameters can be found in the foregoing explanation and will not be repeated here.

[0143] Therefore, by introducing a registration unit and an image reconstruction unit into the processing device, the image measurement system can first accurately align and fuse multiple first images when processing multiple images to be measured, and then perform morphology fitting based on the reconstructed high-resolution image, thereby enhancing the ability of the morphology fitting result to represent the true shape of the target structure and making the angle measurement result more accurate.

[0144] The following describes a specific implementation of the registration unit:

[0145] In this embodiment, the registration unit includes a corner detector, a feature matching module, and a transformation estimation module.

[0146] The corner detector is used to identify multiple first images and obtain key points and feature descriptors corresponding to each first image.

[0147] Corner detectors are used to identify pixels with localized changes in an image. For example, the FAST corner detector can be used. Corners are pixel locations in an image where the grayscale value changes significantly in multiple directions simultaneously. These locations are typically situated at structural corners, edge intersections, or areas of texture variation, making them easier to detect and repeatedly identify compared to flat areas or areas with changes in only one direction. Corner detectors identify the grayscale change characteristics of local regions in an image to determine whether a pixel meets the corner feature criteria, thus marking those pixels as keypoints. In this way, corner detectors can filter out a limited set of keypoints with high information content from the entire image. Feature descriptors are parametric expressions used to describe the features of keypoints.

[0148] Configure one of the multiple first images as a reference image, and configure any one of the multiple first images other than the reference image as a target image.

[0149] The registration process between the reference image and the target image is explained below.

[0150] The feature matching module is used to match the key points and feature descriptors of the target image with the key points and feature descriptors of the reference image to obtain the matching relationship of the target image.

[0151] In other words, based on the similarity of feature descriptors, the key points in the target image are found to be most likely to correspond to the key points in the reference image, thus forming a matching relationship. The matching relationship is a correspondence of a set of key points, which is used to characterize the corresponding positions that should be aligned in space between the target image and the reference image.

[0152] The transformation estimation module is used to estimate the affine transformation matrix based on the matching relationship of the target image, and to resample the target image based on the affine transformation matrix using bilinear interpolation to obtain the sub-pixel information of the target image.

[0153] The affine transformation matrix is ​​a parametric matrix used to describe linear transformations and translations in two-dimensional space. It characterizes the geometric mapping from the target image to the reference image, allowing the target image to be mapped to the coordinate system of the reference image through the affine transformation matrix. After obtaining the affine transformation matrix, the transformation estimation module performs a spatial transformation on the target image based on this matrix, and resamples the target image during the spatial transformation process. Resampling is necessary when a pixel in the target image falls at a non-integer pixel position after the transformation. This requires recalculating the pixel value at that position to generate aligned image data. For this purpose, bilinear interpolation is used to calculate the pixel value. Bilinear interpolation is an interpolation method that uses weighted calculations of the pixel values ​​in the surrounding neighborhood of the target position to obtain the pixel value at that position, thus supporting fine-grained position adjustments smaller than one pixel.

[0154] Subpixel information represents the pixel positional relationships and pixel value representations of the target image under non-integer pixel offset conditions, reflecting the alignment result of the target image relative to the reference image at subpixel precision.

[0155] The corresponding first image is registered based on the subpixel information of each target image, and multiple registered images are obtained through subpixel-level alignment.

[0156] Finally, the corresponding first image is registered based on the subpixel information of each target image, so that each target image is transformed and aligned to the reference image, thereby achieving subpixel-level alignment, that is, controlling the relative deviation between the two images within a scale of less than one pixel, thus obtaining multiple registered images that are spatially aligned with the reference image. These multiple registered images provide a positionally consistent input basis for subsequent information fusion and image reconstruction.

[0157] Therefore, by matching the target image with the reference image based on key points and feature descriptors, and further using the matching relationship to estimate the affine transformation matrix to perform sub-pixel level resampling alignment of the target image, multiple first images can be aligned in spatial position within an accuracy range of less than one pixel. This eliminates the relative displacement error between multiple images caused by acquisition offset or jitter, providing a consistent input basis for subsequent image information fusion and high-resolution reconstruction, improving the detail expression ability of the reconstructed image, and the accuracy of shape fitting based on the reconstructed image.

[0158] In one possible embodiment, the image reconstruction unit includes multiple parallel multiply-accumulate units.

[0159] Multiple multiply-accumulate units are used to perform forward convolution, pixel ratio calculation, and backward convolution operations on multiple registered images to obtain estimated images, and to obtain high-resolution reconstructed images through multiple rounds of iterative processing.

[0160] The multiply-accumulate unit is a computational unit capable of performing multiplication and addition operations, suitable for numerical computation tasks such as convolution, pixel value calculation, and iterative updates. By setting multiple multiply-accumulate units to work in parallel, the large number of numerical calculations involved in image reconstruction can be executed in parallel, thereby improving overall computational efficiency. The estimated image is the estimation result of the target structure image.

[0161] During image reconstruction, multiple multiply-accumulate units are used to perform forward convolution, pixel ratio calculation, and backward convolution operations on multiple registered images. Forward convolution is used to model the registered images to simulate the degradation relationship of the images during the imaging process. Pixel ratio calculation is used to compare the pixel value relationship between the current estimated image and the registered image. Backward convolution is used to backpropagate error information to the estimated image to correct the obtained estimated image.

[0162] Based on this, the image reconstruction unit performs multiple iterative processes on the estimated image, gradually updating and correcting it to continuously accumulate effective information, thereby progressively improving the spatial resolution and detail representation of the image, ultimately obtaining a high-resolution reconstructed image. The high-resolution reconstructed image is obtained by reconstructing multiple registered images at high resolution.

[0163] Therefore, by employing multiple parallel multiply-accumulate units in the image reconstruction unit to perform iterative operations of forward convolution, pixel ratio calculation, and backward convolution on multiple registered images, the image reconstruction process can be completed efficiently under parallel computing conditions, thereby shortening the processing time while ensuring reconstruction accuracy. Furthermore, by continuously refining the estimated image through multiple iterations, the effective information from multiple registered images is fully integrated, improving the resolution and detail representation of the reconstructed image.

[0164] The following example, using the i-th round of a multi-round iteration (where i is a positive integer), illustrates a possible iterative process:

[0165] In the i-th iteration, multiple multiply-accumulate units perform forward convolution on the estimated image of the (i-1)-th iteration using the target convolution kernel to obtain the simulated blurred image of the i-th iteration.

[0166] The simulated blurred image is an image obtained by performing a forward convolution operation between the previous estimated image and the target convolution kernel during one iteration of image reconstruction. It is used to simulate the imaging degradation process of the previous estimated image.

[0167] Multiple multiply-accumulate units first use the estimated image from the (i-1)th round as input for the current iteration, and perform forward convolution processing on the input image according to the target convolution kernel to obtain the simulated blurred image from the i-th round. The target convolution kernel is used to characterize the blurring effect of the imaging system on the image. Through forward convolution, the degradation effect of the input image from the (i-1)th round during the imaging process can be simulated at the mathematical model level, making the obtained simulated blurred image morphologically comparable to the actually acquired registered image.

[0168] Multiple multiply-accumulate units calculate the pixel ratio between multiple registered images and the simulated blurred image in the i-th round, and perform inverse convolution based on the result of the pixel ratio calculation to obtain the grayscale adjustment amount.

[0169] Pixel ratio calculation characterizes the pixel-level difference between the estimated image of the current round, after processing by the imaging model, and the registered image. This difference reflects which regions in the current estimated image have excessively high or low brightness estimates. Based on the pixel ratio calculation results, multiple multiply-accumulate units further perform inverse convolution processing on the comparison results, allowing pixel-level error information to be propagated back to the image space according to the inverse action of the target convolution kernel, thereby obtaining the grayscale adjustment amount used to correct the image brightness distribution. The grayscale adjustment amount is used to adjust the grayscale value of each pixel.

[0170] The estimated image for round i-1 is updated based on the grayscale adjustment amount to obtain the estimated image for round i.

[0171] After obtaining the grayscale adjustment amount, the image reconstruction unit updates the estimated image of round i-1 based on the grayscale adjustment amount. By adjusting the pixel values ​​of the estimated image accordingly, the estimated image of round i is obtained. The estimated image of round i is the result of correcting the estimated image of round i-1 under the constraints of multiple registered image information and target convolution kernel. Compared with the estimated image of the previous round, it is closer to the image representation of the real target structure in terms of resolution or detail representation.

[0172] Therefore, by calculating the pixel-by-pixel ratio between multiple registered images and the simulated blurred image in each iteration, the image reconstruction process can more accurately characterize the difference between the estimated image and the actual imaging result in the current iteration at the pixel level, thus reflecting areas where the brightness in the estimated image is overestimated or underestimated. Furthermore, based on the pixel ratio calculation results, inverse convolution processing is performed, effectively feeding back the aforementioned difference relationship into the image space in the reverse manner of the imaging model. This forms a grayscale adjustment amount used to correct the image brightness distribution, thereby providing targeted correction to the estimated image in subsequent image updates. This results in better convergence and stability of the reconstruction process, gradually improving the accuracy and detail consistency of the high-resolution reconstructed image in each iteration.

[0173] In one possible implementation, the target convolutional kernel is determined as follows:

[0174] Obtain the initial convolution kernel. The initial convolution kernel is the initialized convolution kernel, and its structure is the same as the target convolution kernel.

[0175] For the target pixel of the initial convolution kernel, the value of the target pixel is determined based on the coordinates of the target pixel, the center coordinates of the initial convolution kernel, and the standard deviation of the initial convolution kernel.

[0176] The standard deviation is used to identify the degree of influence of a pixel on the target convolution kernel.

[0177] Each pixel of the initial convolution kernel is used as a target pixel, and the value of each pixel is determined to obtain the target convolution kernel. The following example uses the point spread function (PSF) matrix as the target convolution kernel to illustrate the above calculation process.

[0178] ;

[0179] Where i and j are the row and column indices in the PSF matrix. x = j - center_x, y = i - center_y represent the horizontal and vertical distances of the current point relative to the center point of the PSF matrix. =center_x, =center_y, is the center (mean) of the Gaussian distribution, that is ( , The coordinates of the center point of the PSF matrix are given by σ. σ is the standard deviation of the Gaussian distribution. exp represents the natural exponential function. In this embodiment, the PSF matrix can be pre-calculated and stored in a ROM lookup table, thereby enabling fast calculation on the FPGA.

[0180] In one possible implementation, the image reconstruction unit also includes a state machine.

[0181] Taking the acquisition of the estimated image in round j and the estimated image in round j+1 as an example, where j is a positive integer, the state machine controls multiple multiply-accumulate units to stop iterative processing when the difference between the estimated image in round j and the estimated image in round j+1 is less than a preset threshold or the maximum number of iterations is reached, and the estimated image in round j+1 is used as the high-resolution reconstructed image.

[0182] The preset threshold is the upper limit of the difference used for table collection convergence. The maximum number of iterations is the preset number of iterations.

[0183] Therefore, by introducing a state machine into the image reconstruction unit to control the iterative process, the image measurement system can compare the differences between adjacent estimated images in real time. When the difference is less than the preset threshold for convergence of the table or the number of iterations reaches the preset maximum number of iterations, the iterative processing of multiple multiply-accumulate units is automatically stopped. This avoids the waste of computational resources caused by invalid or excessive iterations, prevents the iterative process from running for a long time or failing to converge, and achieves a good balance between accuracy and efficiency in the image reconstruction process.

[0184] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an image reconstruction unit provided in an embodiment of this application.

[0185] The data flow and coordination relationships between the functional modules are as follows. First, the input image caching module is used to cache and schedule multiple registration images from the registration unit. This module includes multiple Block RAMs and an address controller, which controls the alternating reading and writing of registration images between different storage areas through write and read signals.

[0186] On the output side of the input image buffer module, image data is fed into a multiply-accumulate (MAC) unit array consisting of multiple processing elements (PEs). The MAC unit array is distributed in a two-dimensional array, and the MAC units share data and perform parallel computation through cross-connection. Each MAC unit performs multiply-accumulate operations to complete the core numerical calculations required for forward or backward convolution. The pre-computed PSF matrix is ​​stored in read-only memory (ROM) and provided to the MAC unit array as needed during the operation to serve as the target convolution kernel, thereby ensuring the stability and consistency of the convolution process.

[0187] The intermediate results obtained after parallel computation by the MAC unit array are fed into the pipeline adder tree, which accumulates and integrates partial results from multiple processing units to form a complete convolution or error propagation result. Subsequently, this result enters the ratio calculation module and the parallel divider module, which convert the pixel relationship between multiple registered images and the simulated blurred image into a pixel-by-pixel ratio result, thereby characterizing the difference between the currently estimated image and the actual registered image.

[0188] The data after the ratio calculation is written to the output Block RAM for temporary storage, and then further input to the iterative update module. The iterative update module corrects and updates the previous round of input images based on the ratio results, and generates the estimated image for the next round.

[0189] In one possible embodiment, the shape fitting unit includes a shape extraction module and a fitting calculation module:

[0190] The shape extraction module is used to segment the target structure of the input image to obtain the shape parameters of the shape region to be tested. The input image is a high-resolution reconstructed image.

[0191] As mentioned above, a high-resolution reconstructed image is the result obtained by an image reconstruction unit through information fusion and iterative reconstruction based on multiple registered images. Compared to a single first image, a high-resolution reconstructed image has higher spatial resolution and richer detail information, and can more clearly reflect the true shape of the target structure.

[0192] The region to be tested is the image region corresponding to the target structure obtained from the input image through segmentation processing. The morphology parameters are parameter information extracted from the region to be tested to describe the geometric morphological features of the target structure.

[0193] Segmentation is a process of distinguishing the image region corresponding to the target structure from the background or other irrelevant regions. Its purpose is to determine the test morphology region for subsequent morphology fitting.

[0194] For example, the shape extraction module can analyze and process the image based on the grayscale distribution features, edge variation features, or geometric continuity features of the target structure in the high-resolution reconstructed image to identify the differences between the target structure and the background. For instance, the shape extraction module can first perform grayscale analysis on the high-resolution reconstructed image and determine candidate regions based on the differences in brightness or contrast between the target structure and the background. Alternatively, the shape extraction module can extract edge information based on locations with significant grayscale changes in the image and determine the contour region of the target structure based on the connectivity and overall orientation of the edges. Furthermore, the shape extraction module can combine the spatial continuity features of the target structure to aggregate adjacent pixels or adjacent regions, thereby forming a continuous shape region to be measured.

[0195] The fitting calculation module is used to fit the morphological parameters of the morphological region to be measured according to the least squares method, and the optimal fitting parameters obtained by the calculation are used as the morphological fitting parameters of the target structure.

[0196] The least squares method is a mathematical fitting method that determines the optimal solution by minimizing the sum of squared fitting errors. It can obtain a fitting result that reflects the overall trend of the target structure by combining the common constraints of multiple morphological parameters on the fitting result. The optimal fitting parameters are the morphological parameters obtained by the least squares method that minimize the overall fitting error, and they can be used as morphological fitting parameters for the target structure.

[0197] Therefore, by first segmenting the target structure of the high-resolution reconstructed image in the shape fitting unit, the shape region to be measured corresponding to the target structure is obtained and the shape parameters are extracted. Then, the shape parameters of the shape region to be measured are fitted as a whole based on the least squares method, so that the shape fitting process is based on the constraints of the high-resolution image and multiple pixels. This reduces the influence of background interference, local noise or other abnormal pixels on the fitting results, thereby obtaining more stable and accurate shape fitting parameters.

[0198] In one possible embodiment, the input device is signal-connected to an inertial measurement unit (IMU). The IMU is positioned near the image acquisition device or is fixedly installed with it, and is used to synchronously acquire corresponding attitude information during image acquisition, while the image acquisition device acquires the image to be measured.

[0199] The inertial measurement unit (IMU) outputs inertial measurement data corresponding to the attitude of the image acquisition device. The inertial measurement data includes at least one or more of roll angle, pitch angle, and yaw angle. Simultaneously with receiving the image to be measured, the input device acquires the inertial measurement data output by the IMU, associates the image to be measured with the corresponding inertial measurement data, and then inputs it to the processing device.

[0200] The processing device can determine the initial attitude information of the image acquisition device at the moment of image acquisition based on inertial measurement data, and use the initial attitude information as an auxiliary constraint condition for subsequent image processing.

[0201] For example, when registering multiple first images, the rotational relationship between the images can be estimated using the initial pose information to constrain the rotational search range during the registration process.

[0202] Furthermore, when multiple images to be tested are acquired continuously, if there is an attitude change between adjacent images due to equipment vibration or movement, the processing device can compensate for the attitude change based on inertial measurement data, thereby reducing the image rotation error caused by the shaking of the image acquisition device.

[0203] In another scenario, when the image under test is insufficient due to occlusion, blurring, or changes in ambient lighting, the processing device can still maintain a continuous estimate of the target structure's attitude change based on the attitude information output by the inertial measurement unit. After the image information is recovered, the inertial measurement data and the image information are used for joint correction.

[0204] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0205] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0206] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0207] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0208] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0209] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image measurement system, characterized in that, include: An input device is used to input an image to be tested, wherein the image to be tested is an image formed for a target structure; The processing device integrates a preprocessing unit, a shape fitting unit, and an angle calculation unit; the preprocessing unit is used to perform two-dimensional convolution filtering and sliding window value filtering on the image to be tested through a parallel filtering architecture to obtain a first image; the shape fitting unit is used to obtain the shape fitting parameters of the target structure in the first image by fitting using the least squares method. The angle calculation unit is used to compare the shape fitting parameters with the preset benchmark model parameters to obtain the angle measurement results of the target structure with respect to a specific direction; An output device is used to output and display the angle measurement results.

2. The image measurement system according to claim 1, characterized in that, The preprocessing unit includes a one-dimensional row-direction convolution module, a row-column buffer transformation module, and a one-dimensional column-direction convolution module. The one-dimensional row-direction convolution module is used to perform convolution operation on the image to be tested according to a preset filtered convolution kernel to obtain an initial feature map; The row and column buffer conversion module is used to cache the initial feature map through a ping-pong mechanism, and convert the initial feature map into a column direction before inputting it into the one-dimensional column direction convolution module; The one-dimensional column-direction convolution module is used to perform a convolution operation on the initial feature map according to the preset filter convolution kernel to obtain a two-dimensional feature map. The two-dimensional feature map is configured as the result of the two-dimensional convolution filtering performed by the preprocessing module.

3. The image measurement system according to claim 2, characterized in that, The preprocessing unit also includes a row buffer module, N deserializers, and a multi-layer comparator, where N is a positive integer; The row buffer module is used to perform pixel-by-pixel serial input to the two-dimensional feature map and obtain multiple pixel values ​​within a preset sliding window through a caching mechanism; The N deserializers are used to deserialize the multiple pixel groups in parallel, and input the deserialized pixel values ​​into the multi-layer comparator. The multi-layer comparator compares and sorts the input pixel values ​​of different sizes, and outputs the median value pixel by pixel. The median value output pixel by pixel is configured as the result of the sliding window mid-value filtering performed by the preprocessing module. The first image is formed by traversing the two-dimensional feature map using the preset sliding window and the result of sliding window mid-value filtering.

4. The image measurement system according to claim 3, characterized in that, The multilayer comparator includes a first-layer comparator, a second-layer comparator, and a third-layer comparator; The first layer comparator is used to compare the size of multiple deserialized pixel values ​​and sort them in three levels to obtain multiple first comparison results. The first low pixel value, the first middle pixel value and the first high pixel value in the multiple first comparison results are respectively input into the corresponding second layer comparator. The second layer comparator is used to perform size comparison and three-level sorting on the first low pixel value, first middle pixel value and first high pixel value in multiple first comparison results to obtain multiple second comparison results. The second low pixel value, second middle pixel value and second high pixel value in the multiple second comparison results are input into the third layer comparator. The third-layer comparator is used to compare the second comparison result and directly output the median value.

5. The image measurement system according to claim 1, characterized in that, If there are multiple images to be tested, the input device is specifically used for: The sequence of images to be tested is received serially via a dedicated input / output interface; The input unit includes a first storage buffer and a second storage buffer. The first storage buffer and the second storage buffer are used to alternately write the image to be tested and output the image to be tested to the preprocessing unit, so as to realize continuous acquisition and processing of the sequence of the image to be tested.

6. The image measurement system according to claim 1, characterized in that, If there are multiple images to be tested, the processing device further includes a registration unit and an image reconstruction unit; The registration unit is signal-connected to the preprocessing unit and obtains multiple first images from the preprocessing unit for sub-pixel-level alignment of the multiple first images to obtain multiple registration images. The image reconstruction unit is signal-connected to the registration unit and is used to perform information fusion and image reconstruction on multiple registered images to obtain a high-resolution reconstructed image. The high-resolution reconstructed image is input to the morphology fitting unit to obtain the morphology fitting parameters corresponding to the target structure in the first image by least squares fitting.

7. The image measurement system according to claim 6, characterized in that, The registration unit includes a corner detector, a feature matching module, and a transform estimation module; The corner detector is used to identify multiple first images to obtain key points and feature descriptors corresponding to the multiple first images respectively; The feature matching module configures one of the multiple first images as a reference image, and configures any one of the multiple first images other than the reference image as a target image; The feature matching module is used to match the key points and feature descriptors of the target image with the key points and feature descriptors of the reference image to obtain the matching relationship of the target image; The transformation estimation module is used to estimate the affine transformation matrix based on the matching relationship of the target image, and to resample the target image based on the affine transformation matrix using bilinear interpolation to obtain the sub-pixel information of the target image; The transformation estimation module registers the corresponding first image based on the sub-pixel information of each target image, and obtains multiple registered images through sub-pixel level alignment.

8. The image measurement system according to claim 6, characterized in that, The image reconstruction unit includes multiple parallel multiply-accumulate units; the multiple multiply-accumulate units are used to perform forward convolution, pixel ratio calculation and backward convolution operations on multiple registered images to obtain an estimated image, and to obtain the high-resolution reconstructed image through multiple rounds of iterative processing.

9. The image measurement system according to claim 8, characterized in that, In the i-th iteration, multiple multiply-accumulate units perform forward convolution processing on the estimated image of the (i-1)-th iteration through the target convolution kernel to obtain the simulated blurred image of the i-th iteration, where i is a positive integer; The multiple multiply-accumulate units calculate the pixel ratio between the multiple registered images and the simulated blurred image of the i-th round, and perform inverse convolution processing based on the result of the pixel ratio calculation to obtain the grayscale adjustment amount; The estimated image for the (i-1)th round is updated based on the grayscale adjustment amount to obtain the estimated image for the i-th round.

10. The image measurement system according to claim 9, characterized in that, The image reconstruction unit also includes a state machine; The state machine is used to obtain the estimated image of the j-th round and the estimated image of the (j+1)-th round, where j is a positive integer. When it is determined that the difference between the estimated image of the j-th round and the estimated image of the (j+1)-th round is less than a preset threshold or the maximum number of iterations is reached, the state machine controls the multiple multiply-accumulate units to stop iterative processing and uses the estimated image of the (j+1)-th round as the high-resolution reconstructed image.

11. The image measurement system according to claim 6, characterized in that, The morphology fitting unit includes a morphology extraction module and a fitting calculation module; The shape extraction module is used to segment the target structure in the input image to obtain the shape parameters of the shape region to be tested. The input image is the high-resolution reconstructed image. The fitting calculation module is used to fit the morphological parameters of the morphological region to be tested according to the least squares method, and the optimal fitting parameters obtained by the calculation are used as the morphological fitting parameters of the target structure.

12. The image measurement system according to claim 1, characterized in that, The angle calculation unit includes an included angle calculation module and a direction determination module; The included angle calculation module is used to obtain the current slope of the line segment to be tested in the target structure according to the shape fitting parameters of the target structure, and to obtain the reference slope of the reference line corresponding to the line segment to be tested in the target structure according to the reference model parameters of the target structure, and to calculate the angle offset based on the current slope and the reference slope. When the baseline corresponding to the line segment to be tested in the target structure points to the vertical direction, the angle calculation module calculates the angle between the line segment to be tested in the target structure and the vertical direction based on the shape fitting parameters as the angle offset. The direction determination module is used to obtain the average coordinates of the line segment to be tested in the target structure according to the shape fitting parameters, and to obtain the angular offset direction by comparing the position with the baseline corresponding to the line segment to be tested in the target structure. The angle offset and the angle offset direction are configured to form the angle measurement result of the target structure with respect to a specific direction.

13. The image measurement system according to claim 1, characterized in that, The processing device is equipped with a field-programmable gate array (FPGA), which implements the functions of the preprocessing unit, the shape fitting unit, and the angle calculation unit through multiple logic operation regions.

Images