Line scanning camera image correction method, device, equipment, medium and product

Abstract

The invention discloses a line scanning camera image correction method and device, equipment, a medium and a product. The method comprises the following steps: acquiring an original scanning image obtained by scanning a target object by a line scanning camera; determining the pixel size of the corrected image based on the target physical size and the target resolution of the target object; for each target pixel in the corrected image, determining a target physical coordinate of the target pixel according to a target pixel coordinate and a target resolution of the target pixel; according to a pre-generated coordinate mapping relation, determining a source pixel coordinate corresponding to the target physical coordinate in the original scanning image; wherein the coordinate mapping relation is a segmented continuous mapping relation determined based on analysis of a calibration image of a standard calibration board; and determining the pixel value of the target pixel in the corrected image according to the pixel data at the source pixel coordinate in the original scanning image. According to the embodiment of the invention, the composite distortion in the line scanning image can be systematically and automatically corrected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a line scan camera image correction method, device, equipment, medium and product. BACKGROUND

[0002] In the fields of semiconductor, electronic manufacturing and precision machining, high-precision visual inspection is a key link to ensure product quality. As an important sensor for obtaining high-resolution images, line scan cameras are widely used in continuous surface imaging of various products, especially for high-speed scanning detection of strip-shaped, sheet-shaped materials or regularly arranged elements (such as wafers, chips, PCB boards). Through synchronous relative motion between the line scan camera and the measured object, image data is collected line by line and spliced into a complete two-dimensional image, which can theoretically achieve high efficiency and high precision imaging.

[0003] However, in actual industrial field applications, due to the inherent characteristics of the hardware system, the precision of mechanical motion control, and environmental interference, the original images obtained by the line scan camera often have various inherent defects, which seriously affect the accuracy and reliability of subsequent image analysis, measurement and defect detection. These defects mainly manifest in the following four aspects:

[0004] 1. Image gap: Since line scan cameras are usually composed of multiple linearly arranged photosensitive chips, there are physical gaps between the chips. During scanning, these gaps will cause regular white bands in the collected image data at the corresponding positions, resulting in missing image information.

[0005] 2. Arrangement error: The precision limitation of mechanical transmission systems, installation errors, and non-ideal orthogonality between motion axes will cause translation, rotation or affine deformation of the scanned image, resulting in deviations between the positions and angles of the measured objects in the image and the theoretical coordinate system, affecting the precision of positioning and dimension measurement.

[0006] 3. Non-uniform speed distortion: The motion units (such as servo motors, linear motors) driving the measured object or the camera inevitably have instantaneous speed fluctuations, acceleration and deceleration jitter, or wear after long-term operation. The mismatch between speed and image acquisition line frequency will cause local stretching or compression distortion of the image in the motion direction, destroying the authenticity of the geometric shape.

[0007] 4. Brightness unevenness: The non-uniformity of the light source, the lens vignetting effect, the response difference of each pixel of the camera, and the change in the reflection characteristics of the object surface will cause the brightness or gray value of the final spliced image to show gradual or regional differences within the field of view, making it difficult to segment or extract features based on global thresholding.

[0008] For the above single problem, the industry has proposed several preliminary optimization processing methods, but there are limitations, and no systematic solution has been formed:

[0009] For image gaps, image gap filling methods such as bilinear interpolation or nearest neighbor interpolation are often used. Such methods only rely on the gray value of the pixels around the gap for mathematical interpolation, without considering key information such as texture continuity and structural features of the chip edge, resulting in poor consistency between the filled area and the original image in terms of texture and brightness, and easy appearance of blur, artifacts or transition discontinuity, affecting the accuracy of defect identification.

[0010] For arrangement errors, arrangement error correction methods are often used, which rely on manual marking of feature points or pre-set fixed coordinates for geometric transformation calibration. The manual marking method is inefficient and introduces subjective errors; the fixed coordinate method lacks flexibility and cannot adapt to the rapid changeover requirements of products of different sizes and different arrangement specifications, limiting its application in flexible manufacturing scenarios.

[0011] For uneven speed, speed unevenness compensation methods are often used, which are based on the ideal preset motion curve of the motor for software reverse interpolation correction. However, this method does not fully consider the real-time nonlinear errors caused by load changes, mechanical wear, control system delays and other factors in the actual operation of the motor, and the compensation model is too idealistic, making it difficult to completely eliminate complex dynamic distortion.

[0012] For uneven brightness, image uniformization processing methods such as global histogram equalization or gamma correction are widely used. Global histogram equalization enhances the contrast of the entire image without distinction, often amplifying background noise and leading to loss of local details; gamma correction mainly adjusts the overall gray response curve, and has limited improvement effect on complex local brightness differences caused by lighting or the sensor itself.

[0013] In summary, existing technical means usually deal with defects in one aspect of line scan camera images in isolation, and each has obvious shortcomings. In the face of high-precision detection needs, especially for complex microstructure and strict detection standards such as semiconductor chips, the residual or improper handling of any single defect can become a performance bottleneck. There is currently a lack of a comprehensive image optimization solution that can consider and systematically solve the four core problems of gaps, geometric distortion, motion distortion and uneven lighting. Therefore, developing an efficient, robust and automated line scan camera image comprehensive optimization method to obtain geometrically accurate, clearly textured and uniformly bright high-quality images at once has important theoretical and engineering significance for improving the visual detection capability of high-end equipment and meeting the precision detection needs of the Industry 4.0 era. SUMMARY

[0014] The application provides a line-scan camera image correction method, device, equipment, medium and product to solve the problem that the prior art usually processes a certain aspect of a line-scan camera image in isolation and has obvious deficiencies.

[0015] According to an aspect of the application, a line-scan camera image correction method is provided, comprising:

[0016] obtaining an original scan image obtained by scanning a target object by a line-scan camera;

[0017] determining a pixel size of a corrected image based on a target physical size and a target resolution of the target object;

[0018] for each target pixel in the corrected image, determining a target physical coordinate of the target pixel according to a target pixel coordinate and the target resolution of the target pixel;

[0019] determining a source pixel coordinate corresponding to the target physical coordinate in the original scan image according to a pre-generated coordinate mapping relationship; wherein the coordinate mapping relationship is a piecewise continuous mapping relationship determined based on analysis of a calibration image of a standard calibration board, and is used to compensate for nonlinear geometric distortion of the line-scan camera imaging system in the scanning direction and the motion direction;

[0020] determining a pixel value of the target pixel in the corrected image according to pixel data at the source pixel coordinate in the original scan image.

[0021] According to another aspect of the application, a line-scan camera image correction device is provided, comprising:

[0022] a scan image acquisition module configured to obtain an original scan image obtained by scanning a target object by a line-scan camera;

[0023] a correction size determination module configured to determine a pixel size of a corrected image based on a target physical size and a target resolution of the target object;

[0024] a physical coordinate determination module configured to, for each target pixel in the corrected image, determine a target physical coordinate of the target pixel according to a target pixel coordinate and the target resolution of the target pixel;

[0025] a source pixel coordinate determination module configured to determine a source pixel coordinate corresponding to the target physical coordinate in the original scan image according to a pre-generated coordinate mapping relationship; wherein the coordinate mapping relationship is a piecewise continuous mapping relationship determined based on analysis of a calibration image of a standard calibration board, and is used to compensate for nonlinear geometric distortion of the line-scan camera imaging system in the scanning direction and the motion direction;

[0026] a pixel value determining module configured to determine a pixel value of the target pixel in the corrected image according to pixel data at the source pixel coordinate in the original scan image.

[0027] According to another aspect of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of correcting a line scan camera image according to any embodiment of the present application.

[0028] According to another aspect of the present application, there is provided an electronic device comprising:

[0029] at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores a computer program which can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method of correcting a line scan camera image according to any embodiment of the present application.

[0030] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the method of correcting a line scan camera image according to any embodiment of the present application when executed by the processor.

[0031] According to another aspect of the present application, there is provided a computer program product comprising computer program / instructions which, when executed by a processor, implements the method of correcting a line scan camera image according to any embodiment of the present application.

[0032] The embodiments of the present application provide a complete image geometric correction solution for a line scan camera system by constructing and applying an accurate coordinate mapping relationship obtained based on a calibration board; the specifications of a corrected image are planned according to the physical size of a target object and a desired resolution, then the accurate source coordinates of each pixel position of the corrected image in an original distorted image are reversely calculated, and accurate mapping and reconstruction of pixel data are completed accordingly. The composite geometric distortion caused by lens distortion, perspective error and uneven scanning motion can be systematically compensated, the original scan image with nonlinear distortion can be restored to a non-distorted corrected image which strictly corresponds to the real physical size of the object in high fidelity, thereby providing a reliable and size-consistent data basis for subsequent precision measurement, positioning and defect detection.

[0033] It should be understood that the details described in this section are not intended to identify key or critical features of the embodiments of the present application, nor are they used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a first flowchart of a line scan camera image correction method provided in an embodiment of the present invention;

[0036] Figure 2 This is a second flowchart of a line scan camera image correction method provided in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the structure of a line scan camera image correction device provided in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of an electronic device that implements an embodiment of the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] Figure 1This is a first flowchart of a line scan camera image correction method provided in this embodiment of the invention. This embodiment is applicable to applications facing high-precision detection requirements, especially for complex microstructures such as semiconductor chips and applications with stringent detection standards. It is a comprehensive image optimization method that simultaneously solves problems such as gaps, geometric distortion, motion distortion, and uneven illumination in line scan camera images, aiming to obtain high-quality images with geometric accuracy, clear texture, and uniform brightness in a single operation. This method can be executed by a line scan camera image correction device, which can be implemented in hardware and / or software and can be configured in an electronic device with corresponding data processing capabilities. Figure 1 As shown, the method includes:

[0042] S110. Acquire the original scanned image of the target object obtained by scanning the target object with a line scan camera.

[0043] The target object is an industrial product to be inspected, such as a semiconductor chip, printed circuit board, or display panel. The original scan image is a two-dimensional digital image formed by a line scan camera acquiring and stitching data line by line as the target object moves continuously relative to the camera. This image may contain geometric distortions due to the inherent characteristics of the imaging system.

[0044] S120. Based on the target physical size and target resolution of the target object, determine the pixel size of the corrected image.

[0045] The target physical size is the known size of the object to be detected in the real world, such as length and width. The target resolution is a pre-set accuracy expected to be achieved in the corrected image, defined as the physical size represented by each pixel (e.g., 0.01 mm / pixel). Dividing the target physical size by the target resolution calculates the total number of pixels the corrected image should have in the corresponding dimension, thus determining its overall pixel size. For example, if the target resolution is 0.1 mm / px (i.e., 10 pixels per millimeter) and the target physical size is 50 mm * 50 mm, then the pixel width of the corrected image should be: 50 mm ÷ 0.1 mm / px = 500 pixels.

[0046] S130. For each target pixel in the corrected image, determine the target physical coordinates of the target pixel based on the target pixel coordinates and the target resolution.

[0047] For each target pixel in the corrected image, its target physical coordinates are determined based on its target pixel coordinates and target resolution. Specifically, the target pixel coordinates (such as row number and column number) are multiplied by the target resolution to obtain the precise physical location of the target pixel in the standard physical coordinate system; this is the target physical coordinate. For example, if the target pixel's row number is 250 and the target resolution is 0.1 mm / px, then the corresponding physical coordinates are: 250 × 0.1 mm / px = 25.0 mm. That is, the 250th row pixel in the corrected image will be mapped to a height of 25.0 mm in the standard physical coordinate system.

[0048] S140. Based on the pre-generated coordinate mapping relationship, determine the source pixel coordinates corresponding to the target physical coordinates in the original scanned image.

[0049] The coordinate mapping relationship is a piecewise continuous mapping relationship determined based on the analysis of the calibration image of the standard calibration plate, which is used to compensate for the nonlinear geometric distortion of the line scan camera imaging system in the scanning direction and the motion direction.

[0050] Coordinate mapping relationships are used to describe the geometric distortion characteristics of an imaging system. They are obtained through calibration of a standard calibration plate and describe the transformation relationship between pixel coordinates and physical coordinates. For any given target physical coordinates, by applying this coordinate mapping relationship for reverse calculation or lookup, it is possible to determine which pixel location (i.e., the source pixel coordinates) in the pixel coordinate system of the original scanned image acquired optical information from that physical location.

[0051] S150. Determine the pixel value of the target pixel in the corrected image based on the pixel data at the coordinates of the source pixel in the original scanned image.

[0052] Based on the source pixel coordinates, the grayscale or color value is read from the corresponding position in the original scanned image. If the source pixel coordinates are integers, they are read directly; if they are sub-pixel positions (non-integer coordinates), the pixel value at that position is calculated from neighboring pixels using image resampling algorithms such as bilinear interpolation. This pixel value is then assigned to the target pixel position currently being processed in the corrected image, thus completing the mapping and assignment of that point. If the source pixel coordinates are located within an invalid region of the image data in the original scanned image, the corresponding target pixel position is marked as missing data.

[0053] By repeating the above steps on all target pixels of the corrected image, and integrating all target pixel positions where pixel data and marker data are missing, a corrected image that strictly corresponds to the true physical size of the target object and eliminates the inherent geometric distortion of the system is finally generated.

[0054] Optionally, the process of generating the coordinate mapping relationship is as follows: A calibration image is obtained by scanning a standard calibration board using the line scan camera. The standard calibration board has positioning marks and scale lines with known physical distances. The calibration image is rotated and corrected to eliminate image distortion caused by camera installation or tilting of the calibration board. In the corrected calibration image, the positioning marks are located to determine spatial reference coordinates. Using the spatial reference coordinates as a reference point, a sub-image containing the scale lines is extracted from the calibration image. The pixel coordinates of each scale line in the sub-image are identified and extracted. The physical coordinates of each scale line are determined based on the spatial reference coordinates and the known physical distances. The coordinate mapping relationship between the pixel coordinates and the physical coordinates is determined based on the physical coordinates of each scale line and the pixel coordinates.

[0055] Acquire a calibration image obtained by scanning a standard calibration plate with the same line scan camera. The calibration plate is a precision physical tool with special markings for positioning and a series of scale lines with known, constant physical spacing printed on its surface.

[0056] The contour features of the calibration image are extracted using an edge detection algorithm to identify key edge lines (such as the four sides of the calibration image). The angle θ (accuracy 0.1°) between the edge lines and the horizontal / vertical coordinate axes (the camera's field of view coordinate system) of the image is calculated. The calibration image is then rotated and corrected based on the rotation transformation matrix to make the edges of the calibration image parallel to the coordinate axes. This eliminates the basic arrangement error caused by camera installation deviation or tilting of the calibration plate, and establishes a horizontal / vertical reference for subsequent accurate measurements. The rotation transformation formulas are shown in formulas (1)-(2) below:

[0057] x'=xcosθ−ysinθ (1)

[0058] y'=xsinθ+ycosθ (2)

[0059] Where (x,y) are the original pixel coordinates of the calibration image, and (x',y') are the pixel coordinates of the calibration image after rotation correction.

[0060] In the calibration image after rotation correction, a template matching algorithm is used to locate the positioning marks. Specifically, the corrected calibration image is subjected to normalized cross-correlation matching calculation with a pre-made high-quality template image corresponding to the positioning marks (such as specific corner marks) on the calibration board, generating a matching degree response matrix for the calibration image. By analyzing this matrix and setting a matching degree threshold (e.g., 0.85), regions with response values ​​higher than the threshold can be filtered out, and the region with the highest response value is determined as the target positioning region. The coordinates (X0, Y0) of the center point of the target positioning region in the pixel coordinate system of the calibration image are determined, and this point is defined as the mapping reference of the origin of the physical coordinate system in pixel space. This coordinate point is used as the spatial reference coordinate.

[0061] Using the spatial reference coordinates as the reference point, two specific rectangular local image regions are extracted from the rotation-corrected calibration image along the X direction (usually corresponding to the horizontal direction of the scanning sensor) and the Y direction (usually corresponding to the vertical direction of the object's movement) of the image coordinate system, namely the X-direction sub-image and the Y-direction sub-image.

[0062] The X-direction subgraph is a rectangular area centered on a reference point, extending a certain length (width) along the X-axis, and having a preset height. This area contains multiple vertically arranged scale lines. The Y-direction subgraph is a rectangular area centered on a reference point, extending a certain length (height) along the Y-axis, and having a preset width. This area contains multiple horizontally arranged scale lines.

[0063] These two orthogonal subgraphs will serve as precise samples for subsequent independent analysis of the scale mapping relationship between the X and Y directions.

[0064] The extracted X-direction and Y-direction sub-images are processed separately. In each sub-image, each tick mark is identified using image processing algorithms (such as binarization and connected component analysis), and the pixel coordinates of the center line of each tick mark in the image coordinate system of that sub-image are accurately calculated.

[0065] Based on known physical calibration data—that is, with the spatial reference coordinates as the origin of the physical coordinate system and the known, constant physical spacing between the scale lines—the true physical coordinates of each identified scale line in the standard physical coordinate system can be determined. Thus, for each direction (X and Y), a one-to-one data pair can be obtained, each pair containing a pixel coordinate value and its corresponding true physical coordinate value.

[0066] Using the data pairs obtained in each direction as modeling samples, a mathematical transformation relationship from pixel coordinates to physical coordinates is established. The average scale mapping coefficient in that direction can be calculated. Alternatively, to more accurately compensate for nonlinear distortion (especially in the Y direction), piecewise linear fitting or polynomial curve fitting methods can be used to construct a continuous mapping function between pixel coordinates and physical coordinates in that direction.

[0067] The transformation relationships between the X and Y directions are combined and encapsulated into a complete system coordinate mapping relationship. This mapping relationship accurately characterizes the overall geometric distortion characteristics of the current line scan camera imaging system and is saved as preset parameters for subsequent high-precision geometric correction of scanned images of arbitrary target objects.

[0068] Optionally, identifying and extracting the pixel coordinates of each tick mark in the sub-image includes: performing binarization processing on the sub-image using a dual-threshold binarization method combining global and local thresholding to generate a binarized image, thereby separating the tick marks from the background; identifying connected regions representing each tick mark in the binarized image; and calculating the center coordinates of each connected region as the pixel coordinates of the corresponding tick mark.

[0069] Optionally, the dual-threshold binarization method combining global and local thresholds is used to binarize the sub-image and generate a binarized image. This includes: performing a first round of binarization on the sub-image based on a global threshold to obtain a first binarization result; dividing the sub-image into multiple local sub-blocks, determining a local threshold for any local sub-block, performing a second round of binarization on the local sub-block using the local threshold to obtain a second binarization result; and performing a logical AND operation between the first binarization result and the second binarization result to generate the binarized image.

[0070] A dual-threshold binarization method combining global and local thresholding is employed to process sub-images, accurately separating tick marks (foreground) from background regions. Specifically, a first round of binarization is performed on the sub-image based on a global threshold, yielding a first binarization result. The global threshold is preferably automatically calculated using the Otsu algorithm, which adaptively determines an optimal threshold T1, thus initially dividing the image into foreground (tick marks) and background (gap regions). The sub-image is then divided into multiple local sub-blocks (e.g., 8×8 pixels or 16×16 pixels). For any given local sub-block, a local threshold T2i is determined based on the pixel distribution within that block, and this local threshold T2i is used to perform a second round of binarization, yielding a second binarization result. Using local thresholds for the second round of binarization of local sub-blocks can address the issue of local image contrast variations caused by uneven illumination. The first and second binarization results are then logically ANDed to generate the final binarized image.

[0071] By fusing the global and local binarization results, a final optimized binarized image was generated. This fusion strategy effectively combines the robust segmentation capability of the global threshold for the overall structure with the enhancement effect of the local adaptive threshold on the details of unevenly lit areas. This achieves high-contrast and high-fidelity separation of the scale lines and the background even in uneven lighting conditions, laying a solid image foundation for subsequent sub-pixel level coordinate extraction.

[0072] Based on this optimized binarized image, connected component analysis is performed to identify and extract all white connected regions representing independent tick marks. The coordinates of the center point of each connected region are calculated (e.g., through moment calculation or bounding box center), and these coordinates are used as the precise pixel position of the corresponding tick mark in the image. This positional information is the raw data necessary for constructing the pixel-physical coordinate mapping relationship.

[0073] This invention provides a complete image geometric correction solution for line scan camera systems by constructing and applying a precise coordinate mapping relationship obtained from calibration plate analysis. Based on the physical dimensions of the target object and the desired resolution, the specifications of the corrected image are planned. Then, the precise source coordinates of each pixel in the corrected image are calculated in reverse for the original distorted image, and the pixel data is accurately mapped and reconstructed accordingly. This systematically compensates for complex geometric distortions caused by lens distortion, perspective errors, and uneven scanning motion, faithfully restoring the original scanned image with nonlinear distortion to a distortion-free corrected image that strictly corresponds one-to-one with the true physical dimensions of the object. This provides a reliable and dimensionally consistent data foundation for subsequent precision measurement, positioning, and defect detection.

[0074] In one optional implementation, determining the coordinate mapping relationship between pixel coordinates and physical coordinates based on the physical coordinates of each tick mark and the pixel coordinates includes: determining a series of continuous coordinate intervals based on the pixel coordinates and physical coordinates of each tick mark, wherein each interval is defined by a pair of adjacent tick marks; for each coordinate interval, calculating the local scale mapping coefficient of the interval based on the difference between the pixel coordinates and the difference between the physical coordinates of the two tick marks at the two ends of the interval; wherein the local scale mapping coefficient represents the physical length corresponding to a unit pixel within the coordinate interval; and determining the coordinate mapping relationship between the pixel coordinates and physical coordinates of the coordinate interval based on the pixel coordinate interval and the physical coordinate interval of the two tick marks at the two ends of the coordinate interval, and the local scale mapping coefficient of the coordinate interval.

[0075] All detected tick marks are arranged in order of their physical coordinates or pixel coordinates. Based on each pair of adjacent tick marks, a series of coordinate intervals that are continuous in both physical and pixel space are determined. Each such interval is defined by a pair of adjacent tick marks at both ends, forming an independent analysis and mapping unit.

[0076] For each coordinate interval defined above, a unique local scale mapping coefficient is calculated based on the difference in pixel coordinates between the two ends of the interval and the corresponding physical coordinate difference. The specific formula is: Local scale mapping coefficient = (Physical coordinates of the end tick mark - Physical coordinates of the beginning tick mark) / (Pixel coordinates of the end tick mark - Pixel coordinates of the beginning tick mark). The local scale mapping coefficient represents the actual physical length represented by one pixel in the image within that specific interval. By calculating different scale mapping coefficients for different intervals, the spatial non-uniformity of image distortion can be accurately described and quantified.

[0077] By combining all coordinate intervals and their corresponding local scale mapping coefficients, a complete piecewise linear mapping model is constructed, serving as the coordinate mapping relationship between pixel coordinates and physical coordinates. This coordinate mapping relationship explicitly records the physical boundaries, pixel boundaries, and their internal proportional relationships for each interval, providing a precise mathematical basis for subsequent inverse coordinate mapping.

[0078] For example, Table 1 shows a piecewise mapping example based on five detected tick marks, illustrating a non-linear, piecewise defined correspondence between the physical world and the pixel world:

[0079] Table 1

[0080]

[0081] In Table 1, the local scale mapping coefficients (dx) in intervals 3 and 4 are significantly smaller than those in the first two intervals, revealing that the scanning motion system (such as a motor) slows down within these physical intervals, causing the same physical length to be "stretched" and occupy more pixels in the image. By establishing piecewise mapping relationships, rather than a single global scale mapping coefficient, it is possible to accurately model and compensate for this type of nonlinear distortion caused by uneven motion, which is key to achieving high-precision geometric reconstruction.

[0082] Figure 2 This is a second flowchart of a line scan camera image correction method provided in an embodiment of the present invention. This embodiment is an optimization and improvement based on the above embodiment. Figure 2 As shown, the method includes:

[0083] S210. Acquire the original scanned image of the target object obtained by scanning the target object with a line scan camera.

[0084] S220. Based on the target physical size and target resolution of the target object, determine the pixel size of the corrected image.

[0085] S230. For each target pixel in the corrected image, determine the target physical coordinates of the target pixel based on the target pixel coordinates and the target resolution.

[0086] S240. Determine the specific coordinate range to which the target's physical coordinates belong.

[0087] By comparing the target's physical coordinates with the physical ranges of each coordinate interval recorded in the coordinate mapping relationship, the specific coordinate interval into which the target's physical coordinates fall is determined.

[0088] S250, Obtain the local scale mapping coefficients corresponding to the specific coordinate interval, as well as the physical coordinates and pixel coordinates of the starting tick line of the interval.

[0089] From the mapping relationship corresponding to a specific coordinate interval, obtain three key parameters: the local scale mapping coefficient of the interval, the physical coordinates of the starting tick line of the interval, and the pixel coordinates of the starting tick line of the interval.

[0090] S260. Based on the target physical coordinates, the physical coordinates of the starting scale line, the local scale mapping coefficient, and the pixel coordinates of the starting scale line, the source pixel coordinates are calculated.

[0091] The coordinate mapping relationship is a piecewise continuous mapping relationship determined based on the analysis of the calibration image of the standard calibration plate, which is used to compensate for the nonlinear geometric distortion of the line scan camera imaging system in the scanning direction and the motion direction.

[0092] Based on the target physical coordinates, the obtained starting tick line physical coordinates, the local scale mapping coefficients, and the starting tick line pixel coordinates, the source pixel coordinates are calculated using the following formula (3).

[0093] Source pixel coordinates = pixel coordinates of the starting tick mark + (target physical coordinates - physical coordinates of the starting tick mark) / local scale mapping coefficient. (3)

[0094] Within a specific linear interval where the target's physical coordinates lie, taking the starting point of the interval as a reference, and based on the fixed local scale mapping coefficient of that interval, the offset of the physical coordinates is converted into the offset of the pixel coordinates, thereby locating the precise position in the original scanned image.

[0095] For example, if the target resolution of the corrected image is set to 0.1 mm / pixel, then for the 250th pixel (target pixel coordinates) in the corrected image, the target physical coordinates of the target pixel are determined to be: 250 × 0.1 mm / pixel = 25.0 mm based on the target pixel coordinates and the target resolution.

[0096] The coordinate intervals recorded in the coordinate mapping relationship are queried (see Table 1 in the previous embodiment). The physical coordinate 25.0 mm belongs to the third interval, and its physical range is from 20.0 mm to 30.0 mm.

[0097] Obtain from the mapping record of the 3rd interval:

[0098] Local scale mapping coefficient: dx3 = 0.0909 mm / pixel;

[0099] The physical coordinates of the starting scale line of the interval are P_phy_start = 20.0 mm;

[0100] The pixel coordinates of the starting tick mark of the interval are P_pix_start = 210.0 pixels;

[0101] The physical offset is calculated according to formula (3): ΔPhy = 25.0 mm - 20.0 mm = 5.0 mm;

[0102] Calculate the pixel offset: ΔPix = ΔPhy / dx3 = 5.0 mm / 0.0909 mm / pixel ≈ 55.0 pixels;

[0103] Calculate the source pixel coordinates: P_pix_source = P_pix_start + ΔPix = 210.0 pixels + 55.0 pixels = 265.0 pixels.

[0104] The calculation shows that the target's physical coordinates of 25.0 mm correspond to the position of 265.0 pixels in the original scanned image.

[0105] S270. Determine the pixel value of the target pixel in the corrected image based on the pixel data at the coordinates of the source pixel in the original scanned image.

[0106] Pixel data is obtained from the source pixel coordinates of the original scanned image (or calculated via interpolation if sub-pixel), and assigned to the 250th pixel of the corrected image. Through piecewise linear inverse mapping of the nonlinear distortion, the distorted image data is accurately reconstructed to its corresponding position in the standard physical size space. If the source pixel coordinates are non-integer coordinates, an image interpolation algorithm is used to obtain the corresponding pixel value from the original scanned image.

[0107] This invention achieves systematic software correction of the inherent geometric distortion of a line scan camera system by constructing a reverse reconstruction method based on physical size and utilizing piecewise coordinate mapping relationships resolved from precise calibration. It abandons the traditional approach of directly transforming the original distorted image. Instead, it first defines a standard distortion-free image space (corrected image) based on the true physical size and desired resolution of the object under test. For each target pixel in this standard space, by querying the precisely calibrated coordinate mapping relationship, it reversely locates the precise data source position (source pixel coordinates) in the original distorted image where the system distortion caused the offset, and completes the accurate transfer of pixel values. This "defining the standard first, then reverse tracing the source" strategy can compensate for nonlinear spatial distortion caused by lens distortion, installation errors, and uneven scanning speed with high fidelity, ultimately generating a distortion-free image that strictly corresponds one-to-one with the true physical size of the object and can be directly used for precise measurement and analysis, greatly improving the accuracy and reliability of the line scan camera-based detection system.

[0108] It should be noted that the foregoing embodiments and examples mainly describe how to map the target physical coordinates back to the source pixel coordinates using coordinate mapping relationships in one direction (e.g., the X or Y direction). In a complete two-dimensional image correction process, the coordinate mapping relationship includes mutually independent X-direction mapping and Y-direction mapping.

[0109] For a given target pixel, it has two-dimensional coordinates (X_new, Y_new) in the corrected image. By applying the coordinate mapping relationship in the X and Y directions respectively, its corresponding source pixel coordinates X_orig in the X direction and Y_orig in the Y direction in the original scanned image can be calculated independently. The combination of these two coordinate values ​​(X_orig, Y_orig) ultimately determines the precise two-dimensional position of the target pixel data in the original scanned image.

[0110] The correction process of this invention is essentially a combination of two one-dimensional inverse mappings. The two-dimensional physical coordinates of each target pixel are calculated separately. Then, using the independent coordinate mapping relationships in the X and Y directions, the corresponding rows and columns of these two physical coordinates in the original image are found, thereby accurately locating the source pixel. This design is particularly suitable for line scan camera systems because the distortion causes and characteristics in the X and Y directions are usually different, requiring independent modeling and correction.

[0111] Figure 3 This is a schematic diagram of the structure of a line scan camera image correction device provided in an embodiment of the present invention. Figure 3 As shown, the device includes:

[0112] The scan image acquisition module 310 is used to acquire the original scan image obtained by the line scan camera scanning the target object;

[0113] The correction size determination module 320 is used to determine the pixel size of the corrected image based on the target physical size and target resolution of the target object;

[0114] The physical coordinate determination module 330 is used to determine the target physical coordinates of each target pixel in the corrected image based on the target pixel coordinates and the target resolution.

[0115] The source pixel coordinate determination module 340 is used to determine the source pixel coordinates corresponding to the target physical coordinates in the original scan image according to the pre-generated coordinate mapping relationship; wherein, the coordinate mapping relationship is a piecewise continuous mapping relationship determined based on the analysis of the calibration image of the standard calibration plate, and is used to compensate for the nonlinear geometric distortion of the line scan camera imaging system in the scanning direction and motion direction.

[0116] The pixel value determination module 350 is used to determine the pixel value of the target pixel in the corrected image based on the pixel data at the coordinates of the source pixel in the original scanned image.

[0117] The line scan camera image correction device provided in this embodiment of the invention can execute the line scan camera image correction method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0118] Optionally, it also includes: a mapping relationship generation module, the mapping relationship generation module comprising:

[0119] The calibration image generation unit is used to acquire the calibration image obtained by the line scan camera scanning the standard calibration plate, wherein the standard calibration plate is provided with positioning marks and scale lines with known physical distances;

[0120] A rotation correction unit is used to perform rotation correction on the calibration image to eliminate image deflection caused by camera mounting or calibration plate placement tilt.

[0121] A reference coordinate determination unit is used to locate the positioning mark in the calibrated image after correction to determine the spatial reference coordinates;

[0122] A scale sub-image determination unit is used to extract a sub-image containing the scale lines from the calibration image, using the spatial reference coordinates as a reference point.

[0123] A tick mark pixel coordinate determination unit is used to identify and extract the pixel coordinates of each tick mark in the sub-image;

[0124] The physical coordinate determination unit for the scale line is used to determine the physical coordinates of each scale line based on the spatial reference coordinates and the known physical spacing.

[0125] The mapping relationship generation unit is used to determine the coordinate mapping relationship between pixel coordinates and physical coordinates based on the physical coordinates of each scale line and the pixel coordinates.

[0126] Optionally, the mapping relationship generation unit includes:

[0127] Interval division sub-units are used to determine a series of continuous coordinate intervals based on the pixel coordinates and physical coordinates of each tick mark, wherein each interval is defined by a pair of adjacent tick marks.

[0128] The interval coefficient generation subunit is used to calculate the local scale mapping coefficient of each coordinate interval based on the pixel coordinate difference and physical coordinate difference between the two ends of the interval; wherein, the local scale mapping coefficient represents the physical length corresponding to a unit pixel within the coordinate interval.

[0129] An interval mapping relationship generation sub-unit is used to determine the coordinate mapping relationship between the pixel coordinates and physical coordinates of the coordinate interval based on the pixel coordinate interval and physical coordinate interval of the two end scale lines of the coordinate interval, as well as the local scale mapping coefficient of the coordinate interval.

[0130] Optionally, the source pixel coordinate determination module includes:

[0131] The target interval determination unit is used to determine the specific coordinate interval to which the physical coordinates of the target belong.

[0132] The mapping relationship acquisition unit is used to acquire the local scale mapping coefficients corresponding to the specific coordinate interval, as well as the physical coordinates and pixel coordinates of the starting tick line of the interval;

[0133] The source pixel coordinate determination unit is used to calculate the source pixel coordinates based on the target physical coordinates, the physical coordinates of the starting tick line, the local scale mapping coefficient, and the pixel coordinates of the starting tick line.

[0134] Optionally, the scale line pixel coordinate determination unit includes:

[0135] The binarization processing subunit is used to perform binarization processing on the sub-image using a dual-threshold binarization processing method that combines global thresholding and local thresholding to generate a binarized image, so as to separate the scale lines from the background.

[0136] Connectivity region determination sub-units are used to identify connected regions representing each scale line in the binarized image;

[0137] The tick mark pixel coordinate determination subunit is used to calculate the center coordinates of each connected region as the pixel coordinates of the corresponding tick mark.

[0138] Optionally, the binarization processing subunit is specifically used for: performing a first round of binarization on the sub-image based on a global threshold to obtain a first binarization result; dividing the sub-image into multiple local sub-blocks, determining a local threshold for any local sub-block, performing a second round of binarization on the local sub-block using the local threshold to obtain a second binarization result; and performing a logical AND operation between the first binarization result and the second binarization result to generate the binarized image.

[0139] The line scan camera image correction device further described can also perform the line scan camera image correction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of performing the method.

[0140] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0141] Figure 4 A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0142] like Figure 4As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory 42 or a random access memory 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 42 or loaded from storage unit 48 into the random access memory 43. The random access memory 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, read-only memory 42, and random access memory 43 are interconnected via a bus 44. An input / output interface 45 is also connected to the bus 44.

[0143] Multiple components in electronic device 40 are connected to input / output interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0144] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as the correction method for line scan camera images.

[0145] In some embodiments, the line scan camera image correction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via read-only memory 42 and / or communication unit 49. When the computer program is loaded into random access memory 43 and executed by processor 41, one or more steps of the line scan camera image correction method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the line scan camera image correction method by any other suitable means (e.g., by means of firmware).

[0146] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube, liquid crystal display, or monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0150] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0151] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system to address the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0152] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for correcting images from a line scan camera, characterized in that, The method includes: Acquire the raw scanned image of the target object obtained by scanning it with a line scan camera; Based on the target physical size and target resolution of the target object, the pixel size of the corrected image is determined; For each target pixel in the corrected image, the target physical coordinates of the target pixel are determined based on the target pixel coordinates and the target resolution. Based on the pre-generated coordinate mapping relationship, the source pixel coordinates corresponding to the target physical coordinates in the original scan image are determined; wherein, the coordinate mapping relationship is a piecewise continuous mapping relationship determined based on the analysis of the calibration image of the standard calibration board, which is used to compensate for the nonlinear geometric distortion of the line scan camera imaging system in the scanning direction and the motion direction; The pixel value of the target pixel in the corrected image is determined based on the pixel data at the coordinates of the source pixel in the original scanned image.

2. The method according to claim 1, characterized in that, The process of generating the coordinate mapping relationship is as follows: Acquire a calibration image obtained by scanning a standard calibration plate with the line scan camera. The standard calibration plate is provided with positioning marks and scale lines with known physical distances. The calibration image is rotated to eliminate image distortion caused by camera mounting or calibration plate placement tilt. In the corrected calibration image, the positioning mark is located to determine the spatial reference coordinates; Using the spatial reference coordinates as a reference point, extract a sub-image containing the scale lines from the calibration image; Identify and extract the pixel coordinates of each tick mark in the sub-image; The physical coordinates of each scale line are determined based on the spatial reference coordinates and the known physical spacing. Based on the physical coordinates of each scale line and the pixel coordinates, the coordinate mapping relationship between the pixel coordinates and the physical coordinates is determined.

3. The method according to claim 2, characterized in that, The determination of the coordinate mapping relationship between pixel coordinates and physical coordinates based on the physical coordinates of each scale line and the pixel coordinates includes: Based on the pixel coordinates and physical coordinates of each tick mark, a series of continuous coordinate intervals are determined, where each interval is defined by a pair of adjacent tick marks. For each coordinate interval, the local scale mapping coefficient of that interval is calculated based on the pixel coordinate difference and physical coordinate difference between the two ends of the interval; wherein, the local scale mapping coefficient represents the physical length corresponding to a unit pixel within the coordinate interval; The coordinate mapping relationship between the pixel coordinates and physical coordinates of the coordinate interval is determined by the pixel coordinate interval and physical coordinate interval of the two end scale lines of the coordinate interval, as well as the local scale mapping coefficient of the coordinate interval.

4. The method according to claim 3, characterized in that, The step of determining the source pixel coordinates corresponding to the target physical coordinates in the original scanned image based on the pre-generated coordinate mapping relationship includes: Determine the specific coordinate range to which the target's physical coordinates belong; Obtain the local scale mapping coefficients corresponding to the specific coordinate interval, as well as the physical coordinates and pixel coordinates of the starting tick line of the interval; The source pixel coordinates are calculated based on the target physical coordinates, the physical coordinates of the starting tick line, the local scale mapping coefficient, and the pixel coordinates of the starting tick line.

5. The method according to claim 2, characterized in that, The process of identifying and extracting the pixel coordinates of each tick mark in the sub-image includes: A dual-threshold binarization method combining global and local thresholding is used to binarize the sub-image to generate a binarized image, thereby separating the scale lines from the background. Identify the connected regions representing each scale line in the binarized image; Calculate the center coordinates of each connected region, and use them as the pixel coordinates of the corresponding tick mark.

6. The method according to claim 5, characterized in that, The method employing a dual-threshold binarization approach combining global and local thresholding performs binarization on the sub-image to generate a binarized image, including: The subgraph is binarized in the first round based on a global threshold to obtain the first binarization result; The sub-image is divided into multiple local sub-blocks. For any local sub-block, a local threshold is determined. The local threshold is used to perform a second round of binarization on the local sub-block to obtain a second binarization result. The first binarization result and the second binarization result are subjected to a logical AND operation to generate the binarized image.

7. A correction device for line scan camera images, characterized in that, The device includes: The scan image acquisition module is used to acquire the original scan image obtained by the line scan camera scanning the target object; The correction size determination module is used to determine the pixel size of the corrected image based on the target physical size and target resolution of the target object; The physical coordinate determination module is used to determine the target physical coordinates of each target pixel in the corrected image based on the target pixel coordinates and the target resolution. The source pixel coordinate determination module is used to determine the source pixel coordinates corresponding to the target physical coordinates in the original scan image according to the pre-generated coordinate mapping relationship; wherein, the coordinate mapping relationship is a piecewise continuous mapping relationship determined based on the analysis of the calibration image of the standard calibration board, and is used to compensate for the nonlinear geometric distortion of the line scan camera imaging system in the scanning direction and motion direction; The pixel value determination module is used to determine the pixel value of the target pixel in the corrected image based on the pixel data at the coordinates of the source pixel in the original scanned image.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program executable by the at least one processor, which is executed by the at least one processor to enable the at least one processor to perform the correction method for line scan camera images according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the correction method for a line scan camera image according to any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements a method for correcting line scan camera images according to any one of claims 1-6.

Images