Image correction method and device, computer device and storage medium

Abstract

The application relates to an image correction method and device, computer equipment and a storage medium. The method comprises the following steps: performing shape fitting on the contour of a target object in a first image to determine shape parameters; determining the cylindrical parameter coordinates of target pixel points in a to-be-corrected region in a second image corresponding to the first pixel coordinates of the target pixel points and the shape parameters; determining the second pixel coordinates corresponding to each target pixel point according to the cylindrical parameter coordinates; and finally generating a corrected image corresponding to the second image according to the pixel values of the target pixel points and the second pixel coordinates. Thus, the deformed positions of characters or words on the target object in the second image are flattened, the second image is corrected, the image distortion caused by elliptical deformation or circular deformation is eliminated, and the recognition accuracy of the characters or words in the image by OCR is improved.

Description

Technical Field

[0001] This application relates to the field of image correction technology, and in particular to an image correction method, apparatus, computer device, and storage medium. Background Technology

[0002] For characters and symbols printed on surfaces with curved sides, whose side contours can be fitted to circles or ellipses (e.g., elliptical rings), it is impossible to capture the entire ring surface with a single camera due to occlusion or other reasons. Therefore, multiple cameras are typically used to photograph such objects before OCR (Optical Character Recognition) to meet the requirement of fully recognizing all characters and symbols. Because such object surfaces cause image distortion during photography, traditional OCR algorithms struggle to recognize distorted characters and symbols, increasing the probability of detection errors. Summary of the Invention

[0003] Therefore, it is necessary to provide an image correction method, apparatus, computer equipment, and storage medium that can correct deformed characters or symbols in images of objects with curved sides and side contours that can be fitted into circles or ellipses to improve the recognition accuracy of OCR.

[0004] Firstly, this application provides an image correction method. The method includes:

[0005] Acquire a first image and a second image of the target object; the first image contains the cross-sectional outline of the target object; the second image contains the area of ​​the target object to be corrected.

[0006] Shape fitting is performed on the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system, where the first pixel coordinate system is the pixel coordinate system of the first image;

[0007] Based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system, the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system are determined; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image;

[0008] Based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range, determine the second pixel coordinates corresponding to the target pixel in the preset pixel range;

[0009] Based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel, the corrected image corresponding to the second image is generated.

[0010] In one embodiment, determining the cylindrical parameter coordinates of the target pixel in the world coordinate system based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system includes:

[0011] A first coordinate transformation relationship is determined, and based on the first coordinate system transformation relationship, the shape parameters corresponding to the target object in the first pixel coordinate system are transformed into the shape parameters corresponding to the target object in the world coordinate system; the first coordinate system transformation relationship is the transformation relationship between the first pixel coordinate system and the world coordinate system;

[0012] A second coordinate transformation relationship is determined, and based on the second coordinate system transformation relationship, the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system are transformed into the world coordinates corresponding to the target pixel in the world coordinate system; the second coordinate transformation relationship is the transformation relationship between the second pixel coordinate system and the world coordinate system;

[0013] Based on the shape parameters of the target object in the world coordinate system and the world coordinates of the target pixel in the world coordinate system, determine the cylindrical parameter coordinates of the target pixel in the world coordinate system.

[0014] In one embodiment, the cylindrical parameter coordinates include parameter coordinates and height coordinates; determining the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system based on the shape parameters corresponding to the target object in the world coordinate system and the world coordinates corresponding to the target pixel in the world coordinate system includes:

[0015] For each target pixel, the world coordinates are determined based on the two-dimensional plane coordinates of the world coordinates and the shape parameters of the target object in the world coordinate system. The shape parameters corresponding to the two-dimensional plane coordinates are then defined as the parameter coordinates. The two-dimensional plane coordinates represent the position of the world coordinates in a plane parallel to the X-axis and Y-axis of the world coordinate system.

[0016] The Z-axis coordinate corresponding to the two-dimensional plane coordinate in the world coordinate system is determined as the height coordinate.

[0017] In one embodiment, determining the first coordinate transformation relationship includes:

[0018] Obtain the camera intrinsic and extrinsic parameters corresponding to the first image;

[0019] The first coordinate transformation relationship is determined based on the camera intrinsic and extrinsic parameters corresponding to the first image.

[0020] In one embodiment, determining the second coordinate transformation relationship includes:

[0021] Obtain the camera intrinsic and extrinsic parameters corresponding to the second image;

[0022] The second coordinate transformation relationship is determined based on the camera intrinsic and extrinsic parameters corresponding to the second image.

[0023] In one embodiment, the cylindrical parameter coordinates include parameter coordinates and height coordinates; determining the second pixel coordinates corresponding to the target pixel within the preset pixel range based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range includes:

[0024] Based on the cylindrical parameter coordinates corresponding to the target pixel, determine the parameter difference between the maximum and minimum values ​​of the parameter coordinates and the height difference between the maximum and minimum values ​​of the height coordinates;

[0025] The second pixel coordinates corresponding to the target pixel are determined based on the parameter difference, the height difference, the cylindrical parameter coordinates, and the preset pixel range.

[0026] In one embodiment, the step of performing shape fitting on the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system includes:

[0027] Shape fitting is performed on the target object in the first image to determine the shape fitting equation in the first pixel coordinate system;

[0028] The equation parameters of the shape fitting equation in the first pixel coordinate system are determined as the shape parameters corresponding to the target object in the first pixel coordinate system.

[0029] In one embodiment, the image correction method further includes:

[0030] In the second image, the objects to be corrected on the target object are selected by drawing a box to obtain the selected area;

[0031] The selected area is taken as the area to be corrected.

[0032] Secondly, this application also provides an image correction device. The device includes:

[0033] The acquisition module is used to acquire a first image and a second image of the target object; the first image contains the cross-sectional outline of the target object; the second image contains the area of ​​the target object to be corrected.

[0034] A shape fitting module is used to perform shape fitting on the target object in the first image and determine the shape parameters of the target object in the first pixel coordinate system, wherein the first pixel coordinate system is the pixel coordinate system of the first image;

[0035] The cylindrical parameter coordinate determination module is used to determine the cylindrical parameter coordinates of the target pixel in the world coordinate system based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image;

[0036] The pixel coordinate determination module is used to determine the second pixel coordinates corresponding to the target pixel in the preset pixel range based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range.

[0037] The generation module is used to generate the corrected image corresponding to the second image based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel.

[0038] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0039] Acquire a first image and a second image of the target object; the first image contains the cross-sectional outline of the target object; the second image contains the area of ​​the target object to be corrected.

[0040] Shape fitting is performed on the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system, where the first pixel coordinate system is the pixel coordinate system of the first image;

[0041] Based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system, the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system are determined; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image;

[0042] Based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range, determine the second pixel coordinates corresponding to the target pixel in the preset pixel range;

[0043] Based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel, the corrected image corresponding to the second image is generated.

[0044] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0045] Acquire a first image and a second image of the target object; the first image contains the cross-sectional outline of the target object; the second image contains the area of ​​the target object to be corrected.

[0046] Shape fitting is performed on the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system, where the first pixel coordinate system is the pixel coordinate system of the first image;

[0047] Based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system, the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system are determined; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image;

[0048] Based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range, determine the second pixel coordinates corresponding to the target pixel in the preset pixel range;

[0049] Based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel, the corrected image corresponding to the second image is generated.

[0050] The aforementioned image correction method, apparatus, computer equipment, and storage medium determine shape parameters by fitting the contour of the target object in the first image. Based on the first pixel coordinates of the target pixel in the area to be corrected in the second image and the shape parameters, they determine the cylindrical parameter coordinates of the target pixel in the world coordinate system. Based on the cylindrical parameter coordinates, they determine the second pixel coordinates of each target pixel. Finally, based on the pixel values ​​of the target pixels and the second pixel coordinates, they generate the corrected image corresponding to the second image. This process flattens the deformed positions of characters or text on the target object in the second image, correcting the second image and eliminating graphic distortions caused by elliptical or circular deformations, thereby increasing the accuracy of character or text recognition in OCR images. Attached Figure Description

[0051] Figure 1 This is an application environment diagram of the image correction method in one embodiment;

[0052] Figure 2 This is a flowchart illustrating an image correction method in one embodiment;

[0053] Figure 3 This is a flowchart illustrating the steps for determining the coordinates of cylindrical parameters in one embodiment;

[0054] Figure 4 This is a structural block diagram of an image correction device in one embodiment;

[0055] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] The image correction method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0058] In one embodiment, such as Figure 2 As shown, an image correction method is provided, including the following steps:

[0059] Step 202: Obtain a first image and a second image of the target object; the first image contains the cross-sectional outline of the target object; the second image contains the area of ​​the target object to be corrected.

[0060] The target object, viewed from a certain angle, has an elliptical or circular outline, or a near-elliptical or near-circular shape. An elliptical shape can be approximated as an ellipse, and a near-circular shape can be approximated as a circle. The first image is taken from this angle. The target object can be an elliptical ring, a circular ring, an elliptical sphere, an elliptical cylinder, etc. If the target object is a ring, the outline can be either an outer or inner outline. The second image is the image to be corrected. Since the surface of an elliptical object will inevitably have a curved surface, the characters or text in the image obtained by photographing characters on the curved surface of these objects will inevitably be distorted. This application aims to correct the area containing these characters or text. The correction area contains the characters or text that need to be corrected. The size of the correction area can be determined according to the actual situation; it can be the entire second image, the area containing the target object in the second image, or only the area including the characters or text.

[0061] Step 204: Perform shape fitting on the target object in the first image to determine the shape parameters corresponding to the target object in the first pixel coordinate system, where the first pixel coordinate system is the pixel coordinate system of the first image.

[0062] In this context, the first pixel coordinate system is the coordinate system of the first image. Shape fitting of the target object is performed within this first pixel coordinate system to obtain shape parameters. Different types of fitting can be performed depending on the specific situation. For example, the outer contour of the target object in the first image can be fitted. If the target object is ring-shaped, the contour to be fitted can be determined based on the position of the correction area on the target object. For instance, if the correction area is on the outer surface of the ring, the outer contour can be fitted; if the correction area is on the inner surface, the inner contour can be fitted. Alternatively, the shape can be fitted based on the overall shape of the ring. Shape parameters include ellipse equation parameters or circle equation parameters, specifically determined by the fitted shape of the target object. If the fitted shape of the target object is elliptical or quasi-elliptical, then the fitted shape is elliptical, and the shape parameter is the ellipse equation parameter. If the outer contour of the target object is circular or quasi-circular, then the fitted shape is circular, and the shape parameter is the circle equation parameter. The shape parameters represent the pixel coordinates of each point of the outer or inner contour of the target object in the first image, and are used to subsequently construct the cylindrical parameter coordinates.

[0063] Step 206: Determine the cylindrical parameter coordinates of the target pixel in the world coordinate system based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image.

[0064] In this system, the second pixel coordinate system is the coordinate system of the second image, and the first pixel coordinate system refers to the pixel coordinates of the target pixel in the second image within the second pixel coordinate system. The pixel coordinates of each target pixel in the second pixel coordinate system can be converted to world coordinates, which represent the position of the actual object corresponding to that pixel in the world coordinate system. By combining the coordinates of the first and second images, the pixel coordinates corresponding to the two images are converted to coordinates in the world coordinate system, thus enabling coordinate merging.

[0065] Specifically, the first pixel coordinates of the target pixel in the second pixel coordinate system are converted into world coordinates in the world coordinate system, meaning that the first pixel coordinates of each target pixel have corresponding world coordinates. The shape parameters of the target object in the first pixel coordinate system are also converted into shape parameters in the world coordinate system. This application approximates the surface of the target object captured in the second image as a cylinder. Based on the fitted shape of the target object in the first image, this cylinder can be considered an elliptical cylinder or a circular cylinder. If the fitted shape is elliptical or quasi-elliptical, the cylinder is approximated as an elliptical cylinder; if the fitted shape is circular or quasi-circular, the cylinder is approximated as a circular cylinder. The cylinder parameter coordinates corresponding to the cylinder equation are constructed using the shape parameters and the world coordinates. The shape parameters are the equation parameters of the cylinder equation. They can be viewed as the positions of the cylinder's coordinate points on the xoy plane (equivalent to the horizontal plane) of the world coordinate system (the x and y coordinates of each point on the cylinder in the world coordinate system can be represented by the equation parameters). Combining the three axes of the world coordinate system, we can determine the z-coordinate of the cylinder. By combining the shape parameters corresponding to the target object in the world coordinate system with the world coordinates, we can determine the cylinder parameter coordinates. The cylinder parameter coordinates represent the position of the target object's surface captured in the second image in the world coordinate system. Since the world coordinates also represent the coordinates of the target object's surface corresponding to the correction area, the positions corresponding to the world coordinates are the same as the positions corresponding to the cylinder parameter coordinates; that is, each first pixel coordinate corresponds to one cylinder parameter coordinate.

[0066] Step 208: Determine the second pixel coordinates corresponding to the target pixel within the preset pixel range based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range.

[0067] Here, the second pixel coordinates are the pixel coordinates of the corrected image to be obtained, and the preset pixel range is the pixel range of the corrected image. The second pixel coordinates, corresponding one-to-one with the cylindrical parameter coordinates, can be obtained by constructing the cylindrical parameter coordinates of the target pixel point using the shape parameters corresponding to the target object in the world coordinate system and the world coordinates corresponding to the target pixel point in the world coordinate system. Since the cylindrical parameter coordinates and the pixels in the corrected area are in a one-to-one correspondence, the second pixel coordinates and the pixels in the corrected area are also in a one-to-one correspondence. In one embodiment, each second pixel coordinate is determined based on the range of each coordinate of the cylindrical parameter coordinates and the preset pixel range of the corrected image.

[0068] Step 210: Generate the corrected image corresponding to the second image based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel.

[0069] Here, a pixel refers to a pixel in the correction area of ​​the second image. Since each pixel corresponds to each second pixel through cylindrical parameter coordinates, the pixel value of the pixel corresponding to each second pixel coordinate can be determined based on the pixel value of the pixel in the area to be corrected, thereby generating the corrected image. The corrected image is the image formed by tiling the deformed area to be corrected in the second image.

[0070] In the aforementioned image correction method, shape parameters are determined by shape fitting of the outline of the target object in the first image. Based on the first pixel coordinates of the target pixel in the area to be corrected in the second image and the shape parameters, the cylindrical parameter coordinates of the target pixel in the world coordinate system are determined. The second pixel coordinates of each target pixel are then determined based on the cylindrical parameter coordinates. Finally, the corrected image corresponding to the second image is generated based on the pixel value of the target pixel and the second pixel coordinates. This process flattens the deformed positions of characters or text on the target object in the second image, correcting the second image and eliminating graphic distortion caused by elliptical or circular deformation, thereby increasing the accuracy of character or text recognition in OCR images.

[0071] In one embodiment, such as Figure 3 As shown, based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system, the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system are determined, including:

[0072] Step 302: Determine the first coordinate transformation relationship, and based on the first coordinate system transformation relationship, transform the shape parameters corresponding to the target object in the first pixel coordinate system into the shape parameters corresponding to the target object in the world coordinate system; the first coordinate system transformation relationship is the transformation relationship between the first pixel coordinate system and the world coordinate system.

[0073] Step 304: Determine the second coordinate transformation relationship, and based on the second coordinate system transformation relationship, transform the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system into the world coordinates corresponding to the target pixel in the world coordinate system; the second coordinate transformation relationship is the transformation relationship between the second pixel coordinate system and the world coordinate system.

[0074] Step 306: Determine the cylindrical parameter coordinates of the target pixel in the world coordinate system based on the shape parameters of the target object in the world coordinate system and the world coordinates of the target pixel in the world coordinate system.

[0075] Specifically, based on the corresponding coordinate transformation relationships, the shape parameters of the first pixel coordinate system and the first pixel coordinates of the second pixel coordinate system are transformed into shape parameters and world coordinates in the world coordinate system. The first and second coordinate transformation relationships are obtained by calibrating the cameras that captured the first and second images in advance. By determining the first and second coordinate transformation relationships, the shape parameters of the first pixel coordinate system and the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system can be transformed into values ​​in the same coordinate system, thereby combining the two coordinate systems to obtain the cylindrical parameter coordinates.

[0076] In one embodiment, during the determination of the world coordinates of the target object, each pixel in the second image can have its world coordinates determined by the intersection of a ray formed by the camera center and the cylindrical surface of the target object. When the target object is an elliptical ring, a near-elliptical ring, a circular ring, or a near-circular ring, the ray intersects the cylindrical surface of the target object at two points: one intersection point is located on the outer surface of the ring, and the other intersection point is located on the inner surface of the ring. If the correction area is located on the outer surface of the ring, then the point closer to the camera center is selected to remove the other intersection point; if the correction area is located on the inner surface of the ring, then the point farther from the camera center is selected to remove the other intersection point. In some embodiments, the position of the point to be selected is determined based on user input parameters.

[0077] In one embodiment, the cylindrical parameter coordinates include parameter coordinates and height coordinates; determining the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system based on the shape parameters corresponding to the target object in the world coordinate system and the world coordinates corresponding to the target pixel in the world coordinate system includes:

[0078] Step 402: For the world coordinates corresponding to each target pixel, determine the shape parameters corresponding to the two-dimensional plane coordinates based on the two-dimensional plane coordinates of the world coordinates and the shape parameters corresponding to the target object in the world coordinate system, and define the shape parameters corresponding to the two-dimensional plane coordinates as parameter coordinates; the two-dimensional plane coordinates represent the position of the world coordinates on the plane parallel to the X-axis and Y-axis of the world coordinate system.

[0079] Step 404: Determine the Z-axis coordinate corresponding to the two-dimensional plane coordinate in the world coordinate system as the height coordinate.

[0080] Specifically, since the X-axis and Y-axis coordinates of each point on the elliptical or cylindrical surface can be represented by the same parameter (x = asinθ, y = bconθ, where x and y are the X-axis and Y-axis coordinates respectively, a and b are the lengths of the major and minor axes of the ellipse respectively, and θ is the parameter), this parameter is the corresponding shape parameter. Each combination of X-axis and Y-axis coordinates can be considered a two-dimensional planar coordinate. Two-dimensional planar coordinates can be represented by the corresponding shape parameter. Combining the corresponding shape parameter with the corresponding Z-axis coordinate yields the cylindrical parameter coordinates. These cylindrical parameter coordinates have two dimensions and can be considered a two-dimensional coordinate, thus transforming the world coordinates from three dimensions to two dimensions. This allows for a simpler method to establish the mapping relationship between the cylindrical parameter coordinates and the second pixel coordinates.

[0081] For example, if the world coordinates of a point on an elliptical cylinder are (x1, y1, z1), then the corresponding two-dimensional plane coordinates are (x1, y1), the Z-axis coordinates corresponding to the two-dimensional plane coordinates are z1, and the shape parameter corresponding to the two-dimensional plane coordinates (x1, y1) is θ1 (from the parametric equation of the elliptical cylinder, we know: x1 = asinθ1, y1 = bconθ1), then the cylindrical parametric coordinates are (θ1, z1).

[0082] In one embodiment, determining the first coordinate transformation relationship includes:

[0083] Step 502a: Obtain the camera intrinsic and extrinsic parameters corresponding to the first image;

[0084] Step 504a: Determine the first coordinate transformation relationship based on the camera intrinsic and extrinsic parameters corresponding to the first image;

[0085] In one embodiment, determining the second coordinate transformation relationship includes:

[0086] Step 502b: Obtain the camera intrinsic and extrinsic parameters corresponding to the second image;

[0087] Step 504b: Determine the second coordinate transformation relationship based on the camera intrinsic and extrinsic parameters corresponding to the second image.

[0088] Before acquiring the corresponding images using a camera, camera calibration is required for both the cameras that captured the first and second images to obtain the transformation relationship between the pixel coordinate system and the world coordinate system. The camera corresponding to the first image refers to the camera that captured the first image, and the camera corresponding to the second image refers to the camera that captured the second image. The cameras corresponding to the first and second images may be the same or different.

[0089] Specifically, a calibration board is used to calibrate the intrinsic and extrinsic parameters of all cameras. If the same camera is used to capture the first image and the second image from different angles, the camera needs to be recalibrated each time the camera pose changes. During the extrinsic parameter calibration, the calibration board is placed on a designated plane. Assuming the z-direction is the plane normal to the designated plane, the camera takes a picture of the calibration board, the camera pose relative to the calibration board is calculated, and the thickness h of the calibration board is measured. The plane equation of the designated plane is then z = -h. After translating the camera pose, the plane equation of the designated plane becomes z = 0. During image correction, the target object is placed on this designated plane and the first and second images are captured.

[0090] In one embodiment, the first coordinate transformation relationship is a first homography transformation matrix, and the second coordinate transformation relationship is a second homography transformation matrix. The homography transformation matrix is ​​calculated using camera intrinsic and extrinsic parameters. After camera calibration, the camera intrinsic parameter matrix is ​​K, and the camera extrinsic parameters include pose rotation R and pose translation t. Therefore, A = KR, b = Kt, and the resulting homography transformation matrix is:

[0091]

[0092] Where A(0,0) represents the 0th row and 0th column of matrix A, A(0,1) represents the 0th row and 1st column of matrix A, and so on. In some embodiments, the non-standard equation of the ellipse is ax 2 +bxy+cy 2 The equation ∫dx + ey + f = 0, where a, b, c, d, e, and f are coefficients, is also applicable to non-standard equations of circles. The parameters of the parametric equation corresponding to a non-standard equation of an ellipse can be represented by a 3x3 symmetric matrix C:

[0093]

[0094] Where C represents the equation parameters in the first pixel coordinate system. The transformation between the first pixel coordinate system and the world coordinate system, i.e., the effect of the first homography transformation matrix on the ellipse equation, is: C' = H -T CH -1 , where H is the first homography transformation matrix and C' is the equation parameter in the world coordinate system.

[0095] In one embodiment, the cylindrical parameter coordinates include parameter coordinates and height coordinates; determining the second pixel coordinates corresponding to the target pixel within the preset pixel range based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range includes:

[0096] Step 602: Based on the cylindrical parameter coordinates corresponding to the target pixel, determine the parameter difference between the maximum and minimum values ​​of the parameter coordinates and the height difference between the maximum and minimum values ​​of the height coordinates.

[0097] Step 604: Determine the second pixel coordinates corresponding to the target pixel based on the parameter difference, height difference, cylindrical parameter coordinates, and preset pixel range.

[0098] Specifically, from the cylindrical parametric coordinates of all target pixels, the maximum and minimum values ​​of the parametric coordinates and the maximum and minimum values ​​of the height coordinates are found. These extreme values ​​of the parametric and height coordinates constitute the pixel range of the target object's correction area. The preset pixel range is a pre-defined pixel range of the corrected image. Based on the pixel range of the correction area and the preset pixel range, a mapping relationship between the cylindrical parametric coordinates and the second pixel coordinates is established. The coordinate values ​​of the second pixel coordinates corresponding to the cylindrical parametric coordinates are determined based on this mapping relationship.

[0099] In one embodiment, assuming the number of pixel rows in the preset pixel range is r, the number of pixel columns is c, the second pixel coordinate of the corrected image is (x, y), the cylindrical parameter coordinates are represented as (θ, h), the parameter difference is Δθ, and the height difference is Δh, then x = θr / Δθ, y = hc / Δh.

[0100] In one embodiment, shape fitting is performed on the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system, including:

[0101] Step 702: Perform shape fitting on the target object in the first image and determine the shape fitting equation in the first pixel coordinate system;

[0102] Step 704: Determine the equation parameters of the shape fitting equation in the first pixel coordinate system as the shape parameters corresponding to the target object in the first pixel coordinate system.

[0103] Specifically, the shape fitting equation is the curve equation of the fitted shape of the target object in the first image in the first pixel coordinate system. Depending on the fitted shape, the shape fitting equation is determined to be either an elliptic parametric equation or a circular parametric equation. If the fitted shape is elliptical, the shape fitting equation is an elliptic parametric equation; if the fitted shape is circular, the shape fitting equation is a circular parametric equation. After determining the shape fitting equation, the shape parameters corresponding to each contour pixel coordinate can be determined based on the contour pixel coordinates in the first pixel coordinate system. These shape parameters are the shape parameters corresponding to the target object. In some embodiments, the shape parameters of the shape fitting equation can be represented by a 3*3 symmetric matrix C as follows:

[0104]

[0105] Where C represents the equation parameters in the first pixel coordinate system, and a, b, c, d, e, and f are the equation coefficients of the non-standard equation corresponding to the shape fitting equation. The non-standard equation is: ax 2 +bxy+cy 2 +dx+ey+f=0.

[0106] In one embodiment, generating a corrected image corresponding to a second image based on the pixel value of a target pixel and the second pixel coordinates corresponding to each target pixel includes: interpolating the pixel value of the target pixel to the second pixel coordinates corresponding to the target pixel to generate the corrected image corresponding to the second image.

[0107] Specifically, the pixel values ​​corresponding to the pixels in the second image are sampled according to the first pixel coordinates, and the corresponding pixel values ​​are interpolated to the pixel corresponding to the second pixel coordinates of the target pixel to generate the corrected image corresponding to the second image.

[0108] In one embodiment, the image correction method further includes:

[0109] Step 802: Select the objects to be corrected on the target object in the second image to obtain the selected area;

[0110] Step 804: Select the area to be corrected.

[0111] Specifically, the correction area can be narrowed down to a smaller range for the specific object requiring correction. A bounding box can be selected to define the selected area. Any method can be used to select the object, such as using an OCR algorithm to create a polygonal selection box. The area within this polygonal selection box is the selected region. This selected region is then used as the area to be corrected, allowing processing only the pixel coordinates within that region, reducing computational load and improving correction efficiency. In one embodiment, the maximum and minimum values ​​of the cylindrical parameter coordinates, as well as the maximum and minimum values ​​of the height coordinates, can be found from the first pixel coordinates corresponding to the vertex of the selection box. This narrows the search range for coordinate extrema and further improves correction efficiency.

[0112] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0113] Based on the same inventive concept, this application also provides an image correction apparatus for implementing the image correction method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image correction apparatus embodiments provided below can be found in the limitations of the image correction method described above, and will not be repeated here.

[0114] In one embodiment, such as Figure 4 As shown, an image correction device 400 is provided, including: an acquisition module 401, a shape fitting module 402, a cylindrical parameter coordinate determination module 403, a pixel coordinate determination module 404, and a generation module 405, wherein:

[0115] The acquisition module 401 is used to acquire a first image and a second image of the target object; the first image contains the cross-sectional outline of the target object; and the second image contains the area of ​​the target object to be corrected.

[0116] The shape fitting module 402 is used to perform shape fitting on the target object in the first image and determine the shape parameters corresponding to the target object in the first pixel coordinate system, where the first pixel coordinate system is the pixel coordinate system of the first image.

[0117] The cylindrical parameter coordinate determination module 403 is used to determine the cylindrical parameter coordinates of the target pixel in the world coordinate system based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image.

[0118] The pixel coordinate determination module 404 is used to determine the second pixel coordinates of the target pixel in the preset pixel range based on the cylindrical parameter coordinates corresponding to the target pixel and the preset pixel range.

[0119] The generation module 405 is used to generate the corrected image corresponding to the second image based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel.

[0120] In one embodiment, the cylindrical parameter coordinate determination module includes 403: a first transformation module, a second transformation module, and a first coordinate determination module. The first transformation module is used to determine a first coordinate transformation relationship and, based on the first coordinate system transformation relationship, convert the shape parameters corresponding to the target object in the first pixel coordinate system into the shape parameters corresponding to the target object in the world coordinate system; the first coordinate system transformation relationship is the transformation relationship between the first pixel coordinate system and the world coordinate system. The second transformation module is used to determine a second coordinate transformation relationship and, based on the second coordinate system transformation relationship, convert the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system into the world coordinates corresponding to the target pixel in the world coordinate system; the second coordinate transformation relationship is the transformation relationship between the second pixel coordinate system and the world coordinate system. The first coordinate determination module is used to determine the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system based on the shape parameters corresponding to the target object in the world coordinate system and the world coordinates corresponding to the target pixel in the world coordinate system.

[0121] In one embodiment, the cylindrical parameter coordinates include parameter coordinates and height coordinates; the first coordinate determination module includes a parameter coordinate determination module and a height coordinate determination module. The parameter coordinate determination module is used to determine the shape parameter corresponding to the two-dimensional plane coordinates for each target pixel point based on the two-dimensional plane coordinates of the world coordinates and the shape parameters corresponding to the target object in the world coordinate system, and then determines the shape parameter corresponding to the two-dimensional plane coordinates as the parameter coordinates; the two-dimensional plane coordinates represent the position of the world coordinates in a plane parallel to the X and Y axes of the world coordinate system; the height coordinate determination module is used to determine the Z-axis coordinate corresponding to the two-dimensional plane coordinates in the world coordinates as the height coordinates.

[0122] In one embodiment, the first transformation module further includes a first coordinate transformation relationship determination module, which is used to obtain the camera intrinsic parameters and camera extrinsic parameters corresponding to the first image; and determine the first coordinate transformation relationship based on the camera intrinsic parameters and camera extrinsic parameters corresponding to the first image.

[0123] In one embodiment, the second transformation module further includes a second coordinate transformation relationship determination module, which is used to obtain the camera intrinsic parameters and camera extrinsic parameters corresponding to the second image; and determine the second coordinate transformation relationship based on the camera intrinsic parameters and camera extrinsic parameters corresponding to the second image.

[0124] In one embodiment, the cylindrical parameter coordinates include parameter coordinates and height coordinates; the pixel coordinate determination module 404 includes a difference determination module and a second coordinate determination module. The difference determination module is used to determine the parameter difference between the maximum and minimum values ​​of the parameter coordinates and the height difference between the maximum and minimum values ​​of the height coordinates based on the cylindrical parameter coordinates corresponding to the target pixel; the second coordinate determination module is used to determine the second pixel coordinates corresponding to the target pixel based on the parameter difference, the height difference, the cylindrical parameter coordinates, and a preset pixel range.

[0125] In one embodiment, the shape fitting module 402 includes a fitting equation determination module and a shape parameter determination module. The fitting equation determination module is used to perform shape fitting on the target object in the first image and determine the shape fitting equation in the first pixel coordinate system; the shape parameter determination module is used to determine the equation parameters of the shape fitting equation in the first pixel coordinate system as the shape parameters corresponding to the target object in the first pixel coordinate system.

[0126] In one embodiment, the image correction device further includes a selection module. The selection module is used to select the object to be corrected on the target object in the second image to obtain a selected area; the selected area is used as the area to be corrected.

[0127] Each module in the aforementioned image correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0128] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores image data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image correction method.

[0129] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0130] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0131] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0132] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image correction method, characterized in that, The method includes: A first image and a second image of the target object are acquired; the side of the target object is curved, and the side profile of the target object can be fitted into a circle or an ellipse; the first image is an image taken from an angle from which the side profile of the target object can be fitted into a circle or an ellipse, and has the cross-sectional profile of the target object; the second image has the area of ​​the target object to be corrected. Shape fitting is performed on the cross-sectional contour of the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system, where the first pixel coordinate system is the pixel coordinate system of the first image; Based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system, the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system are determined; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image; the cylindrical parameter coordinates represent the position of the surface of the target object captured by the second image in the world coordinate system, including parameter coordinates and height coordinates; Based on the cylindrical parameter coordinates corresponding to the target pixel, determine the parameter difference between the maximum and minimum values ​​of the parameter coordinates and the height difference between the maximum and minimum values ​​of the height coordinates; Based on the parameter difference, the height difference, the cylindrical parameter coordinates, and the preset pixel range of the corrected image, the second pixel coordinates corresponding to the target pixel are determined; the calculation formula is: x=θr / Δθ, y=hc / Δh, where (x, y) are the second pixel coordinates, (θ, h) are the cylindrical parameter coordinates, Δθ is the parameter difference, Δh is the height difference, and r and c are the number of pixel rows and columns of the preset pixel range of the corrected image, respectively. Based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel, the corrected image corresponding to the second image is generated.

2. The method according to claim 1, characterized in that, The step of determining the cylindrical parameter coordinates of the target pixel in the world coordinate system based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system includes: A first coordinate transformation relationship is determined, and based on the first coordinate transformation relationship, the shape parameters corresponding to the target object in the first pixel coordinate system are transformed into the shape parameters corresponding to the target object in the world coordinate system; the first coordinate transformation relationship is the transformation relationship between the first pixel coordinate system and the world coordinate system; A second coordinate transformation relationship is determined, and based on the second coordinate transformation relationship, the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system are transformed into the world coordinates corresponding to the target pixel in the world coordinate system; the second coordinate transformation relationship is the transformation relationship between the second pixel coordinate system and the world coordinate system; Based on the shape parameters of the target object in the world coordinate system and the world coordinates of the target pixel in the world coordinate system, determine the cylindrical parameter coordinates of the target pixel in the world coordinate system.

3. The method according to claim 2, characterized in that, The step of determining the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system based on the shape parameters corresponding to the target object in the world coordinate system and the world coordinates corresponding to the target pixel in the world coordinate system includes: For each target pixel, the world coordinates are determined based on the two-dimensional plane coordinates of the world coordinates and the shape parameters of the target object in the world coordinate system. The shape parameters corresponding to the two-dimensional plane coordinates are then defined as the parameter coordinates. The two-dimensional plane coordinates represent the position of the world coordinates in a plane parallel to the X-axis and Y-axis of the world coordinate system. The Z-axis coordinate corresponding to the two-dimensional plane coordinate in the world coordinate system is determined as the height coordinate.

4. The method according to claim 2, characterized in that, Determining the first coordinate transformation relationship includes: Obtain the camera intrinsic and extrinsic parameters corresponding to the first image; The first coordinate transformation relationship is determined based on the camera intrinsic and extrinsic parameters corresponding to the first image.

5. The method according to claim 2, characterized in that, Determining the second coordinate transformation relationship includes: Obtain the camera intrinsic and extrinsic parameters corresponding to the second image; The second coordinate transformation relationship is determined based on the camera intrinsic and extrinsic parameters corresponding to the second image.

6. The method according to any one of claims 1 to 5, characterized in that, The step of performing shape fitting on the cross-sectional contour of the target object in the first image to determine the shape parameters of the target object in the first pixel coordinate system includes: Shape fitting is performed on the cross-sectional contour of the target object in the first image to determine the shape fitting equation in the first pixel coordinate system; The equation parameters of the shape fitting equation in the first pixel coordinate system are determined as the shape parameters corresponding to the target object in the first pixel coordinate system.

7. The method according to claim 6, characterized in that, The method further includes: In the second image, the objects to be corrected on the target object are selected by drawing a box to obtain the selected area; The selected area is taken as the area to be corrected.

8. An image correction device, characterized in that, The device includes: The acquisition module is used to acquire a first image and a second image of the target object; the side of the target object is curved, and the side profile of the target object can be fitted into a circle or an ellipse; the first image is an image taken from an angle from which the side profile of the target object can be fitted into a circle or an ellipse, and has the cross-sectional profile of the target object; the second image has the area of ​​the target object to be corrected; The shape fitting module is used to perform shape fitting on the cross-sectional contour of the target object in the first image, and determine the shape parameters of the target object in the first pixel coordinate system, wherein the first pixel coordinate system is the pixel coordinate system of the first image; The cylindrical parameter coordinate determination module is used to determine the cylindrical parameter coordinates corresponding to the target pixel in the world coordinate system based on the first pixel coordinates corresponding to the target pixel in the second pixel coordinate system and the shape parameters corresponding to the target object in the first pixel coordinate system; the second pixel coordinate system is the pixel coordinate system of the second image, and the target pixel is the pixel corresponding to the area to be corrected of the target object in the second image; the cylindrical parameter coordinates represent the position of the surface of the target object captured by the second image in the world coordinate system, including parameter coordinates and height coordinates; The pixel coordinate determination module is used to determine the parameter difference between the maximum and minimum values ​​of the cylindrical parameter coordinates and the height difference between the maximum and minimum values ​​of the height coordinates based on the cylindrical parameter coordinates corresponding to the target pixel; and to determine the second pixel coordinates corresponding to the target pixel based on the parameter difference, the height difference, the cylindrical parameter coordinates, and a preset pixel range of the corrected image; the calculation formula is: x=θr / Δθ, y=hc / Δh, where (x, y) are the second pixel coordinates, (θ, h) are the cylindrical parameter coordinates, Δθ is the parameter difference, Δh is the height difference, and r and c are the number of pixel rows and columns of the preset pixel range of the corrected image, respectively; The generation module is used to generate the corrected image corresponding to the second image based on the pixel value of the target pixel and the second pixel coordinates corresponding to the target pixel.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images