Image gray scale estimation method and system, computer equipment and storage medium

Abstract

The invention provides an image gray scale estimation method and system, computer equipment and a storage medium, and belongs to the field of image processing, and the method comprises the steps: projecting light stripes to a plane where a calibration plate is located, and calibrating a plane equation where each light stripe is located; projecting a plurality of pieces of structured light to the surface of the detected tubular object, collecting reflected light rays of the detected tubular object, and extracting light stripe centers; carrying out surface reflection coefficient relative value calibration on the plane of the measured tubular object to obtain a plurality of surface reflection coefficient values of the surface of the measured tubular object; and averaging the plurality of reflection coefficients, and calculating a theoretical gray value of the surface of the measured tubular object according to the average value of the plurality of reflection coefficients. According to the method, the gray level distribution of the surface of the object can be accurately estimated under the specific illumination condition through optical simulation, the influence of factors such as illumination and reflection on the gray level of the image is better simulated and optimized, more reliable data support is provided for defect detection of the tubular object, and the precision and robustness of structured light three-dimensional scanning are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image grayscale estimation method, system, computer equipment and storage medium. Background Art

[0002] For industrial products such as seamless steel pipes, rails, and PVC pipes, most of the key dimensions of pipe products are recorded by handheld measuring instruments during production, and surface defects are identified by naked eyes or traditional non-destructive testing. Manual testing mainly has problems such as low efficiency, large influence of subjective factors, and poor accuracy.

[0003] In response to the demand for efficient and rapid detection in industrial production, a variety of image-based defect detection methods have emerged in recent years. Structured light sensors are based on the principle of triangulation and can obtain distance information of targets illuminated by structured light within a certain range. They also have the advantages of high sampling rate and high precision, and have been widely used in the field of industrial measurement. In structured light 3D scanning technology, after imaging the modulated light stripes, the grayscale value of the bright stripes on one side is often lower than the grayscale value of the dark stripes on the other end, which makes it difficult to extract and match the stripes. In order to solve this problem more thoroughly, it is necessary to simulate and calculate the structured light illumination and the object as a whole to estimate the grayscale characteristics in the imaging.

[0004] In grayscale estimation in structured light 3D scanning, adaptive threshold processing, linear filtering, and histogram equalization are often used to improve image quality and optimize stripe extraction. Adaptive threshold processing analyzes local areas of the image and dynamically adjusts the threshold according to the grayscale distribution of each local area to adapt to areas with different brightness changes in the image. Linear filtering filters the image through a convolution kernel to change the grayscale value of the pixel. This method is often used to reduce noise and smooth images to improve image quality. Histogram equalization redistributes the pixel grayscale values ​​of the image so that the pixel values ​​within the entire grayscale level range are more evenly distributed, thereby enhancing the contrast of the image.

[0005] Although adaptive threshold processing, linear filtering and histogram equalization are widely used in image processing and grayscale estimation, they also have some limitations. Although adaptive threshold processing can flexibly cope with local illumination changes, it may still lead to misjudgment under complex illumination and noise conditions, especially when the surface reflection is uneven or there is a strong light source, it may still lead to misjudgment and make it difficult to accurately extract stripe information. Although linear filtering can effectively remove noise, it may cause blurring of stripes when processing structured light images with rich details, especially in the edge area, which may cause loss of key information and affect the accuracy of three-dimensional reconstruction. Histogram equalization enhances image contrast, but in structured light scanning, it may over-enhance the noise or high-frequency components in the image, especially in the case of uneven illumination, so that the grayscale distribution no longer matches the surface features of the actual object, thus affecting the accuracy of stripe extraction.

[0006] Therefore, traditional image processing methods are difficult to accurately deal with the complex reflection characteristics and uneven lighting of surfaces such as seamless steel pipes, resulting in grayscale estimation errors and affecting the effect of defect detection. Summary of the invention

[0007] In order to solve the problem that traditional image processing methods are difficult to accurately deal with the complex reflection characteristics and uneven illumination of surfaces such as seamless steel pipes, resulting in grayscale estimation errors and affecting defect detection effects, the present invention provides an image grayscale estimation method.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for estimating image grayscale, comprising:

[0010] Project the light stripes onto the plane plate where the calibration plate is located to form a light plane, and determine the center point of the light stripes projected on the light plane;

[0011] An image plane of an imaginary image is set, points on the imaging plane are projected onto the image plane of the imaginary image, the center point of each column of light fringes is connected with the origin of the camera coordinate system to form a ray, a group of equations is constructed through the coordinates of the intersections of multiple rays with the light plane and the image plane of the imaginary image, and the intrinsic and extrinsic parameters of the camera, and the light plane equation is obtained by fitting the group of equations according to the least squares method;

[0012] The measured tubular object is placed at the position of the plane plate where the calibration plate is located, multiple structured lights are projected onto the surface of the measured tubular object and then reflected onto the imaging plane, the image coordinates and three-dimensional coordinates of the multiple structured lights are obtained, the image coordinates, three-dimensional coordinates and camera coordinates are input into the light plane equation, and the geometric parameters of the multiple structured lights on the imaging plane are calculated;

[0013] According to the actual grayscale value of the measured tubular surface and the geometric parameters of multiple structured lights on the imaging plane, multiple reflection coefficients of the measured tubular surface are obtained, the multiple reflection coefficients are averaged, and the theoretical grayscale value of the measured tubular surface is calculated according to the average of the multiple reflection coefficients.

[0014] Preferably, an image plane of an imaginary image is set, points on the imaging plane are projected onto the image plane of the imaginary image, the center point of each column of light fringes is connected to the origin of the camera coordinate system to form a ray, a group of equations is constructed through the coordinates of the intersections of multiple rays with the light plane and the image plane of the imaginary image, and the intrinsic and extrinsic parameters of the camera, and the light plane equation is obtained by fitting the group of equations using the least squares method, specifically:

[0015] Set an imaginary image plane, the origin of the camera coordinate system is O c , project the points on the imaging plane onto the normalized plane; suppose the coordinates of P′(x1, y1, f) on the imaging plane corresponding to the normalized plane are P″(x2, y2, 1), and normalization is achieved by fixing the Z coordinate to 1; at this time, according to the straight line O C -P″ intersects with the plane where the calibration plate is located to obtain point P, straight line O C The expression of -P″ is shown in formula (1):

[0016]

[0017] Among them, (x, y, z) are the coordinates of point P in the camera coordinate system;

[0018] The coordinates of the imaging plane are normalized to the camera coordinate system as shown in formula (2), and the coordinates of point P″ are substituted into formula (3):

[0019]

[0020]

[0021] Substitute formula (3) into the calibration plate plane equation Ax+By-z+c=0, where A, B, and C are obtained through camera calibration, and the coordinates (x, y, z) of point P in the camera coordinate system are obtained as shown in formula (4):

[0022]

[0023] The above calculation is performed for the center point of each light stripe in each image to obtain the camera coordinates of the center point of the light stripe and substitute them into equation group (5):

[0024]

[0025] Among them, (x ci ,y ci , zci ) is the coordinate of the center point of the light stripe obtained by formula (4), A c , B c , C c are the coefficients of the light plane equation:

[0026] The light plane equation is obtained by fitting the equation group using the least square method.

[0027] Preferably, the camera is calibrated by a calibration board to obtain the intrinsic and extrinsic parameters of the camera, specifically:

[0028] A flat plate made of the same material as the measured tubular object is placed in the illumination area, and a calibration plate with negligible thickness is attached to its left side. Multiple images with different viewing angles are taken and the coordinates of feature points are extracted. The homography matrix of each image is determined according to the coordinates of the feature points and the geometric model of camera imaging, and the equation group of the constraint relationship is obtained by rotating the orthogonal characteristics of the homography matrix. The intrinsic and extrinsic parameters of the camera are solved through the equation group.

[0029] Preferably, the determining of the center point of the light stripe projection on the light plane is specifically:

[0030] The left and right boundaries of the light stripes are determined by the set threshold value, the vertical coordinates of the grayscale gravity center of each column of the light stripes within the left and right boundaries are calculated, and the center point of the light stripes is determined according to the vertical coordinates of the grayscale gravity center of the light stripes.

[0031] Preferably, the vertical coordinate of the grayscale center of gravity of each column of light stripes is determined by the grayscale center of gravity method, specifically:

[0032] Scan the plane image row by row, calculate the light stripes column by column, determine the left and right boundaries of the light stripes by the set threshold, calculate the ordinate of the grayscale gravity center of each row of light stripes within the left and right boundaries according to the formula, and determine the center point position of the light stripes according to the ordinate of the grayscale gravity center of the light stripes;

[0033] The specific formula for the ordinate of the grayscale center of gravity of the light stripe is as follows:

[0034]

[0035] Where j is the position of each pixel in each column; T is the set of pixels involved in the calculation; g(*) is the gray value of the pixel; y center is the vertical coordinate of the grayscale center of gravity of the light stripe.

[0036] Preferably, the plane plate is made of the same material as the tubular object to be measured.

[0037] The present invention also provides an image grayscale estimation system, comprising:

[0038] A center point acquisition module, used to project the light stripes onto the plane plate where the calibration plate is located to form a light plane, and determine the center point of the light stripes projected on the light plane;

[0039] An equation construction module is used to set an image plane of an imaginary image, project points on the imaging plane onto the image plane of the imaginary image, connect the center point of each column of light fringes with the origin of the camera coordinate system to form a ray, construct an equation group through the coordinates of the intersection points of multiple rays with the light plane and the image plane of the imaginary image, and the intrinsic and extrinsic parameters of the camera, and obtain the light plane equation by fitting the equation group according to the least squares method;

[0040] The parameter acquisition module is used to place the measured tubular object at the position of the plane plate where the calibration plate is located, project multiple structured lights onto the surface of the measured tubular object and then reflect them to the imaging plane, obtain the image coordinates and three-dimensional coordinates of the multiple structured lights, input the image coordinates, three-dimensional coordinates and camera coordinates into the light plane equation, and calculate the geometric parameters of the multiple structured lights on the imaging plane;

[0041] The grayscale estimation module is used to obtain multiple reflection coefficients of the measured tubular surface according to the actual grayscale value of the measured tubular surface and the geometric parameters of multiple structured lights on the imaging plane, take the average of the multiple reflection coefficients, and calculate the theoretical grayscale value of the measured tubular surface according to the average of the multiple reflection coefficients.

[0042] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any one of the steps in the image grayscale estimation method.

[0043] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is loaded by a processor, it can execute any one of the steps in the image grayscale estimation method.

[0044] The image grayscale estimation method provided by the present invention has the following beneficial effects:

[0045] The grayscale optimization operation can simulate the grayscale value of a flawless object surface. Comparing the grayscale value actually obtained can help infer the shape of the object surface and more accurately determine the location of defects.

[0046] The present invention sets an imaginary image plane, connects the center point of each column of light stripes with the origin of the camera coordinate system to form a ray, constructs an equation group through the coordinates of the intersection of multiple rays with the light plane and the imaginary image plane, and the internal and external parameters of the camera, and obtains the light plane equation by fitting the equation group according to the least square method. The construction of the light plane equation can convert light from a line to a continuous surface, solving the problem of uneven illumination. At the same time, the geometric parameters of multiple structured lights projected onto the measured tubular object on the imaging plane can be calculated through the light plane equation. The calculation of the parameters is not affected by image noise or high-frequency components. Finally, multiple reflection coefficients on the surface of the measured tubular object are obtained by solving the geometric parameters, and the multiple reflection coefficients are averaged. The theoretical grayscale value of the surface of the measured tubular object is calculated according to the average of the multiple reflection coefficients, solving the problem of uneven grayscale of light stripes in three-dimensional scanning of structured light, and providing more reliable data support for defect detection of industrial products such as steel pipes. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a flow chart of the image grayscale estimation method according to Embodiment 1 of the present invention;

[0049] Figure 2 A flow chart of measuring a tubular object using the image grayscale estimation method of Example 1 of the present invention;

[0050] Figure 3 This is the schematic diagram of the line structured light sensor measurement system;

[0051] Figure 4 It is a schematic diagram of the grayscale centroid method;

[0052] Figure 5 Schematic diagram for light plane calibration;

[0053] Figure 6 Schematic diagram of normalization of light plane;

[0054] Figure 7 Schematic diagram of light propagation process;

[0055] Figure 8 This is a comparison chart before and after grayscale optimization of the steel pipe surface;

[0056] Fig. 9 This is the point cloud image of the steel pipe surface. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0058] Example 1

[0059] The present invention starts with a physical model and proposes a grayscale estimation method based on radiation brightness. Optical simulation can be used to accurately estimate the grayscale distribution of the object surface under specific lighting conditions, better simulate and optimize the influence of factors such as lighting and reflection on the grayscale of the image, and provide more reliable data support for defect detection of industrial products such as steel pipes, thereby improving the accuracy and robustness of structured light three-dimensional scanning. The core idea is: project the light stripes onto the plane plate where the calibration plate is located to form a light plane, and determine the center point of the light stripe projection on the light plane; set an image plane of an imaginary image, project the points on the imaging plane onto the image plane of the imaginary image, connect the center point of each column of light stripes with the origin of the camera coordinate system to form rays, and construct a group of equations through the coordinates of the intersection of multiple rays with the light plane and the image plane of the imaginary image, and the internal and external parameters of the camera, and obtain the light plane equation by fitting the group of equations according to the least squares method; place the measured tubular object on the plane plate where the calibration plate is located Position, project multiple structured lights onto the surface of the measured tubular object and then reflect them to the imaging plane, obtain the image coordinates and three-dimensional coordinates of the multiple structured lights, input the image coordinates, three-dimensional coordinates and camera coordinates into the light plane equation, calculate the geometric parameters of the multiple structured lights on the imaging plane; solve the multiple reflection coefficients of the measured tubular surface according to the actual gray value of the measured tubular surface and the geometric parameters of the multiple structured lights on the imaging plane, take the average of the multiple reflection coefficients, and calculate the theoretical gray value of the measured tubular surface according to the average of the multiple reflection coefficients. The use of three-dimensional coordinates is specifically to construct a straight line equation in three-dimensional space through the line connecting the camera origin to the center point, and substitute the straight line equation into the light plane equation to calculate the geometric parameters.

[0060] Specifically, Figure 1 and Figure 2 As shown, the image grayscale estimation method proposed by the present invention comprises the following steps:

[0061] Step 1: calibrate the camera on the flat calibration target to obtain the internal and external parameters of the camera.

[0062] like Figure 3As shown, the multi-line structured light monocular vision measurement system is mainly composed of three parts: an optical projector (generally a laser or a projector), an image collector (a camera in this embodiment), and a computer. In this embodiment, the optical projector of the multi-line structured light monocular vision measurement system is used to project multiple structured lights to the measured tubular object. The camera calibration method adopts Zhang Zhengyou's plane template calibration method. The basic principle of this method is to use a plane calibration plate, move the calibration plate several times within the camera field of view, take multiple images from different perspectives, extract feature points and calculate the homography matrix of each picture, thereby calculating the camera's internal parameters (focal length, principal point position, etc.) and external parameters (position and posture), and use nonlinear optimization parameters to minimize projection errors. This calibration method is easy to use and has strong applicability.

[0063] When in use, the optical projector projects multiple structured lights, which form a set of light planes and are projected onto the surface of the measured tubular object. When the relative position of the optical projector and the camera is fixed, a camera imaging model can be constructed, the conversion formulas between various coordinate systems can be calculated, and the camera parameters obtained through calibration can be substituted to obtain the geometric relationship of converting two-dimensional pixels into three-dimensional points.

[0064] When calibrating the camera, a flat plate made of the same material as the measured tubular object is placed in the illuminated area, and a calibration plate of negligible thickness is attached to its left side. Multiple images from different perspectives are taken and the coordinates of feature points are extracted. The homography matrix of each image is determined according to the geometric model of camera imaging, and the set of equations of the constraint relationship is obtained through the orthogonal characteristics of the rotation matrix, thereby solving the intrinsic and extrinsic parameters of the camera.

[0065] The specific formula of the basic geometric model of camera imaging is as follows:

[0066]

[0067] In the formula, (x w ,y w ,z w ) is the world coordinate of the object, (x c ,y c , z c ) is the camera coordinate, (u,v) is the coordinate of the object in the image coordinate system, corresponding to the pixel position on the image, and f x 、f y is the focal length of the camera, (c x ,c y ) is the coordinate of the origin of the image coordinate system in the pixel coordinate system, R is the rotation matrix, T is the translation vector, z c Represents the depth value (i.e. the distance from the object to the camera).

[0068] When the Zhang Zhengyou planar template calibration method is used for camera calibration, the code logic is implemented by means of the methods in the OpenCv library. Multiple captured images are saved in a folder and read into the program using the imread() function. Then, the findCirclesGrid() function is used to extract the center point data of each image. The plane where the calibration board is located is set to z = 0, and the world coordinates of the first center point are set to (0, 0), initializing the world coordinates of each calibration point. Substituting the pixel coordinates and world coordinates of the calibration points obtained above into the calibrateCamera() function, the internal parameter matrix and distortion matrix of the camera, as well as the rotation matrix and translation vector of each image, can be obtained.

[0069] Step 2: Extract the center of the light stripe.

[0070] Furthermore, as Figure 4 shown, in Step 2, the gray centroid method is used to extract the center point of the light stripe on the plane where the calibration board is located. The basic principle of the gray centroid method is to regard the gray value of the target area as "mass distribution", and determine the exact position by calculating the centroid of the gray distribution in this area. The main idea is to scan the image row by row, calculate each column of the light stripe, determine the left and right boundaries of the light stripe through the set threshold, and calculate the ordinate of the gray centroid of each row of the light stripe according to the formula. All pixel points whose gray values exceed the set threshold are involved in the calculation process.

[0071] The specific formula of the gray centroid method is as follows:

[0072]

[0073] where h is the position of each column of pixel points (a < j < b), a and b are the ordinates of the boundaries of the light stripe; H is the set of pixel points participating in the calculation; g(*) is the gray value of the pixel point; y center is the ordinate of the obtained gray centroid point.

[0074] Specifically, after determining the left and right boundaries of the light stripe, calculate the ordinate of the gray centroid to obtain the exact center position of the light stripe. Since there are multiple light stripes to be extracted in the image, after calculating one light stripe each time, the boundary flag needs to be reset to continue processing the next light stripe. Therefore, after calculating the ordinate, the flags indicating whether the left and right boundaries have been obtained should be set to false to continue scanning the gray centroid of the next light stripe. If the number of light stripes collected in this row is complete, the center points can be classified into the corresponding light stripe sets according to the extraction order.

[0075] Step 3: Calibrate the plane equation where each light stripe is located.

[0076] Furthermore, step three uses the principle of line-plane intersection to construct a set of equations through the intersection of rays and light planes in the camera coordinate system, which can calibrate the orientation and position of the light plane, and obtain the light plane equation by fitting according to the least squares method. For the light plane calibration of a single light streak, the point set in a single image is collinear, and a light plane cannot be directly fitted. Therefore, it is necessary to obtain two or more images at different viewing angles to calibrate the corresponding light plane.

[0077] Specifically, Figure 5 As shown in the figure, the principle of line-plane intersection is used for light plane calibration. The light stripes are projected onto the plane where the calibration plate is located. The origin O of the image coordinate system and the origin O of the camera coordinate system are c The distance is f, P′ is the center point of the light stripe on the imaging plane, and the straight line O c The intersection point P of -P′ and the plane is the corresponding point of the center point P′ of the light stripe on the imaging plane in the actual three-dimensional space. The light plane equation can be obtained by fitting according to the least squares method.

[0078] Through step 1, we get the camera’s intrinsic parameter matrix, as shown in matrix A. Since camera calibration can only get f x 、f y The values ​​correspond to the scaling factors in the x and y directions, respectively, and cannot directly reflect the actual physical distance. In order to obtain the actual focal length, it is necessary to know the physical size of the camera sensor, which will result in additional calculations and searches, and the operation is more complicated.

[0079]

[0080] The present invention considers normalizing the imaging plane, such as Figure 6 As shown, an imaginary image plane is set, which is the normalized plane. Project the points on the imaging plane onto this imaginary normalized plane. Suppose the coordinates of P′(x1, y1, f) on the imaging plane corresponding to the normalized plane are P″(x2, y2, 1), and normalization is achieved by fixing the Z coordinate to 1. At this time, according to the straight line O C -P″ intersects with the plane where the calibration plate is located to obtain point P and straight line O C The expression of -P″ is shown in formula (1).

[0081]

[0082] Among them, (x, y, z) are the coordinates of point P in the camera coordinate system.

[0083] The coordinates of the imaging plane are normalized to the camera coordinate system as shown in formula (2), and the coordinates of point P″ are substituted into the form shown in formula (3).

[0084]

[0085]

[0086] Substitute formula (3) into the calibration plate plane equation Ax+By-z+c=0, where A, B, and C are obtained through camera calibration, and the coordinates (x, y, z) of point P in the camera coordinate system are obtained as shown in formula (4), f x 、f y 、c x 、c y The value of is obtained from step 1.

[0087]

[0088] The above calculation is performed for the center point of each light stripe in each image to obtain the camera coordinates of the center point of the light stripe and substitute them into formula (5).

[0089]

[0090] Among them, (x ci ,y ci , z ci ) is the coordinate of the center point of the light stripe obtained by formula (4), A c , B c , C c are the coefficients of the light plane equation.

[0091] The light plane equation is obtained by fitting the equation group using the least square method, that is, by minimizing the sum of squares of errors to find the best function match for a set of data, the light plane equation can be obtained.

[0092] Step 4: According to the actual grayscale value of the surface of the measured tubular object and the geometric parameters of the multiple structured lights on the imaging plane, multiple reflection coefficients of the surface of the measured tubular object are obtained, the multiple reflection coefficients are averaged, and the theoretical grayscale value of the surface of the measured tubular object is calculated according to the average of the multiple reflection coefficients.

[0093] Specifically, according to the physical process, the propagation of light energy is decomposed into two stages. The first stage is the reflection stage after the light emitted from the (point) light source hits the surface of the object, and the second stage is the post-imaging stage when the light reflected from the surface of the object enters the camera. Light follows the two principles of energy conservation and rectilinear propagation throughout the propagation process. Figure 7 As shown in the figure, C represents the observation camera, L represents the light source, P represents the imaging plane, S represents the light receiving area on the surface of the observed object, A represents a certain pixel point, and A′ represents the tiny area on the surface of the object corresponding to A. The light is emitted from L and enters the camera C after being reflected by S.

[0094] In the first stage, let the radiation energy of the light source be Q and the power be is certain, the electro-optical conversion rate of the light source is α, and its luminous flux is After the light source is modulated by the lens, it radiates outward uniformly in a hemispherical wave shape. The spherical degree is 2π, so the light intensity emitted by the light source is When the area of ​​micro-city A' is S A' , the distance from the light source is R A' , the angle between the light source and the light source is θ, and the luminous flux at A' is The corresponding light intensity Assume that the reflectivity of the surface at A' is β, then the reflected light intensity is The reflected light intensity is

[0095] In the second stage, the light from A' will reach the pixel corresponding to the image point A and be converted into an electrical signal, which is recorded in the form of grayscale value. Let the distance between A' and the center point of the camera lens be R C , image distance is f, pixel area is S A , and the angle between the faces opposite the solid angle is When It can be obtained that the luminous flux of image point A is Corresponding illumination

[0096] The specific calculation of the theoretical gray value is as follows:

[0097] If γ represents the photoelectric conversion coefficient of the pixel (represented by the sensitivity curve of CCD / CMOS), G A for:

[0098]

[0099] When the angle between A' and the corresponding pixel surface of A is ψ, and R C >f, the relationship is satisfied:

[0100]

[0101] Substituting formula (7) into formula (6) yields:

[0102]

[0103] In the above formula in, is the radiation power of the light source, α is the electro-optical conversion rate of the light source, when the light source is determined, the two are constants; γ, f and S A is the camera parameter; β is the reflectivity of the object surface, which is a constant when the surface material of the object is determined; R C , R A' ,θ, and ψ are geometric variables, representing parameters such as the relative position and angle between the camera and the object surface and the light source, respectively.

[0104] When the system composed of the light source and the camera is determined, k is only related to the reflectivity of the surface of the measured tubular object. After observing the gray value of pixel A and calculating the above parameter values, substituting them into formula (8), the relative value of k of this system can be calculated. By calibrating the steel pipe, its axis equation, posture, radius and other information can be obtained, based on which the approximate normal vector at the tiny surface element A' can be determined. By performing similar calculations on all pixels corresponding to all light stripe areas on the image, the relative value of the reflection coefficient k of the steel pipe surface can be estimated.

[0105] Step 5: Apply the reflection estimation coefficient to the whole image, optimize the image grayscale, and obtain the point cloud image of the object being measured.

[0106] Furthermore, in step 5, the surface reflection coefficient of the object is estimated by step 4, and the surface reflection coefficient value of the object surface is obtained, and then the theoretical gray value of the measured tubular surface is obtained. The surface reflection coefficient is applied to the image pixel points, the gray value of the light streak is corrected, and the light streak extraction accuracy is improved. The comparison of the gray value of the steel pipe surface before and after optimization is shown in Figure 1. Figure 8 After image optimization, combined with the plane equation calibration and three-dimensional coordinate calculation in the previous steps, a three-dimensional scanning model is established, and the three-dimensional point cloud data of the steel pipe surface can be obtained from the gray value of each pixel and the optimized image. In the actual measurement process, the defective steel pipe is scanned by a fixed projector and camera, and the generated point cloud results are as follows Fig. 9 shown.

[0107] Based on the same inventive concept, the present invention also provides an image grayscale estimation system, comprising:

[0108] A center point acquisition module, used to project the light stripes onto the plane plate where the calibration plate is located to form a light plane, and determine the center point of the light stripes projected on the light plane;

[0109] An equation construction module is used to set an image plane of an imaginary image, project points on the imaging plane onto the image plane of the imaginary image, connect the center point of each column of light fringes with the origin of the camera coordinate system to form a ray, construct an equation group through the coordinates of the intersection points of multiple rays with the light plane and the image plane of the imaginary image, and the intrinsic and extrinsic parameters of the camera, and obtain the light plane equation by fitting the equation group according to the least squares method;

[0110] The parameter acquisition module is used to place the measured tubular object at the position of the plane plate where the calibration plate is located, project multiple structured lights onto the surface of the measured tubular object and then reflect them to the imaging plane, obtain the image coordinates and three-dimensional coordinates of the multiple structured lights, input the image coordinates, three-dimensional coordinates and camera coordinates into the light plane equation, and calculate the geometric parameters of the multiple structured lights on the imaging plane;

[0111] The grayscale estimation module is used to obtain multiple reflection coefficients of the measured tubular surface according to the actual grayscale value of the measured tubular surface and the geometric parameters of multiple structured lights on the imaging plane, take the average of the multiple reflection coefficients, and calculate the theoretical grayscale value of the measured tubular surface according to the average of the multiple reflection coefficients.

[0112] Each module in the above image grayscale estimation system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0113] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the image grayscale estimation method embodiment. The specific implementation method can be found in the method embodiment, which will not be described in detail here.

[0114] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the image grayscale estimation method can be implemented. The specific implementation method can be referred to the method embodiment, which will not be repeated here.

[0115] It will be appreciated by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.

Claims

1. A method for estimating image grayscale, characterized in that: include: Project the light stripes onto the plane plate where the calibration plate is located to form a light plane, and determine the center point of the light stripes projected on the light plane; An image plane of an imaginary image is set, points on the imaging plane are projected onto the image plane of the imaginary image, the center point of each column of light fringes is connected with the origin of the camera coordinate system to form a ray, a group of equations is constructed through the coordinates of the intersections of multiple rays with the light plane and the image plane of the imaginary image, and the intrinsic and extrinsic parameters of the camera, and the light plane equation is obtained by fitting the group of equations according to the least squares method; The measured tubular object is placed at the position of the plane plate where the calibration plate is located, multiple structured lights are projected onto the surface of the measured tubular object and then reflected onto the imaging plane, the image coordinates and three-dimensional coordinates of the multiple structured lights are obtained, the image coordinates, three-dimensional coordinates and camera coordinates are input into the light plane equation, and the geometric parameters of the multiple structured lights on the imaging plane are calculated; According to the actual grayscale value of the measured tubular surface and the geometric parameters of multiple structured lights on the imaging plane, multiple reflection coefficients of the measured tubular surface are obtained, the multiple reflection coefficients are averaged, and the theoretical grayscale value of the measured tubular surface is calculated according to the average of the multiple reflection coefficients.

2. The image grayscale estimation method according to claim 1, characterized in that: The method sets an image plane of an imaginary image, projects points on the imaging plane onto the image plane of the imaginary image, connects the center point of each column of light fringes with the origin of the camera coordinate system to form a ray, constructs an equation group through the coordinates of the intersection of multiple rays with the light plane and the image plane of the imaginary image, and the internal and external parameters of the camera, and obtains the light plane equation by fitting the equation group using the least squares method, specifically: Set an imaginary image plane, the origin of the camera coordinate system is O c , project the points on the imaging plane onto the normalized plane; let The coordinates corresponding to the normalized plane are By fixing the Z coordinate to 1, normalization is achieved; at this time, according to the straight line O C -P″ intersects with the plane where the calibration plate is located to obtain point P, straight line O C The expression of -P″ is shown in formula (1): Among them, (x, y, z) are the coordinates of point P in the camera coordinate system; The coordinates of the imaging plane are normalized to the camera coordinate system as shown in formula (2), and the coordinates of point P″ are substituted into formula (3): Substitute formula (3) into the calibration plate plane equation Ax+By-z+c=0, where A, B, and C are obtained through camera calibration, and the coordinates (x, y, z) of point P in the camera coordinate system are obtained as shown in formula (4): The above calculation is performed for the center point of each light stripe in each image to obtain the camera coordinates of the center point of the light stripe and substitute them into equation group (5): Among them, (x ci ,y ci , z ci ) is the coordinate of the center point of the light stripe obtained by formula (4), A c , B c , C c are the coefficients of the light plane equation: The light plane equation is obtained by fitting the equation group using the least square method.

3. The image grayscale estimation method according to claim 1, characterized in that: The camera is calibrated through the calibration board to obtain the camera's internal and external parameters, specifically: A flat plate made of the same material as the measured tubular object is placed in the illumination area, and a calibration plate with negligible thickness is attached to its left side. Multiple images with different viewing angles are taken and the coordinates of feature points are extracted. The homography matrix of each image is determined according to the coordinates of the feature points and the geometric model of camera imaging, and the equation group of the constraint relationship is obtained by rotating the orthogonal characteristics of the homography matrix. The intrinsic and extrinsic parameters of the camera are solved through the equation group.

4. The image grayscale estimation method according to claim 1, characterized in that: The step of determining the center point of the light stripe projection on the light plane is specifically: The left and right boundaries of the light stripes are determined by the set threshold value, the vertical coordinates of the grayscale gravity center of each column of the light stripes within the left and right boundaries are calculated, and the center point of the light stripes is determined according to the vertical coordinates of the grayscale gravity center of the light stripes.

5. The image grayscale estimation method according to claim 4, characterized in that: The vertical coordinate of the grayscale center of gravity of each column of light stripes is determined by the grayscale center of gravity method, specifically: Scan the plane image row by row, calculate the light stripes column by column, determine the left and right boundaries of the light stripes by the set threshold, calculate the ordinate of the grayscale gravity center of each row of light stripes within the left and right boundaries according to the formula, and determine the center point position of the light stripes according to the ordinate of the grayscale gravity center of the light stripes; The specific formula for the ordinate of the grayscale center of gravity of the light stripe is as follows: Where j is the position of each pixel in each column; T is the set of pixels involved in the calculation; g(*) is the gray value of the pixel; y center is the vertical coordinate of the grayscale center of gravity of the light stripe.

6. The image grayscale estimation method according to claim 1, characterized in that: The plane plate is made of the same material as the measured tubular object.

7. An image grayscale estimation system, characterized in that: include: A center point acquisition module, used to project the light stripes onto the plane plate where the calibration plate is located to form a light plane, and determine the center point of the light stripes projected on the light plane; An equation construction module is used to set an image plane of an imaginary image, project points on the imaging plane onto the image plane of the imaginary image, connect the center point of each column of light fringes with the origin of the camera coordinate system to form a ray, construct an equation group through the coordinates of the intersection points of multiple rays with the light plane and the image plane of the imaginary image, and the intrinsic and extrinsic parameters of the camera, and obtain the light plane equation by fitting the equation group according to the least squares method; The parameter acquisition module is used to place the measured tubular object at the position of the plane plate where the calibration plate is located, project multiple structured lights onto the surface of the measured tubular object and then reflect them to the imaging plane, obtain the image coordinates and three-dimensional coordinates of the multiple structured lights, input the image coordinates, three-dimensional coordinates and camera coordinates into the light plane equation, and calculate the geometric parameters of the multiple structured lights on the imaging plane; The grayscale estimation module is used to obtain multiple reflection coefficients of the measured tubular surface according to the actual grayscale value of the measured tubular surface and the geometric parameters of multiple structured lights on the imaging plane, take the average of the multiple reflection coefficients, and calculate the theoretical grayscale value of the measured tubular surface according to the average of the multiple reflection coefficients.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded into a processor, it can execute the steps of the method according to any one of claims 1 to 6.

Images