A screen image-based camera calibration method, device and storage medium

Abstract

The application discloses a camera calibration method and device based on a screen image and a storage medium, and is used for improving the accuracy of an image collected by an industrial camera. A calibration image is designed for an industrial camera in a collection system according to screen parameters of a reference display screen; the reference display screen is lighted according to environmental parameters of the collection system and the calibration image; the industrial camera is used to collect calibration photos of the reference display screen in different postures; feature points and posture correction mark points in the calibration photos are extracted; a world coordinate system is constructed according to the reference display screen, and the world coordinate system comprises a plurality of world coordinates; a posture correction matrix is generated by using coordinate data of the posture correction mark points and reference coordinate data in a horizontal alignment posture; posture correction processing is performed on coordinate data of the feature points by using the posture correction matrix, and posture correction coordinate data is generated; and the industrial camera is calibrated by using world coordinates in the world coordinate system and the posture correction coordinate data.

Description

Technical Field

[0001] This application relates to the field of display screen testing, and more particularly to a camera calibration method, apparatus, and storage medium based on screen images. Background Technology

[0002] With technological innovation, new display technologies (such as MicroLED and flexible foldable screens) are rapidly reshaping the industry landscape. As a core link in the display screen industry chain, the quality inspection technology of display screens continues to receive attention. Various tests on the surface of display screens have become key factors determining the quality of display screen products. Among the many testing items for display screens, using industrial cameras to capture images of the display screen is an indispensable step. However, industrial cameras often require camera calibration before image acquisition.

[0003] Traditional camera calibration methods require specialized calibration boards, which have several insurmountable drawbacks. Firstly, the physical properties of these calibration boards are fixed, especially their size and patterns, which cannot be flexibly adjusted according to different display screen parameters, testing items, and actual testing scenarios. For example, the recognition effect of these calibration boards is heavily dependent on ambient lighting conditions. In environments with insufficient light intensity, the ambient reflected light on the calibration board is insufficient, making it difficult for the camera to clearly capture its features, thus affecting calibration accuracy. In strong light environments, these calibration boards are prone to glare, which also interferes with the industrial camera's recognition of feature points on the calibration board. Furthermore, with the continuous emergence of new types of displays, the number of testing items for these displays is constantly increasing, and the precision of these testing items is also constantly improving. The calibration quality of the industrial camera affects the quality of the images acquired by the new displays, and thus affects the testing quality of the new displays. Existing camera calibration methods do not provide targeted calibration for the different orientations of new displays. This means that during the camera calibration process, it is necessary to frequently change calibration boards of different specifications and constantly adjust the system light source to adapt to the testing items of the new display screen. However, the effect is very poor, which not only reduces the efficiency of industrial camera calibration in display screen testing, but also reduces the accuracy of the images acquired by the industrial camera. Summary of the Invention

[0004] This application discloses a camera calibration method, apparatus, and storage medium based on screen images, which is used to improve the accuracy of images acquired by industrial cameras.

[0005] In a first aspect, embodiments of this application provide a camera calibration method based on screen images, comprising:

[0006] Based on the screen parameters of the reference display screen, a calibration image is designed for the industrial camera in the acquisition system. The acquisition system includes an industrial camera, and the calibration image is set with several feature points and attitude correction mark points. The reference display screen is illuminated based on the environmental parameters of the acquisition system and the calibration image. Calibration photos of the reference display screen under different attitudes are acquired using the industrial camera. Feature points and attitude correction mark points are extracted from the calibration photos. A world coordinate system is constructed based on the reference display screen, which includes several world coordinates. An attitude correction matrix is ​​generated using the coordinate data of the attitude correction mark points and the reference coordinate data under a horizontally aligned attitude. The attitude correction matrix is ​​used to perform attitude correction processing on the coordinate data of the feature points to generate attitude correction coordinate data. The industrial camera is calibrated using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0007] Optionally, the steps for extracting feature points and pose correction mark points from the calibration photos include:

[0008] The coordinates and gray values ​​of pixels in each calibration photo are fitted with a quadratic polynomial. After fitting, the Hessian matrix of the quadratic polynomial is constructed. Feature points on the calibration photo are extracted based on the Hessian matrix. The pose correction mark points on the calibration photo are extracted through binarization and morphological processing.

[0009] Optionally, the step of using an attitude correction matrix to perform attitude correction processing on the coordinate data of feature points to generate attitude-corrected coordinate data includes:

[0010] The coordinate data of feature points are processed by attitude correction matrix to obtain attitude-corrected coordinate data. The attitude-corrected coordinate data is then sorted in a row-first, column-second manner to generate ordered coordinates that correspond one-to-one with each feature point in the world coordinate system. The ordered coordinates are then multiplied by the inverse of the attitude correction matrix to generate attitude-corrected coordinate data.

[0011] Optionally, the step of calibrating the industrial camera using world coordinates and attitude correction coordinate data in the world coordinate system includes:

[0012] The intrinsic parameter matrix of the industrial camera is constructed based on the equivalent focal length and principal point coordinates in different axial directions, and the distortion coefficient expression of the industrial camera is also constructed. The extrinsic parameter matrix of the industrial camera is constructed based on the rotation matrix and translation vector. The perspective projection equation of the industrial camera is constructed based on the world coordinates in the world coordinate system, attitude correction coordinate data, intrinsic parameter matrix, extrinsic parameter matrix, and the scale factor of the industrial camera. A homography matrix is ​​generated based on the intrinsic and extrinsic parameter matrices, and the homography matrix is ​​solved using the attitude correction coordinate data. Equations for each intrinsic parameter in the intrinsic parameter matrix are listed based on the solved homography matrix, and the intrinsic parameter matrix is ​​solved using the perspective projection equation and the least squares method. The extrinsic parameter matrix is ​​solved based on the solved homography matrix and the solved intrinsic parameter matrix. The distortion coefficient expression is solved based on the world coordinates in the world coordinate system and the attitude correction coordinate data, and the distortion correction coefficients are generated.

[0013] Optionally, the reference display is a quantum dot electroluminescent display;

[0014] The steps for illuminating the reference display screen based on the environmental parameters and calibration image of the acquisition system include:

[0015] The acquisition system is activated with its light source according to the subsequent testing items for the reference display screen. Under various orientations of the reference display screen, acquisition points are determined based on the film thickness distribution, and the ambient light intensity is detected at these points. When the ambient light intensity is less than the low-light environment threshold, the base brightness, dynamic contrast coefficient, and adaptive Gamma value of the reference display screen are generated based on the ambient light intensity. The base brightness is adjusted based on the dynamic contrast coefficient and adaptive Gamma value to generate an adjusted brightness. The reference display screen is then illuminated with a calibration image, and the luminous intensity of the reference display screen is adjusted according to the adjusted brightness.

[0016] Optionally, the reference display is a quantum dot electroluminescent display;

[0017] After acquiring calibration photos of the reference display screen in different poses using an industrial camera, and before extracting feature points and pose correction mark points from the calibration photos, the camera calibration method also includes:

[0018] Obtain the nominal light conversion film thickness of the reference display screen; obtain the actual light conversion film thickness distribution and the actual refractive index distribution corresponding to the actual quantum dot concentration of the reference display screen; calculate the thickness adjustment factor of the reference display screen in each region based on the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflectance coefficient; calculate the interface transmission adjustment factor based on the actual refractive index distribution; adjust the grayscale value of the effective area of ​​the display screen in the calibration photograph based on the thickness adjustment factor and the interface transmission adjustment factor.

[0019] Optionally, after acquiring calibration photos of the reference display screen in different postures using an industrial camera and before extracting feature point coordinate data and posture correction feature coordinate data from the calibration photos, the camera calibration method may further include:

[0020] The calibration photos are filtered.

[0021] Secondly, embodiments of this application provide a camera calibration device based on screen images, comprising:

[0022] The design unit is used to design calibration images for the industrial camera in the acquisition system based on the screen parameters of the reference display screen. The acquisition system includes an industrial camera, and the calibration images are set with several feature points and attitude correction mark points. The illumination unit is used to illuminate the reference display screen based on the environmental parameters of the acquisition system and the calibration images. The acquisition unit is used to acquire calibration photos of the reference display screen in different attitudes using the industrial camera. The extraction unit is used to extract feature points and attitude correction mark points from the calibration photos. The construction unit is used to construct a world coordinate system based on the reference display screen. The world coordinate system includes several world coordinates. The first generation unit is used to generate an attitude correction matrix using the coordinate data of the attitude correction mark points and the reference coordinate data in a horizontally aligned attitude. The second generation unit is used to perform attitude correction processing on the coordinate data of the feature points using the attitude correction matrix to generate attitude correction coordinate data. The calibration unit is used to calibrate the industrial camera using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0023] Optionally, the extraction unit specifically includes:

[0024] The coordinates and gray values ​​of pixels in each calibration photo are fitted with a quadratic polynomial. After fitting, the Hessian matrix of the quadratic polynomial is constructed. Feature points on the calibration photo are extracted based on the Hessian matrix. The pose correction mark points on the calibration photo are extracted through binarization and morphological processing.

[0025] Optionally, the second generation unit specifically includes:

[0026] The coordinate data of feature points are processed by attitude correction matrix to obtain attitude-corrected coordinate data. The attitude-corrected coordinate data is then sorted in a row-first, column-second manner to generate ordered coordinates that correspond one-to-one with each feature point in the world coordinate system. The ordered coordinates are then multiplied by the inverse of the attitude correction matrix to generate attitude-corrected coordinate data.

[0027] Optionally, the calibration unit specifically includes:

[0028] The intrinsic parameter matrix of the industrial camera is constructed based on the equivalent focal length and principal point coordinates in different axial directions, and the distortion coefficient expression of the industrial camera is also constructed. The extrinsic parameter matrix of the industrial camera is constructed based on the rotation matrix and translation vector. The perspective projection equation of the industrial camera is constructed based on the world coordinates in the world coordinate system, attitude correction coordinate data, intrinsic parameter matrix, extrinsic parameter matrix, and the scale factor of the industrial camera. A homography matrix is ​​generated based on the intrinsic and extrinsic parameter matrices, and the homography matrix is ​​solved using the attitude correction coordinate data. Equations for each intrinsic parameter in the intrinsic parameter matrix are listed based on the solved homography matrix, and the intrinsic parameter matrix is ​​solved using the perspective projection equation and the least squares method. The extrinsic parameter matrix is ​​solved based on the solved homography matrix and the solved intrinsic parameter matrix. The distortion coefficient expression is solved based on the world coordinates in the world coordinate system and the attitude correction coordinate data, and the distortion correction coefficients are generated.

[0029] Optionally, the reference display is a quantum dot electroluminescent display;

[0030] The lighting unit specifically includes:

[0031] The acquisition system is activated with its light source according to the subsequent testing items for the reference display screen. Under various orientations of the reference display screen, acquisition points are determined based on the film thickness distribution, and the ambient light intensity is detected at these points. When the ambient light intensity is less than the low-light environment threshold, the base brightness, dynamic contrast coefficient, and adaptive Gamma value of the reference display screen are generated based on the ambient light intensity. The base brightness is adjusted based on the dynamic contrast coefficient and adaptive Gamma value to generate an adjusted brightness. The reference display screen is then illuminated with a calibration image, and the luminous intensity of the reference display screen is adjusted according to the adjusted brightness.

[0032] Optionally, the reference display is a quantum dot electroluminescent display;

[0033] The camera calibration device, following the acquisition unit and preceding the extraction unit, also includes:

[0034] The first acquisition unit is used to acquire the nominal light conversion film thickness of the reference display screen; the second acquisition unit is used to acquire the actual light conversion film thickness distribution and the actual refractive index distribution corresponding to the actual quantum dot concentration of the reference display screen; the first calculation unit is used to calculate the thickness adjustment factor of the reference display screen in each region based on the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflectance coefficient; the second calculation unit is used to calculate the interface transmission adjustment factor based on the actual refractive index distribution; and the adjustment unit is used to adjust the grayscale value of the effective area of ​​the display screen in the calibration photograph based on the thickness adjustment factor and the interface transmission adjustment factor.

[0035] Optionally, after the acquisition unit and before the extraction unit, the camera calibration device further includes:

[0036] The filtering unit is used to filter the calibration photos.

[0037] Thirdly, embodiments of this application provide a camera calibration device based on screen images, comprising:

[0038] Processor, memory, input / output units, and bus;

[0039] The processor is connected to memory, input / output units, and a bus;

[0040] The memory stores a program, which the processor calls to execute, such as the first aspect and any optional camera calibration method of the first aspect.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that, when executed on a computer, performs the first aspect and any optional camera calibration method of the first aspect.

[0042] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0043] In this application, firstly, a calibration image is designed for the industrial camera in the acquisition system based on the screen parameters of a reference display screen. The acquisition system includes an industrial camera, and the calibration image contains several feature points and attitude correction mark points. The reference display screen is then illuminated based on the environmental parameters of the acquisition system and the calibration image. The industrial camera is used to acquire calibration photos of the reference display screen in different attitudes. Feature points and attitude correction mark points are extracted from the calibration photos. A world coordinate system is constructed based on the reference display screen, including several world coordinates. An attitude correction matrix is ​​generated using the coordinate data of the attitude correction mark points and the reference coordinate data in a horizontally aligned attitude. The attitude correction matrix is ​​used to perform attitude correction processing on the coordinate data of the feature points, generating attitude correction coordinate data. The industrial camera is calibrated using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0044] A reference display screen of the same type as the display screen under test is set up as a calibration board, and a dedicated calibration image is designed based on the parameters of the reference display screen and the test items. The designed calibration image is displayed on the reference display screen according to the test items or the screen characteristics of the reference display screen, and the brightness of the display screen can be adjusted according to environmental parameters (ambient lighting). Feature points and attitude correction mark points are extracted from the calibration photos captured by the industrial camera, and the attitude correction matrix generated by the attitude correction mark points is used to correct the coordinates of the feature points. The generated attitude correction coordinate data can better calibrate the industrial camera, resulting in higher accuracy images captured by the industrial camera after calibration, which are more suitable for the characteristics of the new display screen and the test items. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the first embodiment of the camera calibration method based on screen images of this application;

[0047] Figure 2 This is a schematic diagram of the first embodiment of the method for extracting feature points and posture correction mark points according to this application;

[0048] Figure 3 A schematic diagram of a first embodiment of the method for generating attitude correction coordinate data according to this application;

[0049] Figure 4 This is a schematic diagram of the first embodiment of the method for calibrating an industrial camera according to this application;

[0050] Figure 5 This is a schematic diagram of a first embodiment of the method for illuminating a reference display screen according to this application;

[0051] Figure 6 This is a schematic diagram of the first embodiment of the method for adjusting the grayscale value of the effective area of ​​the display screen in the calibration photograph according to this application;

[0052] Figure 7 This is a schematic diagram of the first embodiment of the method for preprocessing calibration photographs according to this application;

[0053] Figure 8 This is a schematic diagram of the first embodiment of the camera calibration device based on screen images of this application;

[0054] Figure 9 This is a schematic diagram of a second embodiment of the camera calibration device based on screen images according to this application;

[0055] Figure 10 This is a schematic diagram of the first embodiment of the chessboard calibration plate of this application;

[0056] Figure 11 A schematic diagram of the first embodiment of the inclined checkerboard calibration plate of this application;

[0057] Figure 12 This is a schematic diagram of the first embodiment of the cellular calibration board of this application;

[0058] Figure 13This is a schematic diagram of the first embodiment of the checkerboard calibration board after feature point extraction according to this application. Detailed Implementation

[0059] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0060] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0061] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0062] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0063] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0065] Existing technologies require specialized calibration boards, which present numerous insurmountable drawbacks. Firstly, the physical properties of these calibration boards are fixed, particularly their size and patterns, making them inflexible for adjustments based on different display screen parameters, testing items, and actual testing scenarios. For instance, the recognition performance of these calibration boards heavily relies on ambient lighting conditions. In environments with insufficient light, the reflected light from the environment on which the calibration board relies is insufficient, making it difficult for the camera to clearly capture its features, thus affecting calibration accuracy. In strong light environments, these calibration boards are prone to glare, which also interferes with the industrial camera's recognition of feature points on the calibration board. Furthermore, with the continuous emergence of new types of displays, the number of testing items for these displays is constantly increasing, and the precision of these testing items is also constantly improving. The calibration quality of the industrial camera affects the quality of the images acquired by the new displays, thereby affecting the overall testing quality of the new displays. This necessitates not only changing calibration boards of different specifications during camera calibration but also constantly adjusting the system light source to adapt to the testing items and precision requirements of the new displays, reducing the efficiency of industrial camera calibration in display screen testing.

[0066] Based on this, this application discloses a camera calibration method, apparatus and storage medium based on screen images, which can improve the accuracy of images acquired by industrial cameras.

[0067] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0068] The method described in this application can be applied to servers, devices, terminals, or other devices with logical processing capabilities; therefore, this application does not limit its application. For ease of description, the following description uses a terminal as the executing entity.

[0069] Please see Figure 1 This application provides an embodiment of a camera calibration method based on screen images, comprising:

[0070] 101. Design a calibration image for the industrial camera in the acquisition system based on the screen parameters of the reference display screen. The acquisition system includes an industrial camera, and the calibration image is set with several feature points and attitude correction mark points.

[0071] In this embodiment, the terminal designs calibration images for the industrial camera in the acquisition system based on the screen parameters of the reference display screen. The purpose is to create calibration images according to product characteristics. Specifically, a calibration image of appropriate size is created based on the product resolution. Please refer to [reference needed]. Figure 10 and Figure 12 , Figure 10 The calibration image corresponding to the checkerboard calibration board (reference display calibration board). Figure 12 The calibration image is the one corresponding to the circular cellular calibration board. Besides the two types mentioned above, it can also be other types of calibration images. In this application, the calibration image needs to have feature points and attitude correction mark points for attitude correction.

[0072] 102. Light up the reference display screen according to the environmental parameters of the acquisition system and the calibration image.

[0073] In this embodiment, the terminal illuminates the reference display screen based on the environmental parameters and calibration image of the acquisition system. The terminal inputs the calibration image into the reference display screen and adjusts the brightness of the reference display screen according to the real-time monitored environmental parameters. The specific illumination process will be described in subsequent embodiments.

[0074] 103. Use an industrial camera to capture calibration photos of the reference display screen in different orientations.

[0075] The terminal uses an industrial camera to capture calibration photos of the reference display screen in different orientations. Specifically, when the terminal uses the industrial camera to take pictures, the orientation of the reference display screen relative to the industrial camera needs to be continuously changed during the shooting process, including rotation, translation, and tilting. The industrial camera simultaneously captures images, obtaining at least 20 calibration photos (calibration photos) showing the display area of ​​the reference display screen in different orientations. Please refer to [reference needed]. Figure 11 , Figure 11 This is a schematic diagram of a checkerboard calibration board (referencing the display calibration board) in an inclined orientation.

[0076] 104. Extract feature points and posture correction mark points from the calibration photos.

[0077] In this embodiment, the terminal extracts feature points and attitude correction mark points from the calibration image. In the checkerboard calibration board, feature points are the vertices of black and white squares, while attitude correction mark points are located inside the black and white squares. Five special mark points (five circular attitude correction mark points, three black circles and two white circles) on the checkerboard calibration board are used for attitude correction. The specific extraction method will be described in detail in subsequent embodiments.

[0078] 105. Construct a world coordinate system based on the reference display screen. The world coordinate system includes several world coordinates.

[0079] The terminal constructs a world coordinate system based on the reference display screen. This world coordinate system includes several world coordinates. Specifically, the terminal establishes the coordinate system with the first pixel at the top left corner of the reference display screen as the origin. Given that the screen resolution is W*H, the physical size of a single pixel is m, the row spacing of feature points is R, and the column spacing is C, the world coordinate Q is calculated.

[0080]

[0081] 106. Generate the attitude correction matrix using the coordinate data of the attitude correction mark points and the reference coordinate data under the horizontally aligned attitude.

[0082] The terminal uses the coordinate data of the attitude correction mark points and the reference coordinate data under the horizontally aligned attitude to generate an attitude correction matrix. The coordinates of the attitude correction matrix are denoted as P1(row1, col1), and the ideal coordinates (i.e., the coordinates under the horizontally aligned attitude) are denoted as Q1(row2, col2). The attitude correction matrix Mat1 is then calculated.

[0083]

[0084] P1X and P1Y correspond to row1 and col1, respectively, and Q1X and Q1Y correspond to row2 and col2, respectively. The attitude correction matrix Mat1 can be calculated using the above formula, and the attitude of the feature points can be corrected.

[0085] 107. Use the attitude correction matrix to perform attitude correction processing on the coordinate data of feature points to generate attitude correction coordinate data.

[0086] In this embodiment, the terminal uses an attitude correction matrix to perform attitude correction processing on the coordinate data of feature points, so that the coordinate data of feature points under different attitudes can be used for targeted calibration of the industrial camera after the correction is completed. The specific attitude correction processing method will be described in detail in subsequent embodiments.

[0087] 108. Use world coordinates and attitude correction coordinates in the world coordinate system to calibrate the industrial camera.

[0088] Once the attitude correction is complete, the terminal can use the world coordinate system constructed based on the reference display screen and the attitude correction coordinate data to calibrate the industrial camera, enabling the industrial camera to adapt to the size and characteristics of the display screen under test.

[0089] In this embodiment, firstly, a calibration image is designed for the industrial camera in the acquisition system based on the screen parameters of the reference display screen. The acquisition system includes an industrial camera, and the calibration image is set with several feature points and attitude correction mark points. The reference display screen is illuminated according to the environmental parameters of the acquisition system and the calibration image. Calibration photos of the reference display screen under different attitudes are acquired using the industrial camera. Feature points and attitude correction mark points are extracted from the calibration photos. A world coordinate system is constructed based on the reference display screen, including several world coordinates. An attitude correction matrix is ​​generated using the coordinate data of the attitude correction mark points and the reference coordinate data under a horizontally aligned attitude. The attitude correction matrix is ​​used to perform attitude correction processing on the coordinate data of the feature points, generating attitude correction coordinate data. The industrial camera is calibrated using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0090] A reference display screen of the same type as the display screen under test is set up as a calibration board, and a dedicated calibration image is designed based on the parameters of the reference display screen and the test items. The designed calibration image is displayed on the reference display screen according to the test items or the screen characteristics of the reference display screen, and the brightness of the display screen can be adjusted according to environmental parameters (ambient lighting). Feature points and attitude correction mark points are extracted from the calibration photos captured by the industrial camera, and the attitude correction matrix generated by the attitude correction mark points is used to correct the coordinates of the feature points. The generated attitude correction coordinate data can better calibrate the industrial camera, resulting in higher accuracy images captured by the industrial camera after calibration, which are more suitable for the characteristics of the new display screen and the test items.

[0091] The camera calibration method provided in this embodiment displays calibration images on a self-emissive reference display screen, completely eliminating the reliance on a dedicated calibration board. The reference display screen is the same type as the display screen to be tested, but its size is larger or equal to that of the display screen under test. This allows the reference display screen to flexibly switch between different types, sizes, and brightness of calibration images according to the needs of different scenarios, without requiring additional calibration tools. Simultaneously, the screen's self-emissive nature ensures good display performance under various lighting conditions, guaranteeing that the industrial camera can stably and clearly capture calibration features, effectively improving the calibration method's adaptability to complex scenarios. Furthermore, products with screens, such as mobile phones and tablets, are portable and easy to operate, greatly simplifying the calibration process, lowering the operational threshold, and enabling widespread application in various industrial camera parameter calibration scenarios.

[0092] The advantages of the technology in this embodiment over the prior art are as follows:

[0093] (1) Flexibility far exceeds traditional methods: In the existing technology, the size and pattern of the dedicated calibration plate are fixed, and replacement requires additional cost and time. The present invention can flexibly switch between different types (dot matrix, checkerboard, etc.), sizes and brightness of calibration images through the screen according to different scenarios or requirements, without replacing the physical calibration plate, which greatly improves calibration flexibility, reduces equipment costs and improves calibration efficiency.

[0094] (2) Wider range of applications: Traditional calibration boards rely on ambient light. In dim environments, they are difficult to identify clearly because they do not emit light themselves, and in strong light, they are prone to glare and interference. However, this invention utilizes a product screen with self-illuminating properties, whose light intensity is adjustable. It can ensure that the calibration image is clear and distinguishable in various complex scenarios such as dimness, strong light, and uneven lighting, and enables the camera to stably capture calibration features. It is applicable to far more scenarios than traditional methods.

[0095] (3) Significantly improved ease of operation: In the prior art, the dedicated calibration board is inconvenient to carry and difficult to operate in confined spaces or mobile calibration scenarios. The present invention uses the screen of portable devices such as mobile phones and tablets, which is convenient to carry and easy to operate, eliminating the steps of carrying and placing the calibration board, simplifying the calibration process, reducing the difficulty of operation, and allowing non-professionals to easily complete the calibration.

[0096] (4) Guaranteed calibration accuracy: Traditional methods are affected by wear and tear on the calibration plate and deviation in placement. This invention automatically identifies multiple images with different poses, combines precise coordinate calculation with an effective camera calibration algorithm, reduces human error, and can guarantee the calibration accuracy of the camera's internal and external parameters, achieving or even surpassing the calibration effect of traditional methods.

[0097] Please see Figure 2 This application provides an embodiment of a method for extracting feature points and pose correction mark points, comprising:

[0098] 201. Perform quadratic polynomial fitting on the coordinates and grayscale values ​​of the pixels in each calibration photo.

[0099] 202. After fitting, construct the Hessian matrix of the quadratic polynomial.

[0100] 203. Extract feature points from the calibration photos based on the Hessian matrix.

[0101] 204. Extract pose correction mark points from the calibration photos through binarization and morphological processing.

[0102] In this embodiment, the terminal performs quadratic polynomial fitting on the coordinates and grayscale values ​​of pixels in each calibration photo. After fitting, a Hessian matrix of the quadratic polynomial is constructed. Feature points on the calibration photo are extracted based on the Hessian matrix. Finally, pose correction mark points on the calibration photo are extracted through binarization and morphological processing. The specific steps for extracting feature points on the calibration board (reference display screen) are as follows:

[0103] First, a local approximation of the grayscale of the entire image is obtained. For each point in the calibration photograph, the input calibration photograph is approximated using a quadratic polynomial in x and y. The specific form of the polynomial is:

[0104]

[0105] In this embodiment, the coefficients of the polynomial are obtained by fitting the local grayscale values ​​of the image. Here, f(x, y) is an approximate function of the grayscale values ​​of all pixels in the calibration photo, x and y are the row and column coordinates of the image pixels, a is the second-order coefficient of the x term, b is the first-order coefficient of the xy term, c is the second-order coefficient of the y term, d is the first-order coefficient parameter of the x term, e is the first-order coefficient of the y term, and g is a constant. The coefficient values ​​are calculated by substituting the grayscale data and coordinates from the calibration photo.

[0106] After finding the coefficients, calculate the partial derivatives to construct the Hessian matrix H:

[0107]

[0108] in, , , and These are the second partial derivatives of f(x,y) with respect to x, the mixed second partial derivatives with respect to x and y, the mixed second partial derivatives with respect to y and x, and the second partial derivative with respect to y, respectively. The derivatives are:

[0109]

[0110] Calculate the eigenvalues ​​λ of the Hessian matrix H. The eigenvalues ​​satisfy the following equation:

[0111]

[0112] Two eigenvalues, λ1 and λ2, are obtained through equations. If the absolute values ​​of both eigenvalues ​​are greater than a set threshold (Threshold), and they have different signs, then the point is considered a feature point. This is used to calculate the feature points in the calibration image for each pose. Please refer to [reference needed]. Figure 13 , Figure 13 A calibration photograph with marked feature points.

[0113] In this embodiment, the terminal can extract the posture correction mark points on the calibration photo using binarization and morphological processing.

[0114] Please see Figure 3 This application provides an embodiment of a method for generating attitude correction coordinate data, comprising:

[0115] 301. The coordinate data of the feature points are processed by the attitude correction matrix to obtain the coordinate data after attitude correction.

[0116] 302. Sort the coordinate data after attitude correction in a row-first, column-second manner to generate ordered coordinates that correspond one-to-one with each feature point in the world coordinate system.

[0117] 303. Multiply the ordered coordinates by the inverse of the attitude correction matrix to generate attitude correction coordinate data.

[0118] In this embodiment, the coordinate data of all feature points obtained in step 104 or embodiment 2 is denoted as P2(x2, y2), and the coordinates after attitude correction are obtained by multiplying the attitude correction matrix Mat1 by left, and are denoted as P3(x3, y3).

[0119] Next, the terminal sorts the feature points of P3 in a row-first, column-second manner. At this point, the coordinates P4(x4, y4) corresponding to each feature point in the world coordinates Q are obtained. However, since P4(x4, y4) are the coordinates after attitude correction, it is necessary to affine the coordinates P4(x4, y4) back to obtain the actual image coordinates under the current attitude. Therefore, the terminal multiplies P4(x4, y4) by the inverse of the attitude correction matrix Mat1 on the left, and finally generates the attitude correction coordinate data P5(x5, y5).

[0120] Please see Figure 4 This application provides an embodiment of a method for calibrating an industrial camera, comprising:

[0121] 401. Construct the intrinsic parameter matrix of the industrial camera based on the equivalent focal length and principal point coordinates in different axial directions, and construct the distortion coefficient expression of the industrial camera.

[0122] 402. Construct the extrinsic parameter matrix of the industrial camera based on the rotation matrix and translation vector.

[0123] 403. Construct the perspective projection equation of the industrial camera based on the world coordinates in the world coordinate system, attitude correction coordinate data, intrinsic parameter matrix, extrinsic parameter matrix, and the scale factor of the industrial camera.

[0124] Let the world coordinates in the world coordinate system be Q(X, Y, Z), and the corresponding point in the image coordinate system be P5(x5, y5) = P5(u, v). Then the transformation relationship between the two satisfies the perspective projection equation:

[0125]

[0126] Where s is the scale factor (scaling ratio), K is the intrinsic parameter matrix of the industrial camera, R is the rotation matrix under different poses, and T is the translation vector under different poses. The expression for the intrinsic parameter matrix is:

[0127]

[0128] In the intrinsic parameter matrix expression, and These are the equivalent focal lengths along the x-axis and y-axis, respectively. and The coordinates of the main point, together with the coordinates of the main point, constitute the camera's extrinsic parameters.

[0129] In cases of distortion, distortion correction is required:

[0130] The expression for the radial distortion coefficient is:

[0131]

[0132]

[0133] The expression for the tangential distortion coefficient is:

[0134]

[0135]

[0136] in, u and v are the coordinates of feature points in the attitude correction coordinate data. and These are the corrected coordinates.

[0137] p1 and p2 are tangential distortion coefficients, and k1, k2 and k3 are radial distortion coefficients.

[0138] 404. Generate the homography matrix based on the intrinsic and extrinsic parameter matrices, and solve the homography matrix using the attitude correction coordinate data.

[0139] 405. Based on the solved homography matrix, list the equations for each intrinsic parameter in the intrinsic parameter matrix, and solve the intrinsic parameter matrix by combining the perspective projection equation and the least squares method.

[0140] 406. Solve for the extrinsic matrix based on the solved homography matrix and the solved intrinsic parameter matrix.

[0141] 407. Solve for the distortion coefficient expression based on the world coordinates and attitude correction coordinates in the world coordinate system, and generate the distortion correction coefficient.

[0142] In this embodiment, the coordinates calculated for each image in step 2 are first used to obtain several sets of feature points (pose correction coordinate data) under different poses, and their coordinate information in the image coordinate system and the world coordinate system is substituted into the calibration algorithm. For each set of feature points, according to the perspective projection equation, we can obtain:

[0143]

[0144] in , and Let t be the column vector of the rotation matrix R, and t be the translation vector T. Let be the scale factor (scaling ratio) of the i-th calibration photo. and The pose correction coordinate data corresponding to the i-th calibration photo. and These are the world coordinates corresponding to the attitude correction coordinate data.

[0145] make Then H is a 3x3 homography matrix, which can be represented as:

[0146]

[0147] The homography matrix can be solved using the coordinates of feature points (attitude correction coordinate data). Based on the relationship between H and the intrinsic parameter matrix K, an equation about the intrinsic parameter can be derived. Combining multiple homography matrices obtained from multiple images, the intrinsic parameter matrix K is solved using the least squares method.

[0148]

[0149] Where n is the number of images, and m is the number of feature points in each image.

[0150] After determining the intrinsic parameters, calculate the extrinsic parameters R and T according to the following formulas:

[0151]

[0152]

[0153]

[0154]

[0155] Calculate the extrinsic parameters R and T, and simultaneously solve for the distortion coefficients. , and Let H be a column vector. By solving the system of equations, the camera's intrinsic parameters (such as focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (such as rotation matrix, translation vector, etc.) are calculated, thus completing the calibration of the camera parameters.

[0156] Please see Figure 5 This application provides an embodiment of a method for illuminating a reference display screen, wherein the reference display screen is a quantum dot electroluminescent display screen, comprising:

[0157] 501. Turn on the system light source of the acquisition system according to the detection items of the subsequent reference display screen.

[0158] 502. Under various orientations of the reference display screen, determine the sampling points based on the film thickness distribution of the reference display screen, and detect the ambient light intensity at the sampling points.

[0159] 503. When the ambient light intensity is less than the low light environment threshold, generate the base brightness, dynamic contrast ratio, and adaptive Gamma value of the reference display screen based on the ambient light intensity.

[0160] 504. Adjust the base brightness based on the dynamic contrast ratio and adaptive Gamma value to generate the adjusted brightness.

[0161] 505. Light up the calibration image on the reference display screen, and adjust the luminous intensity of the reference display screen according to the brightness adjustment.

[0162] In this embodiment, the terminal first sets the system light source of the acquisition system according to the detection items of the display screen to be tested corresponding to the subsequent reference display screen. Then, according to the thickness distribution of the quantum dot electroluminescent display screen, several acquisition points are determined on the reference display screen. Each thickness has an acquisition point, and the ambient light intensity is detected at the acquisition point to generate the average light intensity.

[0163] When the ambient light intensity L_env is less than the low-light environment threshold L_dark, calculate the base brightness B_base:

[0164] B_base=B_floor+(B_dark-B_floor)×(L_env / L_dark)^γ

[0165] B_floor is the absolute minimum brightness used to ensure basic visibility, B_dark is the reference brightness for low-light environments, and γ is the Gamma coefficient for brightness adjustment. The dynamic contrast ratio is calculated as follows:

[0166] C_enhance=C_max-(C_max-C_dark)×(L_env / L_dark)^k

[0167] C_dark is the contrast benchmark for low-light environments, C_max is the maximum contrast enhancement, and k is the contrast adjustment rate coefficient. The adaptive Gamma value is calculated as follows:

[0168] γ_display=γ_min+[log(1+L_env)×(γ_dark-γ_min)^g] / log(1+L_dark)

[0169] γ_min is the minimum Gamma value, used to significantly improve shadow details, γ_dark is the Gamma benchmark for low-light environments, γ_max is the Gamma value for normal environments, and g is the Gamma adjustment rate coefficient.

[0170] The terminal input calibration image illuminates the reference display screen and reaches the basic brightness. Then, pixel-level display enhancement is performed. For each RGB pixel (r, g, b) of the reference display screen, a dynamic contrast coefficient C_enhance is applied for enhancement.

[0171] r_enhanced=128+C_enhance×(r-128)

[0172] g_enhanced=128+C_enhance×(g-128)

[0173] b_enhanced=128+C_enhance×(b-128)

[0174] Next, the terminal application performs Gamma correction:

[0175] r_final=255×(r_enhanced / 255)^(1 / γ_display)

[0176] g_final=255×(g_enhanced / 255)^(1 / γ_display)

[0177] b_final=255×(b_enhanced / 255)^(1 / γ_display)

[0178] After calculating r_final, g_final, and b_final, the brightness of the reference display screen with the input calibration image can be adjusted. This method can actively adjust the brightness of the calibration board, so that the calibration board is actively brightened in the case of low light, and the calibration image can be clearly displayed on the quantum dot electroluminescent display screen.

[0179] Please see Figure 6This application provides an embodiment of a method for adjusting the grayscale value of the effective area of ​​a display screen in a calibration photograph, wherein the reference display screen is a quantum dot electroluminescent display screen, comprising:

[0180] 601. Obtain the nominal light conversion film thickness of the reference display screen.

[0181] 602. Obtain the actual light conversion film thickness distribution and the actual refractive index distribution corresponding to the actual quantum dot concentration of the reference display screen.

[0182] In this embodiment, a quantum dot electroluminescent display (QLED) is used. Essentially, pre-prepared quantum dot material is used as the light conversion film material and placed between the blue LED backlight and the front-end liquid crystal layer in a specific manner. The quantum dot film is excited by the blue backlight, emitting pure red and green light, which mixes with the remaining blue light to produce high-quality white light, which is then controlled by the liquid crystal pixels. In this embodiment, the quantum dot electroluminescent display (QLED) actively emits light from the quantum dot material of each pixel under direct current drive. This is similar to the working principle of traditional OLEDs, replacing the traditional light-emitting layer with quantum dots.

[0183] In existing technologies, quantum dot inks are typically used to fabricate quantum dot electroluminescent displays (QLEDs). First, synthesized quantum dot particles are separated from the original solvent and then dispersed in a solvent with suitable physical properties (e.g., boiling point, surface tension, viscosity) to form "quantum dot ink." Then, a film-forming process is performed using inkjet printing. Specifically, pixel pits are fabricated on a substrate with pre-fabricated TFT circuits and electrodes (anodes) using photolithography. Then, high-precision inkjet printing equipment precisely sprays red, green, and blue quantum dot inks into the corresponding pixel pits. Finally, annealing (heating) evaporates the solvent, leaving a uniform and smooth quantum dot film. However, in existing technologies, during the ink drying process, the evaporation rate at the edges is faster than at the center. This causes the quantum dot particles to aggregate towards the edges due to surface tension, forming a ring-shaped, non-uniform film that is thin in the center and thick at the edges. Even by adjusting the solvent and evaporation rate, a small amount of this ring-shaped non-uniform film may still exist. Before performing defect detection on this type of new display, the industrial camera also needs to be calibrated. Due to the characteristics of this type of display, such as the presence of light conversion film with lower inner and higher outer pixel layer, uneven quantum dot concentration, and light source refraction, the terminal needs to adjust the grayscale of the calibration plate area on the reference display based on the thickness of the light conversion film and the distribution of quantum dots on the reference display, so that the feature points and posture correction mark points on the calibration photo are clearer.

[0184] The terminal first obtains the nominal light conversion film thickness of the reference display screen. That is, to obtain the standard thickness of the reference display screen.

[0185] Then the terminal obtains the actual light conversion film thickness distribution of the reference display screen. Actual refractive index distribution corresponding to actual quantum dot concentration .

[0186] 603. Calculate the thickness adjustment factor of the reference display screen in each region based on the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflectance coefficient.

[0187] The terminal is based on the nominal light conversion film thickness. Actual light conversion film thickness distribution and the reflectivity of the light conversion film The thickness adjustment factor for the reference display screen in each region is calculated using the following formula:

[0188]

[0189] With varying light conversion film thicknesses across different areas of the reference display screen, the light conversion film will reflect a portion of the light; the light conversion film's reflectance coefficient... Used to adjust the increase in grayscale caused by the thickness of the light conversion film. The actual light conversion film thickness distribution in different areas of the reference display screen. The reflectance coefficient of the light conversion film.

[0190] 604. Calculate the interface transmission adjustment factor based on the actual refractive index distribution.

[0191] In addition to thickness causing refraction, uneven quantum dot concentration can also cause additional light source refraction. In this case, the terminal needs to generate an interface transmission adjustment factor based on the actual refractive index distribution, specifically by directly determining the refractive index as the interface transmission adjustment factor.

[0192] 605. Adjust the grayscale value of the effective area of ​​the display screen in the calibration photo according to the thickness adjustment factor and the interface transmission adjustment factor.

[0193] The terminal adjusts the grayscale value of the effective area of ​​the display screen in the calibration photo according to the thickness adjustment factor and the interface transmission adjustment factor, as shown in the following formula:

[0194]

[0195] The grayscale of the adjusted calibration photo. The grayscale of the calibration photo before adjustment. As an interface transmission adjustment factor, the grayscale value of the calibration photo is adjusted in the above manner, which can solve the problem of unclear feature points and attitude correction mark points caused by uneven thickness of light conversion film and uneven concentration of photons in quantum dot electroluminescent displays, and improve the extraction effect of feature points and attitude correction mark points.

[0196] Please see Figure 7 This application provides an embodiment of a method for preprocessing calibration photographs, comprising:

[0197] 701. Filter the calibration photos.

[0198] In this embodiment, filtering the calibration photos can effectively remove noise.

[0199] Please see Figure 8 This application provides an embodiment of a camera calibration device based on screen images, comprising:

[0200] Design unit 801 is used to design calibration images for an industrial camera in an acquisition system based on the screen parameters of a reference display screen. The acquisition system includes an industrial camera, and the calibration images are set with several feature points and attitude correction mark points.

[0201] The illumination unit 802 is used to illuminate the reference display screen according to the environmental parameters and calibration image of the acquisition system.

[0202] Optionally, the reference display is a quantum dot electroluminescent display.

[0203] The lighting unit 802 specifically includes:

[0204] Turn on the system light source according to the test items of the subsequent reference display screen.

[0205] Under various orientations of the reference display screen, the sampling points are determined based on the film thickness distribution of the reference display screen, and the ambient light intensity is detected at the sampling points.

[0206] When the ambient light intensity is less than the low light environment threshold, the base brightness, dynamic contrast ratio, and adaptive Gamma value of the reference display screen are generated based on the ambient light intensity.

[0207] Adjusted brightness is generated by adjusting the base brightness based on the dynamic contrast ratio and adaptive Gamma value.

[0208] Light up the calibration image on the reference display screen, and adjust the luminous intensity of the reference display screen according to the brightness adjustment.

[0209] The acquisition unit 803 is used to acquire calibration photos of the reference display screen in different orientations using an industrial camera.

[0210] The filtering unit 804 is used to filter the calibration photos.

[0211] The first acquisition unit 805 is used to acquire the nominal refractive index corresponding to the nominal light conversion film thickness and nominal quantum dot concentration of the reference display screen.

[0212] The second acquisition unit 806 is used to acquire the actual light conversion film thickness distribution and the actual refractive index distribution corresponding to the actual quantum dot concentration of the reference display screen.

[0213] The first calculation unit 807 is used to calculate the thickness adjustment factor of the reference display screen in each region based on the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflectance coefficient.

[0214] The second calculation unit 808 is used to calculate the interface transmission adjustment factor based on the nominal refractive index and the actual refractive index distribution.

[0215] The adjustment unit 809 is used to adjust the grayscale value of the effective area of ​​the display screen in the calibration photograph according to the thickness adjustment factor and the interface transmission adjustment factor.

[0216] The extraction unit 810 is used to extract feature points and pose correction mark points from the calibration photo.

[0217] Optionally, the extraction unit 810 specifically includes:

[0218] The coordinates and gray values ​​of each pixel in each calibration photo are fitted using a quadratic polynomial.

[0219] After fitting, the Hessian matrix of the quadratic polynomial is constructed.

[0220] Feature points on the calibration photo are extracted based on the Hessian matrix.

[0221] Posture correction markers were extracted from the calibration photos through binarization and morphological processing.

[0222] Construction unit 811 is used to construct a world coordinate system based on a reference display screen. The world coordinate system includes several world coordinates.

[0223] The first generation unit 812 is used to generate an attitude correction matrix using the coordinate data of the attitude correction mark point and the reference coordinate data under the horizontally aligned attitude.

[0224] The second generation unit 813 is used to perform attitude correction processing on the coordinate data of feature points using the attitude correction matrix to generate attitude correction coordinate data.

[0225] Optionally, the second generation unit 813 specifically includes:

[0226] The coordinate data of feature points are processed by attitude correction matrix to obtain the attitude-corrected coordinate data.

[0227] The coordinate data after attitude correction are sorted in a row-first, column-second manner to generate ordered coordinates that correspond one-to-one with each feature point in the world coordinate system.

[0228] Multiply the ordered coordinates by the inverse of the attitude correction matrix to generate attitude correction coordinate data.

[0229] The calibration unit 814 is used to calibrate the industrial camera using world coordinates and attitude correction coordinates data in the world coordinate system.

[0230] Optionally, the calibration unit 814 specifically includes:

[0231] The intrinsic parameter matrix of the industrial camera is constructed based on the equivalent focal length and principal point coordinates in different axial directions, and the distortion coefficient expression of the industrial camera is also constructed.

[0232] The extrinsic parameter matrix of the industrial camera is constructed based on the rotation matrix and translation vector.

[0233] The perspective projection equation of the industrial camera is constructed based on the world coordinates in the world coordinate system, attitude correction coordinate data, intrinsic parameter matrix, extrinsic parameter matrix, and the scale factor of the industrial camera.

[0234] The homography matrix is ​​generated based on the intrinsic and extrinsic parameter matrices, and then solved using the attitude correction coordinate data.

[0235] Based on the solved homography matrix, equations for each intrinsic parameter in the intrinsic parameter matrix are derived, and the intrinsic parameter matrix is ​​solved by combining the perspective projection equation and the least squares method.

[0236] Solve for the extrinsic matrix based on the solved homography matrix and the solved intrinsic parameter matrix.

[0237] The distortion coefficient expression is solved based on the world coordinates and attitude correction coordinates in the world coordinate system, and the distortion correction coefficient is generated.

[0238] Please see Figure 9 This application provides a camera calibration device based on screen images, comprising:

[0239] Processor 901, memory 902, input / output unit 903, and bus 904.

[0240] The processor 901 is connected to the memory 902, the input / output unit 903, and the bus 904.

[0241] The memory 902 stores a program, and the processor 901 calls the program to execute it, such as... Figure 1, Figure 2 and Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 Camera calibration method in [the context of the text].

[0242] This application provides a computer-readable storage medium on which a program is stored, and when the program is executed on a computer, it performs the following... Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 Camera calibration method in [the context of the text].

[0243] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0244] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0245] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0246] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0247] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A camera calibration method based on screen images, characterized in that, include: A calibration image is designed for the industrial camera in the acquisition system based on the screen parameters of the reference display screen. The acquisition system includes an industrial camera, and the calibration image is provided with several feature points and attitude correction mark points. The reference display screen is illuminated according to the environmental parameters of the acquisition system and the calibration image; The industrial camera was used to capture calibration photos of the reference display screen in different orientations. Extract feature points and pose correction mark points from the calibration photos; A world coordinate system is constructed based on the reference display screen, and the world coordinate system includes several world coordinates; The attitude correction matrix is ​​generated using the coordinate data of the attitude correction mark points and the reference coordinate data under the horizontally aligned attitude. The coordinate data of the feature points are processed using the attitude correction matrix to generate attitude correction coordinate data. The step of using the attitude correction matrix to perform attitude correction processing on the coordinate data of the feature points to generate attitude-corrected coordinate data includes: performing attitude correction processing on the coordinate data of the feature points using the attitude correction matrix to obtain attitude-corrected coordinate data; sorting the attitude-corrected coordinate data in a row-first, column-second manner to generate ordered coordinates that correspond one-to-one with each feature point in the world coordinates; and multiplying the ordered coordinates on the left by the inverse of the attitude correction matrix to generate attitude-corrected coordinate data. The industrial camera is calibrated using world coordinates and attitude correction coordinates in the world coordinate system.

2. The camera calibration method according to claim 1, characterized in that, The step of extracting feature points and pose correction mark points from the calibration photo includes: The coordinates and gray values ​​of each pixel in the calibration photo are fitted with a quadratic polynomial. After fitting, construct the Hessian matrix of the quadratic polynomial; Feature points on the calibration image are extracted based on the Hessian matrix; The pose correction mark points on the calibration photos were extracted by binarization and morphological processing.

3. The camera calibration method according to claim 1, characterized in that, The step of calibrating the industrial camera using world coordinates and attitude correction coordinate data in the world coordinate system includes: The intrinsic parameter matrix of the industrial camera is constructed based on the equivalent focal length and principal point coordinates in different axial directions, and the distortion coefficient expression of the industrial camera is also constructed. The extrinsic parameter matrix of the industrial camera is constructed based on the rotation matrix and translation vector. The perspective projection equation of the industrial camera is constructed based on the world coordinates in the world coordinate system, the attitude correction coordinate data, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of the industrial camera. A homography matrix is ​​generated based on the intrinsic parameter matrix and the extrinsic parameter matrix, and the homography matrix is ​​solved using the attitude correction coordinate data. Based on the solved homography matrix, list the equations for each intrinsic parameter in the intrinsic parameter matrix, and solve the intrinsic parameter matrix by combining the perspective projection equation and the least squares method. Solve for the extrinsic matrix based on the solved homography matrix and the solved intrinsic parameter matrix; The distortion coefficient expression is solved based on the world coordinates and attitude correction coordinates in the world coordinate system to generate the distortion correction coefficient.

4. The camera calibration method according to any one of claims 1 to 3, characterized in that, The reference display screen is a quantum dot electroluminescent display screen; The step of illuminating the reference display screen based on the environmental parameters of the acquisition system and the calibration image includes: The acquisition system will turn on its light source according to the subsequent detection items of the reference display screen; Under various orientations of the reference display screen, the sampling points are determined based on the film thickness distribution of the reference display screen, and the ambient light intensity is detected at the sampling points. When the ambient light intensity is less than the low light environment threshold, the base brightness, dynamic contrast ratio and adaptive Gamma value of the reference display screen are generated according to the ambient light intensity. The base brightness is adjusted based on the dynamic contrast coefficient and the adaptive Gamma value to generate the adjusted brightness; The reference display screen is made to light up the calibration image, and the luminous intensity of the reference display screen is adjusted according to the brightness adjustment.

5. The camera calibration method according to any one of claims 1 to 3, characterized in that, The reference display screen is a quantum dot electroluminescent display screen; After the step of acquiring calibration photos of the reference display screen in different postures using the industrial camera, and before the step of extracting feature points and posture correction mark points from the calibration photos, the camera calibration method further includes: Obtain the nominal light conversion film thickness of the reference display screen; Obtain the actual light conversion film thickness distribution and the actual refractive index distribution corresponding to the actual quantum dot concentration of the reference display screen; The thickness adjustment factor of the reference display screen in each region is calculated based on the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflectance coefficient. Calculate the interface transmission adjustment factor based on the actual refractive index distribution; The grayscale value of the effective area of ​​the display screen in the calibration photograph is adjusted according to the thickness adjustment factor and the interface transmission adjustment factor.

6. The camera calibration method according to any one of claims 1 to 3, characterized in that, After acquiring calibration photos of the reference display screen in different postures using the industrial camera, and before the step of extracting feature points and posture correction mark points from the calibration photos, the camera calibration method further includes: The calibration photos are filtered.

7. A camera calibration device based on screen images, characterized in that, include: The design unit is used to design a calibration image for an industrial camera in the acquisition system based on the screen parameters of a reference display screen. The acquisition system includes an industrial camera, and the calibration image is provided with several feature points and attitude correction mark points. The illumination unit is used to illuminate the reference display screen according to the environmental parameters of the acquisition system and the calibration image; The acquisition unit is used to acquire calibration photos of the reference display screen in different postures using the industrial camera; The extraction unit is used to extract feature points and pose correction mark points from the calibration photo; A construction unit is used to construct a world coordinate system based on the reference display screen, the world coordinate system including a plurality of world coordinates; The first generation unit is used to generate an attitude correction matrix using the coordinate data of the attitude correction mark point and the reference coordinate data under the horizontally aligned attitude. The second generation unit is used to perform attitude correction processing on the coordinate data of the feature points using the attitude correction matrix to generate attitude correction coordinate data. The second generation unit specifically includes: The coordinate data of feature points are processed by attitude correction matrix to obtain attitude-corrected coordinate data; the attitude-corrected coordinate data are sorted in a row-first, column-second manner to generate ordered coordinates that correspond one-to-one with each feature point in the world coordinate system; the ordered coordinates are multiplied by the inverse of attitude correction matrix to generate attitude-corrected coordinate data. The calibration unit is used to calibrate the industrial camera using world coordinates and attitude correction coordinate data in the world coordinate system.

8. The camera calibration device according to claim 7, characterized in that, The extraction unit specifically includes: The coordinates and gray values ​​of each pixel in the calibration photo are fitted with a quadratic polynomial. After fitting, construct the Hessian matrix of the quadratic polynomial; Feature points on the calibration image are extracted based on the Hessian matrix; The pose correction mark points on the calibration photos were extracted by binarization and morphological processing.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a program that, when executed on a computer, performs the camera calibration method as described in any one of claims 1 to 6.

Images