Camera calibration method and device based on screen image and storage medium

Abstract

The invention discloses a camera calibration method and device based on a screen image, and a storage medium, and is used for improving the precision of an image collected by an industrial camera. Designing a calibration image for an industrial camera in the acquisition system according to the screen parameters of the reference display screen; lightening the reference display screen according to the environment parameters of the acquisition system and the calibration image; using an industrial camera to collect calibration photos of the reference display screen in different postures; extracting feature points and posture correction mark points in the calibration photo; constructing a world coordinate system according to the reference display screen, wherein the world coordinate system comprises a plurality of world coordinates; generating an attitude correction matrix by using the coordinate data of the attitude correction mark point and the reference coordinate data under the horizontal correction attitude; performing attitude correction processing on the coordinate data of the feature points by using the attitude correction matrix to generate attitude correction coordinate data; and calibrating the industrial camera by using the world coordinates in the world coordinate system and the attitude correction coordinate data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of display screen detection, and in particular to a camera calibration method and device based on a screen image and a storage medium. BACKGROUND

[0002] With the innovation of science and technology, new display technologies such as MicroLED and flexible folding screens are reshaping the industry landscape. As a core link in the display screen industry chain, the quality detection technology of display screens continues to be valued, and various detections on the surface of the display screen have become a key factor in determining the quality of the display screen product. Among the many detection items of the display screen, it is an essential part to use an industrial camera to collect images of the display screen. Before image collection, the industrial camera often needs to be calibrated.

[0003] In the traditional camera calibration method, a special calibration board is needed, and there are many disadvantages that cannot be overcome. First, the physical properties of such a calibration board are fixed, especially the size and pattern, which cannot be flexibly adjusted according to the screen parameters, detection items and actual detection scene of different display screens. For example, the recognition effect of such a calibration board is seriously dependent on the environmental light conditions. In an environment with insufficient light intensity, the environmental reflection light relied on by the calibration board itself is insufficient, making it difficult for the camera to clearly capture its features, thereby affecting the calibration accuracy. In a strong light environment, such a calibration board is also prone to reflection, which will also interfere with the recognition of feature points on the calibration board by the industrial camera. Moreover, with the continuous emergence of new display screens, the detection items of new display screens are increasing, and the precision of the detection items is also improving. The calibration quality of the industrial camera affects the quality of the collected images of the new display screen, and further affects the detection quality of the new display screen. The existing camera calibration method does not calibrate different poses of the new display screen. This makes it necessary to frequently replace calibration boards of different specifications and constantly adjust the system light source to adapt to the detection items of the new display screen, but the effect is poor, which not only reduces the efficiency of the industrial camera calibration in the display screen detection, but also reduces the precision of the collected images of the industrial camera. SUMMARY

[0004] The present application discloses a camera calibration method and device based on a screen image and a storage medium for improving the precision of images collected by an industrial camera.

[0005] In a first aspect, an embodiment of the present application provides a camera calibration method based on a screen image, comprising: According to the screen parameters of the reference display screen, a calibration image is designed for an industrial camera in a collection system, the collection system comprising the industrial camera, and the calibration image is provided with a plurality of feature points and attitude correction mark points; the reference display screen is lighted according to the environmental parameters of the collection system and the calibration image; the calibration photos of the reference display screen in different attitudes are collected by using the industrial camera; the feature points and the attitude correction mark points in the calibration photos are extracted; a world coordinate system is constructed according to the reference display screen, the world coordinate system comprising a plurality of world coordinates; an attitude correction matrix is generated by using the coordinate data of the attitude correction mark points and the reference coordinate data in the horizontal alignment attitude; the coordinate data of the feature points are processed by using the attitude correction matrix to generate attitude correction coordinate data; and the industrial camera is calibrated by using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0006] Optionally, the step of extracting the feature points and the attitude correction mark points in the calibration photos comprises: the coordinate positions of the pixel points in each calibration photo and the gray values of the pixel points are fitted by using a quadratic polynomial; after the fitting is completed, a Hessian matrix of the quadratic polynomial is constructed; the feature points on the calibration photo are extracted according to the Hessian matrix; and the attitude correction mark points on the calibration photo are extracted by using binaryzation processing and morphological processing.

[0007] Optionally, the step of processing the coordinate data of the feature points by using the attitude correction matrix to generate the attitude correction coordinate data comprises: the coordinate data of the feature points are processed by using the attitude correction matrix to obtain the coordinate data after the attitude correction; the coordinate data after the attitude correction are sorted in the order of row first and column second to generate the ordered coordinates corresponding to each feature point in the world coordinates; and the ordered coordinates are multiplied by the inverse matrix of the attitude correction matrix to generate the attitude correction coordinate data.

[0008] Optionally, the step of calibrating the industrial camera by using the world coordinates in the world coordinate system and the attitude correction coordinate data comprises: According to the equivalent focal length and the principal point coordinates in different axis directions, an intrinsic matrix of the industrial camera is constructed, and an expression of distortion coefficient of the industrial camera is constructed; according to a rotation matrix and a translation vector, an extrinsic matrix of the industrial camera is constructed; according to the world coordinates in the world coordinate system, the posture correction coordinate data, the intrinsic matrix, the extrinsic matrix and a scale factor of the industrial camera, a perspective projection equation of the industrial camera is constructed; according to the intrinsic matrix and the extrinsic matrix, a homography matrix is generated, and the homography matrix is solved through the posture correction coordinate data; according to the solved homography matrix, equations about each intrinsic parameter in the intrinsic matrix are listed, and the intrinsic matrix is solved in combination with the perspective projection equation and the least square method; according to the solved homography matrix and the solved intrinsic matrix, the extrinsic matrix is solved; according to the world coordinates in the world coordinate system and the posture correction coordinate data, the expression of the distortion coefficient is solved, and a distortion correction coefficient is generated.

[0009] Optionally, the reference display screen is a quantum dot electroluminescent display screen. According to the environment parameters of the acquisition system and the step of lighting the reference display screen with the calibration image, the step comprises: The acquisition system is turned on according to the subsequent detection items of the reference display screen; the acquisition points are determined according to the thickness distribution of the reference display screen in each posture of the reference display screen, and the ambient light intensity is detected at the acquisition points; when the ambient light intensity is less than the low-light environment threshold, the basic brightness, the dynamic contrast coefficient and the adaptive Gamma value of the reference display screen are generated according to the ambient light intensity; the basic brightness is adjusted according to the dynamic contrast coefficient and the adaptive Gamma value to generate an adjusted brightness; and the reference display screen is lit with the calibration image, and the luminous intensity of the reference display screen is adjusted according to the adjusted brightness.

[0010] Optionally, the reference display screen is a quantum dot electroluminescent display screen. After the step of using the industrial camera to acquire the calibration photos of the reference display screen in different postures, and before the step of extracting the feature point coordinates and the posture correction feature coordinate data in the calibration photos, the camera calibration method further comprises: The nominal light conversion film thickness of the reference display screen is acquired; the actual light conversion film thickness distribution and the actual refractive index distribution corresponding to the actual quantum dot concentration of the reference display screen are acquired; the thickness adjustment factor of the reference display screen in each region is calculated according to the nominal light conversion film thickness, the actual light conversion film thickness distribution and the reflection coefficient of the light conversion film; the interface transmission adjustment factor is calculated according to the actual refractive index distribution; and the gray value of the effective area of the display screen in the calibration photo is adjusted according to the thickness adjustment factor and the interface transmission adjustment factor.

[0011] Optionally, after the step of using the industrial camera to acquire the calibration photos of the reference display screen in different postures, and before the step of extracting the feature point coordinates and the posture correction feature coordinate data in the calibration photos, the camera calibration method further comprises: Filtering the calibration photos.

[0012] In a second aspect, embodiments of the present application provide a camera calibration device based on a screen image, comprising: A design unit configured to design a calibration image for an industrial camera in a capture system according to screen parameters of a reference display screen, the capture system comprising the industrial camera, the calibration image being provided with a plurality of feature points and attitude correction mark points; a lighting unit configured to light up the reference display screen according to environmental parameters of the capture system and the calibration image; a capture unit configured to capture calibration photos of the reference display screen in different attitudes using the industrial camera; an extraction unit configured to extract the feature points and the attitude correction mark points in the calibration photos; a construction unit configured to construct a world coordinate system according to the reference display screen, the world coordinate system comprising a plurality of world coordinates; a first generation unit configured to generate an attitude correction matrix using coordinate data of the attitude correction mark points and reference coordinate data in a horizontal attitude; a second generation unit configured to perform attitude correction processing on coordinate data of the feature points using the attitude correction matrix to generate attitude correction coordinate data; and a calibration unit configured to calibrate the industrial camera using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0013] Optionally, the extraction unit specifically comprises: The coordinate positions of the pixel points in each calibration photo and the gray values of the pixel points are fitted by a quadratic polynomial; after the fitting is completed, a Hessian matrix of the quadratic polynomial is constructed; the feature points on the calibration photo are extracted according to the Hessian matrix; and the attitude correction mark points on the calibration photo are extracted through binarization processing and morphological processing.

[0014] Optionally, the second generation unit specifically comprises: The coordinate data of the feature points are processed by the attitude correction matrix to obtain the coordinate data after attitude correction; the coordinate data after attitude correction is sorted in the order of row first and column second to generate ordered coordinates corresponding to each feature point in the world coordinates; and the ordered coordinates are left multiplied by an inverse matrix of the attitude correction matrix to generate the attitude correction coordinate data.

[0015] Optionally, the calibration unit specifically comprises: An internal parameter matrix of the industrial camera is constructed according to equivalent focal lengths and principal point coordinates in different axis directions, and an expression of distortion coefficients of the industrial camera is constructed; an external parameter matrix of the industrial camera is constructed according to a rotation matrix and a translation vector; a perspective projection equation of the industrial camera is constructed according to world coordinates in a world coordinate system, posture correction coordinate data, the internal parameter matrix, the external parameter matrix and a scale factor of the industrial camera; a homography matrix is generated according to the internal parameter matrix and the external parameter matrix, and the homography matrix is solved through the posture correction coordinate data; equations about each internal parameter in the internal parameter matrix are listed according to the solved homography matrix, and the internal parameter matrix is solved in combination with the perspective projection equation and a least square method; the external parameter matrix is solved according to the solved homography matrix and the solved internal parameter matrix; the expression of distortion coefficients is solved according to the world coordinates in the world coordinate system and the posture correction coordinate data, and distortion correction coefficients are generated.

[0016] Optionally, the reference display screen is a quantum dot electroluminescent display screen. The lighting unit specifically includes: The acquisition system is turned on according to subsequent detection items of the reference display screen; in each posture of the reference display screen, the acquisition point is determined according to the thickness distribution of the reference display screen, and the ambient light intensity is detected at the acquisition point; when the ambient light intensity is less than a low-light environment threshold, the basic brightness, the dynamic contrast coefficient and the adaptive Gamma value of the reference display screen are generated according to the ambient light intensity; the basic brightness is adjusted according to the dynamic contrast coefficient and the adaptive Gamma value to generate the adjusted brightness; the reference display screen is lit with the calibration image, and the light intensity of the reference display screen is adjusted according to the adjusted brightness.

[0017] Optionally, the reference display screen is a quantum dot electroluminescent display screen. After the acquisition unit and before the extraction unit, the camera calibration device further includes: The first acquisition unit is configured to acquire a nominal light conversion film thickness of the reference display screen; the second acquisition unit is configured to acquire an actual light conversion film thickness distribution and an actual refractive index distribution corresponding to an actual quantum dot concentration of the reference display screen; the first calculation unit is configured to calculate a thickness adjustment factor of each region of the reference display screen according to the nominal light conversion film thickness, the actual light conversion film thickness distribution and a reflection coefficient of the light conversion film; the second calculation unit is configured to calculate an interface transmission adjustment factor according to the actual refractive index distribution; and the adjustment unit is configured to adjust a gray value of an effective region of the display screen in the calibration photo according to the thickness adjustment factor and the interface transmission adjustment factor.

[0018] Optionally, after the acquisition unit and before the extraction unit, the camera calibration device further includes: The filtering unit is configured to perform filtering processing on the calibration photo.

[0019] In a third aspect, embodiments of the present application provide a camera calibration device based on a screen image, comprising: a processor, a memory, an input / output unit, and a bus; the processor is connected to the memory, the input / output unit, and the bus; the memory stores a program, and the processor invokes the program to execute the camera calibration method as in the first aspect and any optional aspect of the first aspect.

[0020] In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, which stores a program, and the program, when executed on a computer, executes the camera calibration method as in the first aspect and any optional aspect of the first aspect.

[0021] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages: In the present application, first, a calibration image is designed for an industrial camera in a collection system according to screen parameters of a reference display screen, the collection system comprising the industrial camera, and the calibration image is provided with a plurality of feature points and attitude correction mark points. The reference display screen is lighted according to environmental parameters of the collection system and the calibration image. The industrial camera is used to collect calibration photos of the reference display screen in different attitudes. The feature points and the attitude correction mark points in the calibration photos are extracted. A world coordinate system is constructed according to the reference display screen, the world coordinate system comprising a plurality of world coordinates. An attitude correction matrix is generated using coordinate data of the attitude correction mark points and reference coordinate data in a horizontal attitude. The coordinate data of the feature points are processed using the attitude correction matrix to generate attitude correction coordinate data. The industrial camera is calibrated using the world coordinates in the world coordinate system and the attitude correction coordinate data.

[0022] By setting a reference display screen of the same type as the display screen to be measured as a calibration board, and designing a special calibration image according to parameters of the reference display screen and the detected items, the designed calibration image is displayed on the reference display screen according to the detected items or the screen features of the reference display screen, and the brightness of the display screen can be adjusted according to the environmental parameters (environmental light). The feature points and the attitude correction mark points in the calibration photos collected by the industrial camera are extracted, and the attitude correction matrix generated using the attitude correction mark points is used to correct the attitude of the feature points to generate attitude correction coordinate data, which can better calibrate the industrial camera, so that the accuracy of the images collected by the industrial camera after calibration is higher, and the characteristics of the new display screen and the detected items are better adapted. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0024] Figure 1 The first embodiment of the method for calibrating the camera based on the screen image of the present application is shown in the figure. Figure 2 The first embodiment of the method for extracting the feature points and the mark points of the present application is shown in the figure. Figure 3 The first embodiment of the method for generating the posture correction coordinate data of the present application is shown in the figure. Figure 4 The first embodiment of the method for calibrating the industrial camera of the present application is shown in the figure. Figure 5 The first embodiment of the method for lighting the reference display screen of the present application is shown in the figure. Figure 6 The first embodiment of the method for adjusting the gray value of the effective area of the display screen in the calibration photo of the present application is shown in the figure. Figure 7 The first embodiment of the method for preprocessing the calibration photo of the present application is shown in the figure. Figure 8 The first embodiment of the camera calibration device based on the screen image of the present application is shown in the figure. Figure 9 The second embodiment of the camera calibration device based on the screen image of the present application is shown in the figure. Figure 10 The first embodiment of the checkerboard calibration board of the present application is shown in the figure. Figure 11 The first embodiment of the tilted checkerboard calibration board of the present application is shown in the figure. Figure 12 The first embodiment of the honeycomb calibration board of the present application is shown in the figure. Figure 13 The first embodiment of the checkerboard calibration board with the feature points extracted of the present application is shown in the figure. DETAILED DESCRIPTION

[0025] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0026] It will be understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]," depending on the context.

[0029] In addition, the terms "first," "second," "third," etc. are used herein only to describe different instances, and do not imply relative importance.

[0030] The phrases "one embodiment," "some embodiments," "an embodiment," "another embodiment," "at least one embodiment," "at least one other embodiment," "another embodiment," "at least one other embodiment," "some other embodiments," and the like as used herein do not necessarily refer to the same embodiment(s), though they can. The use of the term "embodiment" or "some embodiments" in the description is intended to present the feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearance of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in at least one embodiment," "in at least one other embodiment," "in some other embodiments," and the like in various places in the specification are not necessarily all referring to the same embodiment, although they can. The use of the term "comprises" or "comprising" in the description or the claims means "including, but not limited to" and not "consists of" or "consisting of" unless otherwise noted.

[0031] In the prior art, a special calibration board needs to be used, and there are many disadvantages in the use of the special calibration board. First, the physical properties of the calibration board are fixed, and especially the size and pattern cannot be flexibly adjusted according to the screen parameters of different display screens, detection items and actual detection scenes. For example, the recognition effect of the calibration board is seriously dependent on the environmental light conditions. In an environment with insufficient light intensity, the environmental reflection light relied on by the calibration board itself is insufficient, which makes it difficult for the camera to clearly capture the characteristics, and thus affects the calibration accuracy. In a strong light environment, the calibration board is also prone to produce reflection, which will also interfere with the recognition of the feature points on the calibration board by the industrial camera. Moreover, with the continuous emergence of new display screens, the detection items of the new display screens are increasing, and the precision of the detection items is also increasing. The calibration quality of the industrial camera affects the quality of the collected images of the new display screen, and thus affects the detection quality of the new display screen. This makes it necessary to not only replace calibration boards of different specifications, but also constantly adjust the system light source to adapt to the detection items and detection accuracy of the new display screen, thereby reducing the efficiency of the industrial camera calibration in the display screen detection Based on this, the present application discloses a camera calibration method and device based on screen images and a storage medium, which are used to improve the accuracy of images collected by an industrial camera.

[0032] The technical solutions in the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0033] The method of the present application can be applied to a server, a device, a terminal or other devices with logical processing capability, and the present application is not limited thereto. For convenience of description, the following will be described taking a terminal as an example.

[0034] Please refer to Figure 1 The present application provides an embodiment of a camera calibration method based on screen images, which comprises the following steps. 101. Designing a calibration image for an industrial camera in a collection system according to the screen parameters of a reference display screen, the collection system comprising the industrial camera, and the calibration image being provided with a plurality of feature points and attitude correction mark points.

[0035] In this embodiment, the terminal designs a calibration image for an industrial camera in a collection system according to the screen parameters of a reference display screen, which aims to make a calibration image according to product features. Specifically, a calibration image of a corresponding size is made according to the product resolution. Please refer to Figure 10 and Figure 12 , Figure 10The calibration image corresponding to the chessboard calibration plate (reference display screen calibration plate) is shown in the following figure, Figure 12 The calibration image corresponding to the honeycomb calibration plate (circle) is shown in the following figure, and other types of calibration images can also be used. In this application, the calibration image needs to set feature points and pose correction mark points for pose correction.

[0036] 102. The reference display screen is lit according to the environmental parameters of the acquisition system and the calibration image.

[0037] In this embodiment, the terminal lights up the reference display screen according to the environmental parameters of the acquisition system and the calibration image. The terminal inputs the calibration image into the reference display screen, and adjusts the brightness of the reference display screen according to the real-time monitored environmental parameters. The specific lighting process is described in subsequent embodiments.

[0038] 103. The industrial camera is used to acquire calibration photos of the reference display screen in different poses.

[0039] The terminal uses the industrial camera to acquire calibration photos of the reference display screen in different poses. Specifically, when the terminal uses the industrial camera to take photos, the pose of the reference display screen relative to the industrial camera is constantly changed during the shooting process, including rotation, translation, inclination, etc. The industrial camera synchronously shoots, and at least 20 photos of calibration images containing the display area of the reference display screen in different poses (calibration photos) are acquired. Figure 11 Figure 11 The calibration photos of the reference display screen in different poses are shown in the following figure.

[0040] 104. Extract feature points and pose correction mark points from the calibration photos.

[0041] In this embodiment, the terminal extracts feature points and pose correction mark points from the calibration photos. In the chessboard calibration plate, the feature points are the vertex positions of the black and white squares, and the pose correction mark points are set inside the black and white squares. There are 5 special mark points (pose correction mark points) on the chessboard calibration plate, including 5 circular pose correction mark points, 3 black circles, and 2 white circles, which are used for pose correction. The specific extraction method is described in detail in subsequent embodiments.

[0042] 105. Construct a world coordinate system according to the reference display screen, which includes a plurality of world coordinates.

[0043] The terminal constructs a world coordinate system according to the reference display screen, which includes a plurality of world coordinates. Specifically, the terminal establishes a coordinate system with the first pixel in the top left corner of the reference display screen as the origin. Given that the screen resolution is W*H, the physical size of a single pixel is m, the row spacing of the feature points is R, and the column spacing is C, the world coordinates Q are calculated as follows:​

[0044] 106. Generating a pose correction matrix using the coordinate data of the pose correction mark point and the reference coordinate data in the horizontal pose.

[0045] The terminal generates a pose correction matrix using the coordinate data of the pose correction mark point and the reference coordinate data in the horizontal pose. The coordinates of the pose correction matrix are denoted as P1(row1, col1), and the ideal coordinates (i.e., the coordinates in the horizontal pose) are denoted as Q1(row2, col2). The pose correction matrix Mat1 is obtained as follows:

[0046] P1X and P1Y correspond to row1 and col1, respectively, and Q1X and Q1Y correspond to row2 and col2, respectively. The pose correction matrix Mat1 is calculated through the above formula, and the feature points are subjected to pose correction.

[0047] 107. Subjecting the coordinate data of the feature points to pose correction processing using the pose correction matrix to generate pose correction coordinate data.

[0048] In this embodiment, the terminal subjects the coordinate data of the feature points to pose correction processing using the pose correction matrix, so that the coordinate data of the feature points in different poses can be calibrated for the industrial camera after the correction is completed. The specific pose correction processing method is described in detail in subsequent embodiments.

[0049] 108. Calibrating the industrial camera using the world coordinates in the world coordinate system and the pose correction coordinate data.

[0050] After the pose correction is completed, the terminal can calibrate the industrial camera using the world coordinate system constructed according to the reference display screen and the pose correction coordinate data, so that the industrial camera can adapt to the size and display screen characteristics of the to-be-measured display screen.

[0051] In this embodiment, first, a calibration image is designed for an industrial camera in a collection system according to screen parameters of a reference display screen. The collection system includes the industrial camera, and the calibration image is provided with a plurality of feature points and posture correction mark points. The reference display screen is lighted according to environmental parameters of the collection system and the calibration image. The industrial camera is used to collect calibration photos of the reference display screen in different postures. The feature points and the posture correction mark points in the calibration photos are extracted. A world coordinate system is constructed according to the reference display screen, and the world coordinate system includes a plurality of world coordinates. A posture correction matrix is generated using coordinate data of the posture correction mark points and reference coordinate data in a horizontal posture. The coordinate data of the feature points is subjected to posture correction processing using the posture correction matrix to generate posture correction coordinate data. The industrial camera is calibrated using the world coordinates in the world coordinate system and the posture correction coordinate data.

[0052] By setting a reference display screen of the same type as the display screen to be measured as a calibration board, and designing a special calibration image according to the parameters of the reference display screen and the detection items. The designed calibration image is displayed on the reference display screen according to the detection items or the screen features of the reference display screen, and the brightness of the display screen can be adjusted according to the environmental parameters (environmental light). The feature points and the posture correction mark points in the calibration photos collected by the industrial camera are extracted, and the posture correction matrix generated using the posture correction mark points is used to correct the posture of the feature points, and the generated posture correction coordinate data can better calibrate the industrial camera, so that the accuracy of the image collected by the industrial camera after calibration is higher, and it is more suitable for the characteristics of the new display screen and the detection items.

[0053] The camera calibration method provided in this embodiment displays a calibration image through a reference display screen with self-luminous characteristics, and completely gets rid of the dependence on a special calibration board. The reference display screen is of the same type as the display screen to be measured, and the size is greater than or equal to the display screen to be measured, so that the reference display screen can flexibly switch different types, different sizes and different brightness calibration images according to the needs of different scenes, without the need for additional replacement of calibration tools. At the same time, the self-luminous characteristics of the screen can maintain good display effect in various light environments, ensuring that the industrial camera can stably and clearly capture the calibration features, effectively improving the adaptability of the calibration method to complex scenes. Moreover, products with screens such as mobile phones and tablet computers are easy to carry and simple to operate, greatly simplifying the calibration process and reducing the operation threshold, and can be widely applied to various industrial camera parameter calibration scenes.

[0054] The beneficial effects of the technology of this embodiment are as follows: (1) Flexibility far beyond the traditional method: the size and pattern of the special calibration board in the prior art are fixed, and additional cost and time are required for replacement. The present application can flexibly switch different types (dot matrix, chessboard, etc.), sizes, and brightness of the calibration image on the screen according to different scenes or requirements, without the need to replace the physical calibration board, greatly improving the calibration flexibility, reducing the cost of equipment, and improving the efficiency of calibration.

[0055] (2) More extensive adaptation to scenes: Traditional calibration boards rely on ambient light and are difficult to be clearly identified in dim environments due to their lack of self-luminous properties, and are easily disturbed by reflected light in strong light. The present application uses a product screen with self-luminous properties, and its luminous intensity can be adjusted, ensuring clear and identifiable calibration images in various complex scenes such as dim, strong light, and uneven light, allowing the camera to stably capture the calibration features, and adapting to more scenes than traditional methods.

[0056] (3) Significant improvement in operational simplicity: The special calibration board in the prior art is inconvenient to carry and difficult to operate in small spaces or mobile calibration scenes. The present application uses the screen of a mobile phone, tablet computer, or other portable device, which is convenient to carry and easy to operate, eliminating the need to carry and place the calibration board, simplifying the calibration process, reducing the difficulty of operation, and allowing non-professionals to easily complete the calibration.

[0057] (4) Guaranteed calibration accuracy: Traditional methods are affected by calibration board wear, placement position deviation, and other factors. The present application automatically identifies multiple images in multiple poses, combines precise coordinate calculation and effective camera calibration algorithms, reduces human operation errors, and can guarantee the calibration accuracy of camera internal and external parameters, achieving or even exceeding the calibration effect of traditional methods.

[0058] Please refer to Figure 2 The present application provides an embodiment of a method for extracting feature points and posture correction mark points, comprising: 201. Perform quadratic polynomial fitting on the coordinate position and gray value of each pixel point in the calibration photo.

[0059] 202. After fitting, construct the Hessian matrix of the quadratic polynomial.

[0060] 203. Extract feature points on the calibration photo according to the Hessian matrix.

[0061] 204. Extract posture correction mark points on the calibration photo through binary processing and morphological processing.

[0062] In this embodiment, the terminal performs quadratic polynomial fitting on the coordinate position and the gray value of each pixel point in each calibration photo, constructs a Hessian matrix of the quadratic polynomial after the fitting is completed, extracts feature points on the calibration photo according to the Hessian matrix, and finally extracts the posture correction mark point on the calibration photo through binarization processing and morphological processing. The specific steps of extracting the feature points on the calibration board (reference display screen) are as follows: First, the local approximation of the gray value of the entire image is obtained, and for each point in the calibration photo, the input calibration photo is approximated by a quadratic polynomial with respect to x and y. The specific form of the polynomial is:

[0063] In this embodiment, the coefficients of the polynomial are obtained by fitting the local gray value of the image. Wherein f(x, y) is an approximation function of all pixel gray values in the calibration photo, x and y are image pixel row and column coordinates, a is the second order coefficient of the x term, b is the first order coefficient of the xy term, c is the second order coefficient of the y term, d is the first order coefficient parameter of the x term, e is the first order coefficient of the y term, and g is a constant. The gray data and coordinates in the calibration photo are substituted to obtain the values of each coefficient.

[0064] After the coefficients are obtained, the partial derivative is calculated, and the Hessian matrix H is constructed:

[0065] Wherein, , , and are the second order partial derivative of f(x, y) with respect to x, the second order mixed partial derivative with respect to x and y, the second order mixed partial derivative with respect to y and x, and the second order partial derivative with respect to y, respectively. The derivative is:

[0066] The eigenvalues λ of the Hessian matrix H are calculated, and the eigenvalues satisfy the following equation:

[0067] Two eigenvalues λ1 and λ2 are obtained by the equation. If the absolute values of the two eigenvalues are both greater than the set threshold Threshold, and their signs are different, the point is considered as a feature point, and the feature points in the calibration photo under each posture are obtained. Please refer to Figure 13 , Figure 13 for the calibration photo with the feature points marked.

[0068] In this embodiment, the terminal can extract the posture correction mark point on the calibration photo by using binarization processing and morphological processing.

[0069] Please refer toFigure 3 The application provides one embodiment of a method for generating pose correction coordinate data, comprising: 301. Perform pose correction processing on the coordinate data of the feature points by the pose correction matrix to obtain pose-corrected coordinate data.

[0070] 302. Perform coordinate sorting on the pose-corrected coordinate data in the order of row first and column second to generate ordered coordinates corresponding to each feature point in the world coordinates.

[0071] 303. Multiply the ordered coordinates by the inverse matrix of the pose correction matrix to generate the pose correction coordinate data.

[0072] In this embodiment, the coordinate data of all the feature points obtained in step 104 or in embodiment 2 is denoted as P2(x2, y2), and the pose-corrected coordinate data obtained by multiplying the pose correction matrix Mat1 is denoted as P3(x3, y3).

[0073] Next, the terminal sorts the P3 coordinates according to the order of row first and column second, and at this time, the coordinate P4(x4, y4) corresponding to each feature point in the world coordinates Q can be obtained. However, since the P4(x4, y4) is pose-corrected coordinate data, the actual image coordinate data under the current pose can be obtained only by performing affine transformation on the coordinate P4(x4, y4). Therefore, the terminal multiplies the P4(x4, y4) by the inverse of the pose correction matrix Mat1 to finally generate the pose correction coordinate data P5(x5, y5).

[0074] Please refer to Figure 4 The application provides one embodiment of a method for calibrating an industrial camera, comprising: 401. Construct an intrinsic matrix of the industrial camera according to the equivalent focal lengths and principal point coordinates in different axial directions, and construct an expression of the distortion coefficient of the industrial camera.

[0075] 402. Construct an extrinsic matrix of the industrial camera according to the rotation matrix and the translation vector.

[0076] 403. Construct a perspective projection equation of the industrial camera according to the world coordinates in the world coordinate system, the pose correction coordinate data, the intrinsic matrix, the extrinsic matrix and the scale factor of the industrial camera.

[0077] Suppose the world coordinates Q(X, Y, Z) in the world coordinate system, and the corresponding point in the image coordinate system is P5(x5, y5)=P5(u, v), and the conversion relationship between the two satisfies the perspective projection equation:

[0078] Wherein, s is a scale factor (scaling ratio), K is an intrinsic matrix of an industrial camera, R is a rotation matrix under different poses, T is a translation vector under different poses, and the intrinsic matrix expression is:

[0079] In the intrinsic matrix expression, and are the equivalent focal lengths in the x-axis and y-axis directions respectively, and are principal point coordinates, and both of them together constitute camera extrinsic parameters.

[0080] In the case of distortion, distortion correction needs to be performed: The expression of the radial distortion coefficient is:

[0081]

[0082] The expression of the tangential distortion coefficient is:

[0083]

[0084] Wherein, , u and v are the coordinates of the feature points in the pose rectification coordinate data, and are the coordinates after rectification.

[0085] p1 and p2 are tangential distortion coefficients, and k1, k2 and k3 are radial distortion coefficients.

[0086] 404. A homography matrix is generated according to the intrinsic matrix and the extrinsic matrix, and the homography matrix is solved by the pose rectification coordinate data.

[0087] 405. Equations about each intrinsic parameter in the intrinsic matrix are listed according to the solved homography matrix, and the intrinsic matrix is solved by combining the perspective projection equation and the least square method.

[0088] 406. The extrinsic matrix is solved according to the solved homography matrix and the solved intrinsic matrix.

[0089] 407. Distortion coefficient expressions are solved according to the world coordinates in the world coordinate system and the pose rectification coordinate data, and distortion rectification coefficients are generated.

[0090] In this embodiment, first, the coordinates calculated by step 2 of each image are substituted into the calibration algorithm to obtain the coordinate information of the feature points under different poses (pose rectification coordinate data) in the image coordinate system and the world coordinate system. For each group of feature points, the following can be obtained according to the perspective projection equation:

[0091] wherein 、 and are column vectors of the rotation matrix R, and t is the translation vector T, is a scale factor (scaling ratio) of the i-th calibration photo, 、 are the pose rectified coordinate data corresponding to the i-th calibration photo, 、 are the world coordinates corresponding to the pose rectified coordinate data.

[0092] Let H is a 3x3 homography matrix, which can be expressed as:

[0093] The homography matrix can be solved by the feature point coordinates (pose rectified coordinate data). According to the relationship between H and the intrinsic matrix K, an equation about the intrinsic parameters can be listed. Combined with multiple homography matrices obtained from multiple images, the intrinsic matrix K is solved by using the least square method:

[0094] wherein n is the number of images, and m is the number of feature points of each image.

[0095] After the intrinsic parameters are solved, the extrinsic parameters R and T are calculated according to the following formula:

[0096]

[0097]

[0098]

[0099] The extrinsic parameters R and T are calculated, and the distortion coefficient is solved. Wherein 、 and are column vectors of H. By solving the equation group, the intrinsic parameters (such as focal length, principal point coordinates, distortion coefficient, etc.) and extrinsic parameters (such as rotation matrix, translation vector, etc.) of the camera are calculated, and the camera parameter calibration is completed.

[0100] Referring to Figure 5 , the present application provides an embodiment of a method for lighting a reference display screen, the reference display screen being a quantum dot electroluminescent display screen, comprising: 501. Turning on the system light source according to the subsequent detection items of the reference display screen.

[0101] 502、In each posture of the reference display screen, the acquisition points are determined according to the film thickness distribution of the reference display screen, and the ambient light intensity is detected at the acquisition points.

[0102] 503、When the ambient light intensity is less than the low-light environment threshold, the base brightness, the dynamic contrast coefficient and the adaptive Gamma value of the reference display screen are generated according to the ambient light intensity.

[0103] 504、The base brightness is adjusted according to the dynamic contrast coefficient and the adaptive Gamma value to generate the adjusted brightness.

[0104] 505、The reference display screen is lit with the calibration image, and the luminous intensity of the reference display screen is adjusted according to the adjusted brightness.

[0105] In this embodiment, the terminal first sets the system light source of the acquisition system according to the detection items of the subsequent reference display screen corresponding to the to-be-detected display screen, and then determines a plurality of acquisition points on the reference display screen according to the thickness distribution of the quantum dot electroluminescent display screen, each thickness is provided with an acquisition point, and the ambient light intensity is detected at the acquisition points to generate the average light intensity.

[0106] When the ambient light intensity L_env is less than the low-light environment threshold L_dark, the base brightness B_base is calculated: B_base=B_floor+(B_dark-B_floor)×(L_env / L_dark)^γ B_floor is the absolute minimum brightness, which is used to ensure basic visibility, B_dark is the low-light environment reference brightness, and γ is the Gamma coefficient of brightness adjustment. The dynamic contrast coefficient is calculated: C_enhance=C_max-(C_max-C_dark)×(L_env / L_dark)^k C_dark is the low-light environment contrast reference, C_max is the maximum contrast enhancement, k is the contrast adjustment rate coefficient, and the adaptive Gamma value is calculated: γ_display=γ_min+[log(1+L_env)×(γ_dark-γ_min)^g] / log(1+L_dark) γ_min is the minimum Gamma value, which is used to greatly improve the dark details, γ_dark is the low-light environment Gamma reference, γ_max is the normal environment Gamma value, and g is the Gamma adjustment rate coefficient.

[0107] The terminal inputs a calibration image to illuminate the reference display and reach the base brightness. Pixel-level display enhancement is then performed. For each RGB pixel (r, g, b) of the reference display, the dynamic contrast coefficient C_enhance is applied for enhancement: r_enhanced=128+C_enhance×(r-128) g_enhanced=128+C_enhance×(g-128) b_enhanced=128+C_enhance×(b-128) Next the terminal applies gamma correction: r_final=255×(r_enhanced / 255)^(1 / γ_display) g_final=255×(g_enhanced / 255)^(1 / γ_display) b_final=255×(b_enhanced / 255)^(1 / γ_display) After calculating r_final, g_final, and b_final, the reference display screen with the calibration image input can be re-brightened to achieve brightness adjustment. This method can actively adjust the brightness of the calibration plate, so that it can actively brighten the calibration plate in low light conditions, so that the calibration image can be clearly displayed on the quantum dot electroluminescent display screen.

[0108] See also Figure 6 The present application provides an embodiment of a method for adjusting the grayscale value of an effective area of ​​a display screen in a calibration photo, wherein the reference display screen is a quantum dot electroluminescent display screen, comprising: 601. Obtain a nominal light conversion film thickness of a reference display screen.

[0109] 602. Obtain an actual light conversion film thickness distribution of a reference display screen and an actual refractive index distribution corresponding to an actual quantum dot concentration.

[0110] In this embodiment, a quantum dot electroluminescent display screen is used. In essence, it uses pre-prepared quantum dot materials as the material of the light conversion film, which is placed between the blue LED backlight and the liquid crystal layer at the front end in a specific way. The blue backlight excites the quantum dot film, thereby emitting pure red and green light, which is mixed with the remaining blue light to obtain high-quality white light, which is then controlled by liquid crystal pixels. In the quantum dot electroluminescent display screen QLED in this embodiment, the quantum dot material of each pixel actively emits light when directly driven by electric current. This is similar to the working principle of traditional OLED, replacing the traditional light-emitting layer with quantum dots.

[0111] In the prior art, quantum dot electroluminescent display screens QLED are usually prepared using quantum dot ink. First, the synthesized quantum dot particles are separated from the original solvent and dispersed in a solvent with suitable physical properties (such as boiling point, surface tension, and viscosity) to form “quantum dot ink”. Then, film formation is performed by inkjet printing. Specifically, on a substrate on which a TFT circuit and an electrode (anode) have been prepared, pixel pits are prepared by a photolithography process, and then red, green, and blue quantum dot inks are accurately sprayed into corresponding pixel pits using a high-precision inkjet printing device. Finally, the solvent is volatilized by annealing (heating) to leave a uniform and flat quantum dot film. However, in the prior art, the evaporation speed of the ink is faster at the edge than at the center during the drying process, which causes the quantum dot particles to gather at the edge due to the effect of tension, forming a ring-shaped uneven film that is thin in the middle and thick at the edge. Even if the solvent and evaporation speed are adjusted, there will still be a small amount of ring-shaped uneven film. Before defect detection of this new type of display screen, the industrial camera also needs to be calibrated, and the characteristics used for this type of display screen have a light conversion film that is low inside and high outside in the pixel layer, uneven quantum dot concentration, and light source refraction. Therefore, the terminal needs to adjust the gray scale of the calibration plate area on the reference display screen according to the thickness of the light conversion film and the distribution of the quantum dots of the reference display screen to make the feature points and pose correction mark points on the calibration photo clearer.

[0112] The terminal first acquires the nominal light conversion film thickness of the reference display screen , i.e., the standard thickness of the reference display screen.

[0113] Then the terminal acquires the actual light conversion film thickness distribution of the reference display screen and the actual refractive index distribution corresponding to the actual quantum dot concentration .

[0114] 603、According to the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflection coefficient, the thickness adjustment factor of the reference display screen in each area is calculated.

[0115] The terminal calculates the thickness adjustment factor of the reference display screen in each area according to the nominal light conversion film thickness , the actual light conversion film thickness distribution , and the light conversion film reflection coefficient , and the formula is as follows:

[0116] Under the light conversion film thickness corresponding to different areas of the reference display screen, the light conversion film will reflect a part of the light, and the light conversion film reflection coefficient is used to adjust the increase in gray scale caused by the light conversion film thickness, To adjust the reflectivity of the light conversion film under the actual thickness distribution of the light conversion film corresponding to different areas of the reference display screen.

[0117] 604、Calculate the interface transmission adjustment factor according to the actual refractive index distribution.

[0118] In addition to the thickness that can cause refraction, the quantum dot concentration inhomogeneity can also cause additional light source refraction, at which time the terminal needs to generate an interface transmission adjustment factor according to the actual refractive index distribution, which is to directly determine the refractive index as the interface transmission adjustment factor.

[0119] 605、Adjust the gray value of the effective area of the display screen in the calibration photo according to the thickness adjustment factor and the interface transmission adjustment factor.

[0120] The terminal adjusts the gray value of the effective area of the display screen in the calibration photo according to the thickness adjustment factor and the interface transmission adjustment factor, and the formula is as follows:

[0121] For the gray of the adjusted calibration photo, For the gray of the calibration photo before adjustment, The interface transmission adjustment factor is adjusted by the above-mentioned method. The gray value of the calibration photo can solve the problem that the feature points and the posture correction mark points of the quantum dot electroluminescent display screen are not clear due to the uneven thickness of the light conversion film and the uneven concentration of the quantum dots, and improve the extraction effect of the feature points and the posture correction mark points.

[0122] Please refer to Figure 7 The present application provides an embodiment of a method for preprocessing a calibration photo, comprising: 701, filtering the calibration photo.

[0123] In this embodiment, filtering the calibration photo can effectively remove noise.

[0124] Please refer to Figure 8 The present application provides an embodiment of a camera calibration device based on a screen image, comprising: The design unit 801 is configured to design a calibration image for an industrial camera in a collection system according to screen parameters of a reference display screen. The collection system comprises an industrial camera, and the calibration image is provided with a plurality of feature points and posture correction mark points.

[0125] The lighting unit 802 is configured to light the reference display screen according to the environmental parameters of the collection system and the calibration image.

[0126] Optionally, the reference display screen is a quantum dot electroluminescent display screen. ​

[0127] The lighting unit 802 specifically includes: The acquisition system is turned on according to the subsequent detection items of the reference display screen.

[0128] In each posture of the reference display screen, the acquisition point is determined according to the thickness distribution of the reference display screen, and the environmental light intensity is detected at the acquisition point.

[0129] When the environmental light intensity is less than the low-light environment threshold, the basic brightness, the dynamic contrast coefficient and the adaptive Gamma value of the reference display screen are generated according to the environmental light intensity.

[0130] The basic brightness is adjusted according to the dynamic contrast coefficient and the adaptive Gamma value to generate the adjusted brightness.

[0131] The reference display screen is lit with a calibration image, and the luminous intensity of the reference display screen is adjusted according to the adjusted brightness.

[0132] The acquisition unit 803 is configured to acquire calibration photos of the reference display screen in different postures using an industrial camera.

[0133] The filtering unit 804 is configured to perform filtering processing on the calibration photos.

[0134] The first acquisition unit 805 is configured to acquire a nominal refractive index corresponding to a nominal light conversion film thickness and a nominal quantum dot concentration of the reference display screen.

[0135] The second acquisition unit 806 is configured to acquire an actual refractive index distribution corresponding to an actual light conversion film thickness distribution and an actual quantum dot concentration of the reference display screen.

[0136] The first calculation unit 807 is configured to calculate a thickness adjustment factor of the reference display screen in each region according to the nominal light conversion film thickness, the actual light conversion film thickness distribution and the light conversion film reflection coefficient.

[0137] The second calculation unit 808 is configured to calculate an interface transmission adjustment factor according to the nominal refractive index and the actual refractive index distribution.

[0138] The adjustment unit 809 is configured to adjust the gray value of the effective area of the display screen in the calibration photo according to the thickness adjustment factor and the interface transmission adjustment factor.

[0139] The extraction unit 810 is configured to extract feature points and posture correction mark points in the calibration photo.

[0140] Optionally, the extraction unit 810 specifically includes: The coordinate position and the gray value of each pixel point in each calibration photo are fitted by a second-order polynomial.

[0141] After the fitting is completed, a Hessian matrix of the quadratic polynomial is constructed.

[0142] According to the Hessian matrix, feature points on the calibration photo are extracted.

[0143] The posture correction mark points on the calibration photo are extracted through binarization processing and morphological processing.

[0144] The unit 811 is configured to construct a world coordinate system according to the reference display screen, and the world coordinate system includes a plurality of world coordinates.

[0145] The first generation unit 812 is configured to generate a posture correction matrix using the coordinate data of the posture correction mark points and the reference coordinate data in the horizontal alignment posture.

[0146] The second generation unit 813 is configured to perform posture correction processing on the coordinate data of the feature points using the posture correction matrix, and generate posture correction coordinate data.

[0147] Optionally, the second generation unit 813 specifically includes: The posture correction processing is performed on the coordinate data of the feature points through the posture correction matrix, and the coordinate data after the posture correction is obtained.

[0148] The coordinate data after the posture correction is sorted in the manner of row first and column second, to generate ordered coordinates corresponding to each feature point in the world coordinate system.

[0149] The ordered coordinates are left multiplied by the inverse matrix of the posture correction matrix, to generate the posture correction coordinate data.

[0150] The calibration unit 814 is configured to calibrate the industrial camera using the world coordinates in the world coordinate system and the posture correction coordinate data.

[0151] Optionally, the calibration unit 814 specifically includes: According to the equivalent focal length and the principal point coordinates in different axis directions, an intrinsic matrix of the industrial camera is constructed, and an expression of the distortion coefficient of the industrial camera is constructed.

[0152] According to the rotation matrix and the translation vector, an extrinsic matrix of the industrial camera is constructed.

[0153] According to the world coordinates in the world coordinate system, the posture correction coordinate data, the intrinsic matrix, the extrinsic matrix, and a scale factor of the industrial camera, a perspective projection equation of the industrial camera is constructed.

[0154] According to the intrinsic matrix and the extrinsic matrix, a homography matrix is generated, and the homography matrix is solved through the posture correction coordinate data.

[0155] According to the solved homography matrix, equations about each intrinsic parameter in the intrinsic matrix are listed, and the intrinsic matrix is solved by combining the perspective projection equation and the least square method.

[0156] According to the solved homography matrix and the solved intrinsic matrix, the extrinsic matrix is solved.

[0157] According to the world coordinates in the world coordinate system and the posture correction coordinate data, a distortion coefficient expression is solved, and a distortion correction coefficient is generated.

[0158] Referring to Figure 9 , the application provides a camera calibration device based on a screen image, comprising: The processor 901, the memory 902, the input / output unit 903 and the bus 904.

[0159] The processor 901 is connected with the memory 902, the input / output unit 903 and the bus 904.

[0160] The memory 902 stores a program, and the processor 901 calls the program to perform the camera calibration method in Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 .

[0161] The application provides a computer readable storage medium, and the computer readable storage medium stores a program, and the program performs the camera calibration method in Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 when executed on a computer.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0163] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0164] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0165] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0166] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially, or the part that makes a contribution to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various other media that can store program codes.

Claims

1. A camera calibration method based on screen image, characterized in that: include: Designing a calibration image for an industrial camera in an acquisition system based on screen parameters of a reference display screen, wherein the acquisition system includes an industrial camera, and the calibration image is provided with a plurality of feature points and posture correction mark points; Lighting up the reference display screen according to the environmental parameters of the acquisition system and the calibration image; Using the industrial camera to collect calibration photos of the reference display screen in different postures; Extracting feature points and posture correction mark points from the calibration photo; Constructing a world coordinate system according to the reference display screen, wherein the world coordinate system includes a plurality of world coordinates; Generate a posture correction matrix using the coordinate data of the posture correction mark point and the reference coordinate data in the horizontal alignment posture; Performing posture correction processing on the coordinate data of the feature points using the posture correction matrix to generate posture correction coordinate data; The industrial camera is calibrated using the world coordinates and attitude correction coordinate data in the world coordinate system.

2. The camera calibration method according to claim 1, wherein: The step of extracting feature points and posture correction mark points from the calibration photo includes: Perform quadratic polynomial fitting on the coordinate position and grayscale value of each pixel in the calibration photo; After the fitting is completed, the Hessian matrix of the quadratic polynomial is constructed; Extracting feature points on the calibration photo according to the Hessian matrix; The posture correction mark points on the calibration photo are extracted through binarization and morphological processing.

3. The camera calibration method according to claim 1, wherein: The step of performing posture correction processing on the coordinate data of the feature points using the posture correction matrix to generate posture correction coordinate data includes: Performing posture correction processing on the coordinate data of the feature points through a posture correction matrix to obtain the coordinate data after posture correction; Sorting the coordinate data after posture correction in a row-first and column-later manner to generate ordered coordinates corresponding to each feature point in the world coordinates; The ordered coordinates are left-multiplied by the inverse matrix of the posture correction matrix to generate posture correction coordinate data.

4. The camera calibration method according to claim 1, wherein: The step of calibrating the industrial camera using the world coordinates and attitude correction coordinate data in the world coordinate system includes: Constructing an intrinsic parameter matrix of the industrial camera according to the equivalent focal lengths and principal point coordinates in different axis directions, and constructing a distortion coefficient expression of the industrial camera; Constructing the extrinsic parameter matrix of the industrial camera according to the rotation matrix and translation vector; Constructing a perspective projection equation of the industrial camera according to the world coordinates in the world coordinate system, the posture correction coordinate data, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of the industrial camera; Generate a homography matrix according to the intrinsic parameter matrix and the extrinsic parameter matrix, and solve the homography matrix using the posture correction coordinate data; According to the solved homography matrix, equations for each internal parameter in the internal parameter matrix are listed, and the internal parameter matrix is ​​solved by combining the perspective projection equation and the least squares method; Solve the extrinsic parameter matrix based on the solved homography matrix and the solved intrinsic parameter matrix; A distortion coefficient expression is solved according to the world coordinates and the posture correction coordinate data in the world coordinate system to generate a distortion correction coefficient.

5. The camera calibration method according to any one of claims 1 to 4, characterized in that: The reference display screen is a quantum dot electroluminescent display screen; The step of lighting up the reference display screen according to the environmental parameters of the acquisition system and the calibration image comprises: Turn on the system light source of the acquisition system according to the subsequent detection items of the reference display screen; Under various postures of the reference display screen, determining a collection point according to the film thickness distribution of the reference display screen, and detecting the ambient light intensity at the collection point; When the ambient light intensity is less than a low light environment threshold, generating a basic brightness, a dynamic contrast ratio, and an adaptive gamma value of the reference display screen according to the ambient light intensity; Adjusting the basic brightness according to the dynamic contrast coefficient and the adaptive gamma value to generate an adjusted brightness; The reference display screen is caused to light up the calibration image, and the luminous intensity of the reference display screen is adjusted according to the adjusted brightness.

6. The camera calibration method according to any one of claims 1 to 4, characterized in that: The reference display screen is a quantum dot electroluminescent display screen; After the step of using the industrial camera to capture calibration photos of the reference display screen in different postures, and before the step of extracting feature points and posture correction mark points from the calibration photos, the camera calibration method further includes: Obtaining a nominal light conversion film thickness of the reference display screen; Obtaining an actual light conversion film thickness distribution and an actual refractive index distribution corresponding to an actual quantum dot concentration of the reference display screen; Calculating a thickness adjustment factor of each region of the reference display screen according to the nominal light conversion film thickness, the actual light conversion film thickness distribution, and the light conversion film reflectance; Calculating an interface transmission adjustment factor according to the actual refractive index distribution; The grayscale value of the effective area of ​​the display screen in the calibration photo is adjusted according to the thickness adjustment factor and the interface transmittance adjustment factor.

7. The camera calibration method according to any one of claims 1 to 4, characterized in that: After using the industrial camera to capture calibration photos of the reference display screen in different postures, and before the step of extracting feature point coordinate data and posture correction feature coordinate data from the calibration photos, the camera calibration method further includes: Filter the calibration photos.

8. A camera calibration device based on screen images, characterized in that: include: A design unit, configured to design a calibration image for an industrial camera in an acquisition system according to screen parameters of a reference display screen, wherein the acquisition system includes an industrial camera, and wherein a plurality of feature points and posture correction mark points are set on the calibration image; a lighting unit, configured to light up the reference display screen according to the environmental parameters of the acquisition system and the calibration image; an acquisition unit, configured to use the industrial camera to acquire calibration photos of the reference display screen in different postures; An extraction unit, configured to extract feature points and posture correction mark points from the calibration photo; A construction unit, configured to construct a world coordinate system according to the reference display screen, wherein the world coordinate system includes a plurality of world coordinates; a first generating unit, configured to generate a posture correction matrix using the coordinate data of the posture correction mark point and the reference coordinate data in the horizontal alignment posture; a second generating unit, configured to perform posture correction processing on the coordinate data of the feature points using the posture correction matrix to generate posture-corrected coordinate data; A calibration unit is used to calibrate the industrial camera using world coordinates and posture correction coordinate data in the world coordinate system.

9. The camera calibration device according to claim 8, characterized in that: The extraction unit specifically includes: Perform quadratic polynomial fitting on the coordinate position and grayscale value of each pixel in the calibration photo; After the fitting is completed, the Hessian matrix of the quadratic polynomial is constructed; Extracting feature points on the calibration photo according to the Hessian matrix; The posture correction mark points on the calibration photo are extracted through binarization and morphological processing.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the camera calibration method according to any one of claims 1 to 7 is executed.

Images