Calibration method and device of projection equipment, projection equipment and storage medium

Abstract

The invention discloses a calibration method and device of projection equipment, the projection equipment and a storage medium, and relates to the technical field of projection. The method comprises the following steps: projecting a target calibration pattern to a to-be-projected area through the ray machine; shooting the target calibration pattern displayed in the to-be-projected area through the camera to obtain a target calibration image; according to the target distance information and the target calibration image, a re-projection error value of an angular point in the target calibration image is acquired, and the target distance information comprises the distance between two adjacent feature points in the target calibration image displayed in the to-be-projected area; and optimizing camera calibration parameters of the camera by using a preset optimization algorithm according to the re-projection error value. Thus, calibration of the projection equipment can be completed without using professional equipment such as a calibration board, the repair probability of the projection equipment is greatly reduced, and meanwhile the repair efficiency of maintenance workers is improved.

Description

Technical Field

[0001] The present application relates to the field of projection technology, and in particular to a calibration method and device for a projection device, a projection device, and a storage medium. Background Art

[0002] Projection equipment generally includes components such as cameras and optical machines. The optical machine is used to project image information, and the camera is used to assist the optical machine in focusing, side projection and other functions. The combination of camera and optical machine can be regarded as a binocular system. In order to ensure the projection effect of the projection equipment, it is generally necessary to calibrate it before using it.

[0003] However, in the related art, the calibration process is rather cumbersome and requires the use of a professional calibration board. Especially during the use of the projection equipment, the structure of the projection equipment may become loose due to collision, disassembly, etc. Since the user cannot use the professional calibration board to recalibrate the projection equipment, the projection equipment needs to be returned to the factory for repair, resulting in an excessively high return rate for the projection equipment. Summary of the invention

[0004] The present application proposes a calibration method and device for a projection device, a projection device, and a storage medium, so as to quickly and easily calibrate the projection device and reduce the repair rate of the projection device.

[0005] In a first aspect, an embodiment of the present application provides a calibration method for a projection device, which is applied to the projection device, wherein the projection device includes a camera and an optical machine, and the method includes: projecting a target calibration pattern to an area to be projected by the optical machine; photographing the target calibration pattern displayed in the area to be projected by the camera to obtain a target calibration image; obtaining a reprojection error value of a corner point in the target calibration image according to the target distance information and the target calibration image, wherein the target distance information includes the distance between two adjacent feature points in the target calibration pattern displayed in the area to be projected; optimizing the camera calibration parameters of the camera by using a preset optimization algorithm and according to the reprojection error value.

[0006] In a second aspect, an embodiment of the present application provides a calibration device for a projection device, which is applied to the projection device, wherein the projection device includes a camera and an optical machine, and the device includes: an image projection module, which is used to project a target calibration pattern to an area to be projected through the optical machine; an image capture module, which is used to capture the target calibration pattern displayed in the area to be projected through the camera to obtain a target calibration image; an error acquisition module, which is used to acquire a reprojection error value of a corner point in the target calibration image according to the target distance information and the target calibration image, wherein the target distance information includes the distance between two adjacent feature points in the target calibration pattern displayed in the area to be projected; and a calibration parameter optimization module, which is used to optimize the camera calibration parameters of the camera using a preset optimization algorithm and according to the reprojection error value.

[0007] In a third aspect, an embodiment of the present application provides a projection device, comprising: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the above method.

[0009] In the solution provided by the present application, the target calibration pattern is projected to the area to be projected by the optical machine; the target calibration pattern displayed in the area to be projected is photographed by the camera to obtain a target calibration image; the reprojection error value of the corner point in the target calibration image is obtained according to the target distance information and the target calibration image, and the target distance information includes the distance between two adjacent feature points in the target calibration pattern displayed in the area to be projected; the camera calibration parameters of the camera are optimized by using a preset optimization algorithm and according to the reprojection error value. In this way, the camera calibration parameters of the camera can be optimized and calibrated with the help of the target calibration pattern projected by the optical machine of the projection device without using professional equipment such as a calibration plate; even if the structure of the projection device is loosened due to collision, disassembly, etc. during the use of the projection device, but there is no calibration plate, the user of the projection device can recalibrate the relevant calibration parameters of the projection device, which greatly reduces the probability of rework of the projection device and also improves the repair efficiency of maintenance workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic flow chart of a method for calibrating a projection device provided in an embodiment of the present application is shown.

[0012] Figure 2 A schematic diagram of a calibration scene of a calibration method for a projection device provided in an embodiment of the present application is shown.

[0013] Figure 3 A schematic diagram of a calibration pattern provided in an embodiment of the present application is shown.

[0014] Figure 4 A schematic diagram of a target calibration image taken by a camera provided in an embodiment of the present application is shown.

[0015] Figure 5 A schematic diagram of measuring target distance information provided by an embodiment of the present application is shown.

[0016] Figure 6 A schematic flow chart of a method for calibrating a projection device provided in another embodiment of the present application is shown.

[0017] Figure 7 A schematic diagram of the conversion of multiple coordinate systems provided by an embodiment of the present application is shown.

[0018] Figure 8 A schematic diagram of the conversion of camera corner points into optical-mechanical corner points provided by an embodiment of the present application is shown.

[0019] Fig. 9 It is a block diagram of a calibration device for a projection device provided according to an embodiment of the present application.

[0020] Fig.10 It is a block diagram of a projection device according to an embodiment of the present application for executing a calibration method for a projection device according to an embodiment of the present application.

[0021] Fig.11 It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing a calibration method for a projection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0023] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings of the present application, multiple operations that appear in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The sequence numbers of operations such as S110, S120, etc. are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or less operations, and these operations may be executed in sequence or in parallel. And, the terms "first", "second", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0024] The inventor proposes a calibration method, device, projection device and storage medium for a projection device. The calibration method for a projection device provided in an embodiment of the present application is described in detail below.

[0025] Please refer to Figure 1 , Figure 1 A schematic diagram of a calibration method for a projection device provided in an embodiment of the present application is applied to a projection device, wherein the projection device includes a camera and an optical machine. Figure 1 The calibration method of the projection device provided in the embodiment of the present application is described in detail. The calibration method of the projection device may include the following steps:

[0026] Step S110: projecting the target calibration pattern to the area to be projected through the optical machine.

[0027] In this embodiment, if Figure 2 As shown, the projection device includes a camera and an optical machine, and the target calibration pattern can be as follows Figure 2The black and white checkerboard pattern shown can, of course, be other types of patterns, which are not limited in this embodiment. The optical machine can project the target calibration pattern to the projection area, which can be any display area that can be used for projection, such as a wall, a projection curtain or a projection screen.

[0028] It is understandable that in practical applications, when the optical machine is projected onto the area to be projected, the shape of the original calibration pattern may be distorted, such as Figure 3 The display screen of the area to be projected before adjustment shows the trapezoidal shape of A1; however, the shape of the original calibration pattern is Figure 3 The rectangular shape of B1 in the imaging display area (ie, DMD display area) of the front light is adjusted. Obviously, if the projection device is calibrated directly based on the distorted calibration pattern displayed in the projection area, the calibration parameters finally optimized will be inaccurate.

[0029] Therefore, the original calibration pattern can be firstly projected to the area to be projected by the optical machine; then, according to the original calibration pattern displayed in the area to be projected and the target correction parameters, the original calibration pattern is subjected to side projection correction processing to obtain the target calibration pattern; and then the target calibration pattern obtained by the side projection correction processing is projected to the area to be projected. Among them, the side projection correction processing can be generally understood as anti-distortion correction processing; that is, the target calibration pattern can be obtained by pre-projecting the original calibration pattern in the imaging display area of ​​the optical machine to perform anti-distortion correction processing; thereby, the shape of the calibration pattern displayed by the optical machine on the display area to be projected is a rectangle with the same shape as the original calibration pattern. That is to say, after the original calibration pattern is subjected to anti-distortion correction processing, the target calibration pattern finally projected on the display area to be projected can be free of distortion. For example, still refer to Figure 3 , by adjusting the shape of the original calibration pattern in the imaging display area of ​​the optical machine (i.e., B1 shape) to Figure 3 The B2 shape in the figure means that the original calibration pattern is subjected to anti-distortion correction processing to be a trapezoidal shape shown in B2, so that the final display screen in the area to be projected presents an A2 shape, that is, a rectangular shape. Obviously, the shape of the calibration pattern on the display screen in the area to be projected (i.e., the A2 shape) is ultimately made the same as the shape of the original calibration pattern (i.e., the B1 shape). In other words, by anti-distortion correction processing on the original calibration pattern before projection, it is avoided that the original calibration pattern is distorted in the area to be projected, resulting in deviations in the subsequent parameter calibration of the projection device and other problems.

[0030] In some embodiments, the projection device has an automatic side projection calibration function. The projection device can use the automatic side projection calibration function to adjust the shape of the display screen projected to the projection area; that is, the aforementioned target correction parameter can be a correction parameter pre-set for the automatic side projection calibration function. In this way, the projection device can quickly calibrate the projected target calibration pattern using its own automatic side projection calibration function.

[0031] Optionally, in addition to this, the projection device may also have a manual side projection correction function, that is, provide a manual side projection correction function to the user. If the user observes that the display image of the projection device after automatic side projection calibration is still distorted, the manual side projection correction function can be used to perform fine-tuning so that the image displayed in the area to be projected is not distorted. Based on this, the aforementioned target correction parameters may also include correction parameters manually input by the user. Optionally, the projection device may be configured with a corresponding remote control. During the manual fine-tuning process, if the left and right sides of the target calibration pattern displayed in the area to be projected are not equal in length, the correction parameters may be input by adjusting the left or right button of the remote control so that the lengths of the two sides are equal. If the upper and lower sides of the target calibration pattern displayed in the area to be projected are not equal in length, the upper and lower sides may be made the same length by adjusting the upper or lower button of the remote control.

[0032] Step S120: photographing the target calibration pattern displayed in the to-be-projected area by the camera to obtain a target calibration image.

[0033] Further, when the shape of the target calibration pattern displayed in the area to be projected is adjusted to be the same as the shape of the original target calibration pattern, the camera is controlled to shoot the target calibration pattern displayed in the area to be projected, thereby obtaining a target calibration image. For example, the target calibration pattern is a black and white checkerboard pattern, and the captured target calibration image can be as follows: Figure 4 of the image.

[0034] Step S130: acquiring a reprojection error value of a corner point in the target calibration image according to the target distance information and the target calibration image, wherein the target distance information includes a distance between two adjacent feature points in the target calibration pattern displayed in the area to be projected.

[0035] Before step S130, the projection device also outputs a first prompt information, wherein the first prompt information is used to prompt the user to measure the distance between two target points in the target calibration pattern displayed in the projection area; and receives the distance information input based on the first prompt information as the target distance information. Figure 5 The distance between point A and point B in the target calibration pattern displayed in the projection area can be used as the above-mentioned target distance information.

[0036] Optionally, the projection device may broadcast the first prompt information by voice broadcasting to prompt the user to measure the distance between two target points in the target calibration pattern displayed in the area to be projected.

[0037] Optionally, the projection device is configured with a display touch screen, and the first prompt information can be displayed on the display touch screen to prompt the user to manually measure the distance between two target points in the target calibration pattern displayed in the area to be projected.

[0038] Optionally, the projection device may directly project the first prompt information to the area to be projected, and project the measurement position indication information (eg Figure 5 Two circular circles at the midpoints A and B) are placed on the target calibration pattern displayed in the area to be projected, thereby more intuitively prompting the user to manually measure the distance between the two target points indicated in the target calibration pattern displayed in the area to be projected.

[0039] Based on this, the user can input the measured distance information into the projection device; correspondingly, the projection device receives the distance information input by the user based on the first prompt information, and if the two target points are two adjacent feature points, the distance information is obtained as the target distance information; if the two target points are not two adjacent feature points, the distance between the two adjacent feature points in the target calibration pattern can be calculated based on the distance information as the target distance information. Furthermore, after obtaining the target distance information, the projection device can calculate the absolute positions of the two points in the world coordinate system in combination with the known pixel coordinate information of the corresponding two points in the original target calibration pattern, thereby obtaining the absolute positions of the key points on the target calibration pattern on the area to be projected. In other words, by prompting the user to manually measure and input the measured distance information, an absolute size scale information is provided for the subsequent calibration of the projection device. Furthermore, based on this, since the actual positions of the corner points of the target calibration pattern on the projection area in the world coordinate system are known, the corner points can be mapped in the target calibration image based on the target calibration parameters of the projection device and combined with the actual positions of the corner points, thereby obtaining the difference value between the actual position and the mapped position of the corner points in the target calibration image as the aforementioned reprojection error value.

[0040] Step S140: optimizing the camera calibration parameters of the camera according to the reprojection error value using a preset optimization algorithm.

[0041] It can be understood that the smaller the reprojection error value is, the more accurate the parameter value characterizing the camera calibration parameter is. Therefore, after obtaining the reprojection error value, the preset optimization algorithm can be used to iteratively optimize the parameter value of the target calibration parameter according to the reprojection error value until the preset optimization conditions are met, thereby obtaining the optimized target calibration parameter as the final calibration parameter of the projection device. Among them, the preset optimization conditions include at least: the reprojection error value is less than the first threshold, the reprojection error value no longer changes, and the number of iterative optimizations reaches at least one of the first threshold values. The first threshold value and the first threshold value are both values ​​that can be preset. The preset optimization algorithm can be the Levenberg-Marquardt method (LM algorithm). Of course, other optimization algorithms that can be used to optimize the calibration parameters are also within the scope of protection of the present application, and no examples are given here one by one.

[0042] In this embodiment, there is no need to use professional equipment such as calibration plates, and the camera calibration parameters of the camera can be optimized and calibrated with the help of the target calibration pattern projected by the optical machine of the projection device. In this way, even if the structure of the projection device is loosened due to collision, disassembly, etc. during the use of the projection device, but there is no calibration plate, the user of the projection device can recalibrate the relevant calibration parameters of the projection device, which greatly reduces the probability of the projection device being returned for repair, and also improves the repair efficiency of maintenance workers.

[0043] Please refer to Figure 6 , Figure 6 A schematic diagram of a calibration method for a projection device provided in another embodiment of the present application is applied to a projection device, wherein the projection device includes a camera and an optical machine. Figure 6 The calibration method of the projection device provided in the embodiment of the present application is described in detail. The calibration method of the projection device may include the following steps:

[0044] Step S210: projecting the target calibration pattern onto the area to be projected through the optical machine.

[0045] Step S220: photographing the target calibration pattern displayed in the area to be projected by the camera to obtain a target calibration image.

[0046] In this embodiment, the specific implementation of step S210 to step S220 can refer to the content of the above-mentioned embodiment, which will not be repeated here.

[0047] Step S230: Detecting corner points in the target calibration image.

[0048] In this embodiment, after capturing the target calibration image, the corner points in the target calibration image can be detected, wherein the detection algorithm for detecting the corner points includes but is not limited to a corner point detection algorithm based on edge features, a corner point detection algorithm based on a template, or a corner point detection algorithm based on brightness changes, etc. The algorithms that can be used for corner point detection are all within the protection scope of this application and will not be described in detail here.

[0049] Step S240: If the number of the detected corner points is greater than or equal to a first preset number, the reprojection error value of the corner points in the target calibration image is obtained according to the target distance information and the target calibration image, and the target distance information includes the distance between two adjacent feature points in the target calibration pattern displayed in the projection area.

[0050] Furthermore, since the subsequent calibration process needs to be calibrated based on a sufficient number of corner points, if the number of corner points is too small or no corner points are detected, the subsequent calibration process cannot be accurately performed. Based on this, it can be determined whether the number of detected corner points is greater than or equal to a first preset number; wherein the first preset number can be a preset value, such as 3. Of course, the value of the first preset number can also be adjusted according to the actual required calibration accuracy, which will not be described in detail in this embodiment.

[0051] Optionally, if the number of corner points is greater than or equal to a first preset number, the number of characterization corner points is sufficient, and a step of obtaining reprojection error values ​​of corner points in the target calibration image as reprojection error values ​​can be performed based on the target distance information and the target calibration image.

[0052] See also Figure 7 , Figure 7 A schematic diagram showing the conversion relationship between various coordinate systems.

[0053] In some embodiments, the camera calibration parameters at least include camera intrinsic parameters. Obtaining the reprojection error value in step S240 can be specifically implemented by the following steps: obtaining the pixel coordinate information of the corner point in the target calibration image as the first pixel coordinate information; converting the coordinate information of the corner point in the world coordinate system into the coordinate information in the camera coordinate system according to the target distance information and the coordinate conversion parameters between the camera coordinate system and the world coordinate system to obtain the first coordinate information, wherein the world coordinate system is a three-dimensional coordinate system established based on the area to be projected, and the camera coordinate system is a three-dimensional coordinate system established based on the camera; mapping the first coordinate information to the pixel coordinate information in the target calibration image according to the initial parameter value of the camera intrinsic parameter to obtain the second pixel coordinate information; determining the reprojection error value according to the coordinate difference between the first pixel coordinate information and the second pixel coordinate information. Among them, the coordinate conversion parameters can be the initial rotation parameters and the initial translation parameters estimated by the PNP algorithm, or they can be the standard rotation parameters and the standard translation parameters estimated in advance before the projection device leaves the factory.

[0054] Step S250: optimizing the camera calibration parameters of the camera according to the reprojection error value using a preset optimization algorithm.

[0055] In this manner, the preset optimization algorithm can be used to iteratively optimize the initial parameter value of the camera intrinsic parameter according to the reprojection error value; that is, only the camera intrinsic parameter of the camera of the projection device can be optimized. The specific implementation of parameter optimization can refer to the content of the aforementioned embodiment, which will not be repeated here.

[0056] Step S260: If the number of the detected corner points is less than the first preset number, a second prompt message is output, where the second prompt message is used to prompt an adjustment of the placement posture of the projection device.

[0057] Optionally, if the number of corner points is greater than or equal to a first preset number, it indicates that the number of corner points is too small, which will affect the subsequent calibration precision and accuracy. At this time, the projection device can output a second prompt message to prompt the user to adjust the placement posture of the projection device until the camera in the projection device after the posture is adjusted captures the target calibration image corresponding to the calibration pattern projected by the optical machine, and can extract corner points greater than or equal to the first preset number, and then execute the contents of the aforementioned steps S240 to S250, which will not be repeated here.

[0058] In other embodiments, considering the situations in practical applications such as replacing the optical machine module and affecting the structure and stress of the optical machine, the present application can also recalibrate the optical machine calibration parameters of the optical machine of the projection device. Among them, the optical machine calibration parameters include at least the optical machine intrinsic parameters. Specifically: obtain the corner points in the original calibration image corresponding to the target calibration pattern as the target corner points; according to the homography change relationship of the image before and after the side projection correction, convert the pixel coordinate information of the target corner points in the original calibration image into the pixel coordinate information in the target pixel coordinate system to obtain the third pixel coordinate information, and the target pixel coordinate system is a two-dimensional coordinate system established based on the imaging plane of the optical machine; please refer to Figure 8 , Figure 8 A schematic diagram of corner point conversion is shown, which converts the pixel coordinate information of the corner points in the original calibration image to the pixel coordinate information in the target pixel coordinate system through the homography change relationship. According to the target distance information and the coordinate conversion parameters between the optical-mechanical coordinate system and the world coordinate system, the coordinate information of the corner points of the area to be projected in the world coordinate system is converted to the coordinate information in the optical-mechanical coordinate system to obtain the second coordinate information, the world coordinate system is a three-dimensional coordinate system established based on the area to be projected, and the optical-mechanical coordinate system is a three-dimensional coordinate system established based on the optical machine; according to the initial parameter value of the optical-mechanical internal parameter, the second coordinate information is mapped to the pixel coordinate information in the target calibration image to obtain the fourth pixel coordinate information; according to the coordinate difference between the third pixel coordinate information and the fourth pixel coordinate information, the target reprojection error value is determined. Among them, the coordinate conversion parameters can be the initial rotation parameters and the initial translation parameters estimated by the PNP algorithm, or they can be the standard rotation parameters and the standard translation parameters estimated in advance before the projection equipment leaves the factory. In this manner, the preset optimization algorithm can be used to iteratively optimize the initial parameter values ​​of the optical-mechanical intrinsic parameters according to the target reprojection error value; that is, only the optical-mechanical intrinsic parameters of the optical machine of the projection device can be optimized.

[0059] In some other embodiments, considering that in actual applications, there are situations where disassembly, assembly, collision, etc. have no effect or little effect on the intrinsic parameters of the optical machine and camera in the projector, in this case, it is not necessary to calibrate the camera calibration parameters of the camera and the optical machine calibration parameters of the optical machine, and only the calibration of the relative calibration parameters between the camera and the optical machine is completed. Specifically: the pixel coordinate information of the corner point in the target calibration image is obtained as the fifth pixel coordinate information; according to the target distance information, the initial parameter value of the first coordinate conversion parameter between the camera coordinate system and the world coordinate system, and the camera intrinsic parameter, the coordinate information of the corner point in the world coordinate system is mapped to the pixel coordinate information in the target calibration image to obtain the sixth pixel coordinate information; according to the coordinate difference between the fifth pixel coordinate information and the sixth pixel coordinate information, the camera reprojection error value is determined; using the preset optimization algorithm, and according to the camera reprojection error value, the initial parameter value of the first coordinate conversion parameter between the camera coordinate system and the world coordinate system is optimized. According to the target distance information, the initial parameter value of the second coordinate conversion parameter between the optical-mechanical coordinate system and the world coordinate system, and the optical-mechanical internal parameter, the coordinate information of the corner point in the world coordinate system is mapped to the pixel coordinate information in the target pixel coordinate system to obtain the seventh pixel coordinate information; the pixel coordinate information of the corner point in the original calibration image where the target calibration pattern is located is obtained as the eighth pixel coordinate information; according to the homography change relationship of the image before and after the side projection correction, the eighth pixel coordinate information is converted into the coordinate information in the target pixel coordinate system to obtain the ninth pixel coordinate information; according to the coordinate difference between the seventh pixel coordinate information and the ninth pixel coordinate information, the optical-mechanical reprojection error value is determined; using the preset optimization algorithm and according to the optical-mechanical reprojection error value, the initial parameter value of the second coordinate conversion parameter between the optical-mechanical coordinate system and the world coordinate system is optimized. Finally, the parameter value of the relative posture parameter between the camera coordinate system and the world coordinate system obtained by the optimization and the target value of the second coordinate conversion parameter between the optical-mechanical coordinate system and the world coordinate system obtained by the optimization are calculated. It can be understood that when recalibrating the relative posture parameters between the camera and the optical machine, the parameter values ​​of the optical machine's intrinsic parameters and the camera's intrinsic parameters calibrated by the projection equipment factory can be directly used, without the need to calibrate the camera's intrinsic parameters and the optical machine's intrinsic parameters.

[0060] That is to say, in the process of optimizing the relative pose parameters between the camera and the optical machine, the relative pose parameters between the camera and the area to be projected, as well as the relative pose parameters between the optical machine and the area to be projected are optimized respectively. Since the optical machine and the camera are rigidly connected, the relative pose parameters between the optical machine and the camera can be further calculated based on the optimized relative pose parameters between the camera and the area to be projected, as well as the optimized relative pose parameters between the optical machine and the area to be projected. In other words, by optimizing the relative pose parameters between the camera and the area to be projected, as well as the relative pose parameters between the optical machine and the area to be projected, the calibration of the relative pose parameters between the optical machine and the camera can be indirectly realized.

[0061] In other embodiments, the relative calibration parameters between the camera and the optical machine can also be calibrated in the following manner: according to the target distance information and the coordinate conversion parameters between the optical machine coordinate system and the world coordinate system, the coordinate information of the corner point of the projection area in the world coordinate system is converted into the coordinate information in the optical machine coordinate system to obtain the third coordinate information, wherein the world coordinate system is a three-dimensional coordinate system established based on the projection area, and the optical machine coordinate system is a three-dimensional coordinate system established based on the optical machine. Further, according to the initial parameter value of the relative posture parameter between the camera and the optical machine, the third coordinate information is converted into the coordinate information in the camera coordinate system to obtain the fourth coordinate information, wherein the camera coordinate system is a three-dimensional coordinate system established based on the camera. According to the camera intrinsic parameter of the camera, the fourth coordinate information is mapped to the pixel coordinate information in the target calibration image to obtain the tenth pixel coordinate information; then, according to the coordinate difference between the tenth pixel coordinate information and the preset pixel coordinate information corresponding to the corner point, the reprojection error value is determined, and then the preset optimization algorithm is used, and according to the reprojection error value, the initial parameter value of the relative posture parameter between the camera and the optical machine is optimized. The preset pixel coordinate information corresponding to the corner point is extracted from the captured target calibration image.

[0062] In this embodiment, there is no need to use professional equipment such as calibration plates. With the help of the target calibration pattern projected by the optical machine of the projection device, the camera calibration parameters of the camera, the optical machine calibration parameters of the optical machine, and the relative calibration parameters between the camera and the optical machine can be optimized and calibrated; and the user can choose to optimize and calibrate any one or more of the above three calibration parameters according to the actual needs of the projection device, which improves the flexibility of parameter calibration. Obviously, even if the structure of the projection device is loose due to collision, disassembly, etc. during the use of the projection device, but there is no calibration plate, the user of the projection device can recalibrate one or more calibration parameters of the projection device, which greatly reduces the probability of the projection device being repaired, and also improves the repair efficiency of the maintenance workers, thereby ensuring the normal operation of the focus, side projection and other functions of the camera and optical machine in the projection device.

[0063] Please refer to Fig. 9 , which shows a structural block diagram of a calibration device 300 for a projection device provided by an embodiment of the present application. The device 300 may include: an image projection module 310, an image capture module 320, an error acquisition module 330 and a calibration parameter optimization module 340.

[0064] The image projection module 310 is used to project the target calibration pattern to the projection area through the optical machine.

[0065] The image capturing module 320 is used to capture the target calibration pattern displayed in the to-be-projected area through the camera to obtain a target calibration image.

[0066] The error acquisition module 330 is used to obtain the reprojection error value of the corner point in the target calibration image according to the target distance information and the target calibration image, wherein the target distance information includes the distance between two adjacent feature points in the target calibration pattern displayed in the projection area.

[0067] The calibration parameter optimization module 340 is used to optimize the camera calibration parameters of the camera according to the reprojection error value by using a preset optimization algorithm.

[0068] In some embodiments, the image projection module 310 may be specifically used to project the original calibration pattern to the area to be projected by the optical machine before the target calibration pattern is projected to the area to be projected by the optical machine. The calibration device 300 of the projection device may also include: a side projection correction module. The side projection correction module may be used to perform side projection correction processing on the original calibration pattern according to the original calibration pattern displayed in the area to be projected and the target correction parameter to obtain the target calibration pattern.

[0069] In some embodiments, the calibration device 300 of the projection device may further include: a prompt module, an information receiving module, and a distance acquisition module. The prompt module may be used to output a first prompt message before obtaining the reprojection error value of the corner point in the target calibration image according to the target distance information and the target calibration image, wherein the first prompt message is used to prompt the measurement of the distance between two target points in the target calibration pattern displayed in the area to be projected. The information receiving module may be used to receive distance information input based on the first prompt message. The distance acquisition module may be used to calculate the distance between two adjacent feature points in the target calibration pattern according to the distance information as the target distance information.

[0070] In some embodiments, the calibration device 300 of the projection device may further include: a corner point detection module. The corner point detection module may be used to detect corner points in the target calibration image before obtaining the reprojection error values ​​of the corner points in the target calibration image according to the target distance information and the target calibration image; if the number of the detected corner points is greater than or equal to a first preset number, the step of obtaining the reprojection error values ​​of the corner points in the target calibration image according to the target distance information and the target calibration image is performed; if the number of the detected corner points is less than the first preset number, a second prompt information is output, and the second prompt information is used to prompt the adjustment of the placement posture of the projection device.

[0071] In some embodiments, the camera calibration parameters include at least camera intrinsic parameters; the error acquisition module 330 can be specifically used to: obtain pixel coordinate information of the corner point in the target calibration image as first pixel coordinate information; convert the coordinate information of the corner point in the world coordinate system into coordinate information in the camera coordinate system according to the target distance information and the coordinate conversion parameters between the camera coordinate system and the world coordinate system, and obtain first coordinate information, the world coordinate system is a three-dimensional coordinate system established based on the projection area, and the camera coordinate system is a three-dimensional coordinate system established based on the camera; according to the initial parameter value of the camera intrinsic parameter, map the first coordinate information to pixel coordinate information in the target calibration image, and obtain second pixel coordinate information; determine the reprojection error value according to the coordinate difference between the first pixel coordinate information and the second pixel coordinate information.

[0072] In this manner, the calibration parameter optimization module 340 may be specifically configured to utilize the preset optimization algorithm and perform iterative optimization on the initial parameter value of the camera intrinsic parameter according to the reprojection error value.

[0073] In some other embodiments, the calibration parameter optimization module 340 may include: an optomechanical parameter optimization unit. The optomechanical parameter optimization unit may be specifically used to: obtain the corner points in the original calibration image corresponding to the target calibration pattern as the target corner points; convert the pixel coordinate information of the target corner points in the original calibration image into the pixel coordinate information in the target pixel coordinate system according to the homography change relationship of the image before and after the side projection correction, and obtain the third pixel coordinate information, wherein the target pixel coordinate system is a two-dimensional coordinate system established based on the imaging plane of the optomechanical machine; convert the coordinate information of the corner points of the area to be projected in the world coordinate system into the target pixel coordinate system according to the target distance information and the coordinate conversion parameters between the optomechanical coordinate system and the world coordinate system. The coordinate information in the optomechanical coordinate system is used to obtain second coordinate information, wherein the world coordinate system is a three-dimensional coordinate system established based on the area to be projected, and the optomechanical coordinate system is a three-dimensional coordinate system established based on the optomechanical system; according to the initial parameter value of the optomechanical intrinsic parameter, the second coordinate information is mapped to the pixel coordinate information in the target calibration image to obtain fourth pixel coordinate information; according to the coordinate difference between the third pixel coordinate information and the fourth pixel coordinate information, a target reprojection error value is determined; and using the preset optimization algorithm and according to the target reprojection error value, the initial parameter value of the optomechanical intrinsic parameter is iteratively optimized.

[0074] In other embodiments, the calibration parameter optimization module 340 may include: a relative posture parameter optimization unit. The relative posture parameter optimization unit may be specifically used to: obtain the pixel coordinate information of the corner point in the target calibration image as the fifth pixel coordinate information; map the coordinate information of the corner point in the world coordinate system to the pixel coordinate information in the target calibration image according to the target distance information, the initial parameter value of the first coordinate conversion parameter between the camera coordinate system and the world coordinate system, and the camera intrinsic parameter to obtain the sixth pixel coordinate information; determine the camera reprojection error value according to the coordinate difference between the fifth pixel coordinate information and the sixth pixel coordinate information; optimize the initial parameter value of the first coordinate conversion parameter between the camera coordinate system and the world coordinate system using the preset optimization algorithm and according to the camera reprojection error value; map the coordinate information of the corner point in the world coordinate system to the target pixel coordinate system according to the target distance information, the initial parameter value of the second coordinate conversion parameter between the optomechanical coordinate system and the world coordinate system, and the optomechanical intrinsic parameter. , and obtain seventh pixel coordinate information; obtain pixel coordinate information of the corner point in the original calibration image where the target calibration pattern is located as eighth pixel coordinate information; according to the homography change relationship of the image before and after the side projection correction, convert the eighth pixel coordinate information into coordinate information in the target pixel coordinate system to obtain ninth pixel coordinate information; determine the optical-mechanical reprojection error value according to the coordinate difference between the seventh pixel coordinate information and the ninth pixel coordinate information; use the preset optimization algorithm and optimize the initial parameter value of the second coordinate transformation parameter between the optical-mechanical coordinate system and the world coordinate system according to the optical-mechanical reprojection error value; use the target value of the first coordinate transformation parameter between the camera coordinate system and the world coordinate system obtained by the optimization and the target value of the second coordinate transformation parameter between the optical-mechanical coordinate system and the world coordinate system obtained by the optimization to calculate the parameter value of the relative posture parameter between the camera and the optical-mechanical system.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0076] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0077] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0078] In summary, the target calibration pattern is projected to the projection area by the optical machine; the target calibration pattern displayed in the projection area is photographed by the camera to obtain the target calibration image; the reprojection error value of the corner point in the target calibration image is obtained according to the target distance information and the target calibration image, and the target distance information includes the distance between two adjacent feature points in the target calibration pattern displayed in the projection area; the camera calibration parameters of the camera of the projection device are optimized by using a preset optimization algorithm and according to the reprojection error value. In this way, the camera calibration parameters of the camera can be completed with the help of the target calibration pattern projected by the optical machine of the projection device without using professional equipment such as calibration plates; and the present application can also optimize and calibrate the optical machine calibration parameters of the optical machine and the relative calibration parameters between the camera and the optical machine; even if the structure of the projection device is loose due to collision, disassembly, etc. during the use of the projection device, but there is no calibration plate, the user of the projection device can recalibrate the relevant calibration parameters of the projection device, which greatly reduces the probability of rework of the projection device and also improves the repair efficiency of maintenance workers. .

[0079] The following will be combined Fig.10 A projection device provided in this application is described.

[0080] Reference Fig.10 , Fig.10 A structural block diagram of a projection device 400 provided in an embodiment of the present application is shown. The above method provided in an embodiment of the present application can be executed by the projection device 400.

[0081] The projection device 400 in the embodiment of the present application may include one or more of the following components: a processor 401, a memory 402, and one or more applications, wherein the one or more applications may be stored in the memory 402 and configured to be executed by one or more processors 401, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0082] The processor 401 may include one or more processing cores. The processor 401 uses various interfaces and lines to connect various parts of the entire projection device 400, and executes various functions and processes data of the projection device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 402, and calling data stored in the memory 402. Optionally, the processor 401 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 401 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem can also be integrated into the processor 401 and implemented separately through a communication chip.

[0083] The memory 402 may include a random access memory (RAM) or a read-only memory (ROM). The memory 402 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as the various corresponding relationships described above) created by the projection device 400 during use.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0085] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or modules, which may be electrical, mechanical or other forms.

[0086] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0087] Please refer to Fig.11 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 500 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0088] The computer readable storage medium 500 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer readable storage medium 500 has storage space for program code 510 that performs any method step of the above method. These program codes can be read from or written to one or more computer program products. The program code 510 can be compressed, for example, in an appropriate form.

[0089] In some embodiments, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the projection device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the projection device performs the steps in the above-mentioned method embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for calibrating a projection device, It is characterized in that Applied to a projection device, the projection device includes a camera and an optical machine, and the method includes: Projecting the target calibration pattern onto the area to be projected by the optical machine; photographing the target calibration pattern displayed in the area to be projected by the camera to obtain a target calibration image; According to the target distance information and the target calibration image, obtaining a reprojection error value of a corner point in the target calibration image, wherein the target distance information includes a distance between two adjacent feature points in the target calibration pattern displayed in the area to be projected; The camera calibration parameters of the camera are optimized using a preset optimization algorithm and according to the reprojection error value.

2. The method according to claim 1, It is characterized in that Before projecting the target calibration pattern onto the to-be-projected area by the optical machine, the method includes: Projecting the original calibration pattern to the area to be projected through an optical machine; According to the original calibration pattern displayed in the to-be-projected area and the target correction parameters, the original calibration pattern is subjected to side projection correction processing to obtain the target calibration pattern.

3. The method according to claim 1, It is characterized in that Before acquiring the reprojection error value of the corner point in the target calibration image according to the target distance information and the target calibration image, the method further includes: Outputting first prompt information, where the first prompt information is used to prompt measuring the distance between two target points in the target calibration pattern displayed in the area to be projected; receiving distance information input based on the first prompt information; According to the distance information, the distance between two adjacent feature points in the target calibration pattern is calculated as the target distance information.

4. The method according to claim 1, It is characterized in that Before acquiring the reprojection error value of the corner point in the target calibration image according to the target distance information and the target calibration image, the method further includes: Detecting corner points in the target calibration image; If the number of the detected corner points is greater than or equal to a first preset number, performing the step of acquiring the reprojection error values ​​of the corner points in the target calibration image according to the target distance information and the target calibration image; If the number of the detected corner points is less than the first preset number, a second prompt information is output, where the second prompt information is used to prompt the user to adjust the placement position of the projection device.

5. The method according to claim 1, It is characterized in that The camera calibration parameters at least include camera intrinsic parameters; The obtaining, according to the target distance information and the target calibration image, a reprojection error value of a corner point in the target calibration image comprises: Obtaining pixel coordinate information of the corner point in the target calibration image as first pixel coordinate information; According to the target distance information and the coordinate conversion parameters between the camera coordinate system and the world coordinate system, the coordinate information of the corner point in the world coordinate system is converted into the coordinate information in the camera coordinate system to obtain the first coordinate information, wherein the world coordinate system is a three-dimensional coordinate system established based on the area to be projected, and the camera coordinate system is a three-dimensional coordinate system established based on the camera; According to the initial parameter value of the camera intrinsic parameter, the first coordinate information is mapped to the pixel coordinate information in the target calibration image to obtain the second pixel coordinate information; Determining the reprojection error value according to a coordinate difference between the first pixel coordinate information and the second pixel coordinate information; The using of a preset optimization algorithm and optimizing the target calibration parameters of the projection device according to the reprojection error value includes: The preset optimization algorithm is utilized, and according to the reprojection error value, the initial parameter value of the camera intrinsic parameter is iteratively optimized.

6. The method according to claim 1, It is characterized in that The method further comprises: Obtaining corner points in the original calibration image corresponding to the target calibration pattern as target corner points; According to the homography change relationship of the image before and after the side projection correction, the pixel coordinate information of the target corner point in the original calibration image is converted into the pixel coordinate information in the target pixel coordinate system to obtain the third pixel coordinate information, wherein the target pixel coordinate system is a two-dimensional coordinate system established based on the imaging plane of the optical machine; According to the target distance information and the coordinate conversion parameters between the optical-mechanical coordinate system and the world coordinate system, the coordinate information of the corner point of the area to be projected in the world coordinate system is converted into the coordinate information in the optical-mechanical coordinate system to obtain the second coordinate information, wherein the world coordinate system is a three-dimensional coordinate system established based on the area to be projected, and the optical-mechanical coordinate system is a three-dimensional coordinate system established based on the optical machine; According to the initial parameter value of the optical-mechanical intrinsic parameter, mapping the second coordinate information to pixel coordinate information in the target calibration image to obtain fourth pixel coordinate information; Determine a target reprojection error value according to a coordinate difference between the third pixel coordinate information and the fourth pixel coordinate information; The preset optimization algorithm is utilized, and according to the target reprojection error value, the initial parameter value of the optical-mechanical intrinsic parameter is iteratively optimized.

7. The method according to any one of claims 1 to 6, It is characterized in that The method further comprises: Obtaining pixel coordinate information of the corner point in the target calibration image as fifth pixel coordinate information; Mapping the coordinate information of the corner point in the world coordinate system to pixel coordinate information in the target calibration image according to the target distance information, the initial parameter value of the first coordinate conversion parameter between the camera coordinate system and the world coordinate system, and the camera intrinsic parameter to obtain sixth pixel coordinate information; Determining the camera reprojection error value according to a coordinate difference between the fifth pixel coordinate information and the sixth pixel coordinate information; Optimizing the initial parameter value of the first coordinate transformation parameter between the camera coordinate system and the world coordinate system by using the preset optimization algorithm and according to the camera reprojection error value; According to the target distance information, the initial parameter value of the second coordinate conversion parameter between the optical-mechanical coordinate system and the world coordinate system, and the optical-mechanical internal parameter, the coordinate information of the corner point in the world coordinate system is mapped to the pixel coordinate information in the target pixel coordinate system to obtain seventh pixel coordinate information; Obtaining pixel coordinate information of the corner point in the original calibration image where the target calibration pattern is located as eighth pixel coordinate information; According to the homography change relationship of the image before and after the side projection correction, the eighth pixel coordinate information is converted into coordinate information in the target pixel coordinate system to obtain ninth pixel coordinate information; Determining the optical-mechanical reprojection error value according to a coordinate difference between the seventh pixel coordinate information and the ninth pixel coordinate information; Optimizing the initial parameter value of the second coordinate transformation parameter between the optical-mechanical coordinate system and the world coordinate system by using the preset optimization algorithm and according to the optical-mechanical reprojection error value; The parameter values ​​of the relative posture parameters between the camera and the optomechanical system are calculated using the target values ​​of the first coordinate transformation parameters between the camera coordinate system and the world coordinate system obtained by the optimization and the target values ​​of the second coordinate transformation parameters between the optomechanical coordinate system and the world coordinate system obtained by the optimization.

8. A calibration device for a projection device, It is characterized in that Applied to a projection device, the projection device includes a camera and an optical machine, and the device includes: An image projection module, used to project the target calibration pattern onto the area to be projected through the optical machine; An image capturing module, used for capturing the target calibration pattern displayed in the to-be-projected area through the camera to obtain a target calibration image; an error acquisition module, configured to acquire a reprojection error value of a corner point in the target calibration image according to the target distance information and the target calibration image, wherein the target distance information includes a distance between two adjacent feature points in the target calibration pattern displayed in the area to be projected; The calibration parameter optimization module is used to optimize the camera calibration parameters of the camera according to the reprojection error value by using a preset optimization algorithm.

9. A projection device, It is characterized in that The projection device comprises: Memory; One or more processors coupled to the memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the method as claimed in any one of claims 1 to 7.

Images