Geometric correction method and device for projection picture, equipment and medium thereof

Abstract

The invention discloses a geometric correction method and device for a projection picture, equipment and a medium. The method comprises the following steps: acquiring a preset projection picture set, playing the projection picture set by using a plurality of projectors according to a preset projection sequence, and acquiring camera picture sets matched with the projectors through cameras at fixed positions; calculating an image splitting coefficient and a geometric deformation parameter of each projector according to each camera picture set; and obtaining a to-be-processed image, and processing the to-be-processed image according to the image splitting coefficient and the geometric deformation parameter of each projector to obtain a geometrically corrected target projection image. Through the technical scheme of the invention, geometric correction of the projection picture can be realized, the efficiency of geometric correction operation of the projection picture, the correction precision and the accuracy of a correction result are improved, and the difficulty of geometric correction operation is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, device, equipment and medium for geometric correction of a projection image. Background Art

[0002] In today's multimedia display field, multi-channel projection display systems have been increasingly widely used due to their advantages of providing large size, high resolution and immersive visual experience. In multi-channel projection display systems, geometric correction technology plays a vital role. Its core purpose is to achieve precise geometric alignment of images displayed by multiple projectors to ensure the integrity and coherence of the entire projection image.

[0003] As far as the common geometric correction methods are concerned, they are mainly divided into the following two types. The first is the manual adjustment method. Specifically, it is to manually adjust the geometric grid parameters of each projection screen to achieve geometric alignment of adjacent screens. However, in actual operation, this manual geometric correction has many obvious defects. First, it needs to adjust the geometric deformation parameters of each projection screen one by one, which is an extremely cumbersome and time-consuming process. Especially when facing a large projection area, such as large conference halls, theaters and other places, or complex surfaces, such as curved and uneven display surfaces, it is often necessary to make repeated fine adjustments to achieve the basic alignment effect. Secondly, since the entire adjustment process is completely dependent on manual operation, the correction result is easily affected by personal experience and technical level. Different operators may have different understanding and grasp of the parameters, resulting in uneven final alignment accuracy, and even obvious deviations, which seriously affects the overall visual effect of the multi-channel projection display system. The second is the camera-based geometric correction method. The principle of this method is to obtain the mapping relationship between the point coordinates of the display screen and the point coordinates on each projector image, and then perform corresponding geometric deformation on the image. However, it is not easy to achieve this process. First, a series of marking points need to be carefully and reasonably arranged on the display screen, and then professional measuring equipment and technology are used to accurately measure and record the spatial coordinates of each marking point. The entire operation process not only takes a lot of time, but also requires extremely high measurement accuracy. In order to ensure that the final calculated mapping relationship is accurate and accurate, so as to achieve precise geometric correction, advanced and accurate measurement tools must be equipped, and experienced and skilled professionals are also required to participate. But even so, in actual application scenarios, this method still faces many difficult-to-overcome challenges, such as interference from ambient light and accumulation of measurement errors, which to a certain extent limits its wide application and promotion.

[0004] In summary, the existing geometric correction methods for projection images have the problems of low efficiency, low precision and complex correction operation. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for geometric correction of a projection image, which can solve the problems of low efficiency, low precision and complex correction operation caused by the existing method for geometric correction of projection images.

[0006] In a first aspect, an embodiment of the present invention provides a method for geometric correction of a projected image, the method comprising:

[0007] Obtaining a preset projection picture set, using multiple projectors to play the projection picture set according to a preset projection order, and acquiring camera picture sets respectively matched with each projector through cameras at fixed positions;

[0008] The image splitting coefficient and geometric deformation parameter of each projector are calculated based on the image sets of each camera;

[0009] An image to be processed is acquired, and the image to be processed is processed according to the image splitting coefficient and the geometric deformation parameter of each projector to obtain a target projection image after geometric correction.

[0010] In a second aspect, an embodiment of the present invention provides a geometric correction device for a projection image, the device comprising:

[0011] A camera image acquisition module is used to acquire a preset projection image set, use multiple projectors to play the projection image set according to a preset projection order, and acquire camera image sets that match each projector respectively through cameras at fixed positions;

[0012] A data calculation module is used to calculate the image splitting coefficient and geometric deformation parameters of each projector according to each camera image set;

[0013] The image correction module is used to obtain the image to be processed, and process the image to be processed according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for geometric correction of a projection image as described in any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for geometric correction of a projection image as described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention obtains a preset projection picture set, uses multiple projectors to play the projection picture set according to a preset projection order, and obtains each camera picture set matching each projector respectively through camera acquisition at a fixed position, then calculates the image splitting coefficient and geometric deformation parameter of each projector according to each camera picture set, finally obtains the image to be processed, and processes the image to be processed according to the image splitting coefficient and geometric deformation parameter of each projector to obtain the target projection image after geometric correction, which solves the problems of low efficiency, low precision and complex correction operation caused by the existing geometric correction method of projection pictures, realizes the geometric correction of projection pictures, improves the efficiency of the geometric correction operation of projection pictures, the correction precision and the accuracy of the correction result, and reduces the difficulty of the geometric correction operation.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a method for geometric correction of a projection image provided according to Embodiment 1 of the present invention;

[0023] Figure 2 is a flow chart of a method for geometric correction of a projection image provided according to Embodiment 2 of the present invention;

[0024] Figure 3 is a schematic structural diagram of a geometric correction device for a projection image provided according to a third embodiment of the present invention;

[0025] Figure 4 It is a structural schematic diagram of an electronic device for implementing a method for geometric correction of a projection image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variation of the terms "including" and "having" is intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment 1

[0029] Figure 1 A flowchart of a method for geometric correction of a projection image provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of performing geometric correction on multiple projection images. The method can be executed by a geometric correction device for projection images. The geometric correction device for projection images can be implemented in the form of hardware and / or software. The geometric correction device for projection images can be configured in a terminal or server with a geometric correction function for projection images.

[0030] like Figure 1 As shown, the method includes:

[0031] S110, obtaining a preset projection picture set, using multiple projectors to play the projection picture set according to a preset projection order, and acquiring camera picture sets respectively matched with each projector through cameras at fixed positions.

[0032] The projection image set is a set of pre-set images for achieving a specific projection effect or completing a calibration task. For example, in a calibration process for a multi-channel projection display system, the projection image set may include different grayscale images for testing brightness uniformity, pure color images for detecting color accuracy, etc.

[0033] Among them, a preset projection picture set is obtained, multiple projectors are used to play the projection picture set according to a preset playback order, and each camera picture set matching each projector is obtained by collecting through a camera at a fixed position, including: obtaining the projection picture set, the projection picture set including a pure white image and each structured light image matching the pure white image; making each projector play the projection picture set in sequence according to a preset projector projection order; in the process of the projector playing each projection picture in the projection picture set, a camera at a fixed position is used to collect the projection result of each projection picture to obtain a camera picture set matching the projector; wherein the number of camera pictures contained in the camera picture set is equal to the number of projection pictures in the projection picture set, including a standard camera image matching the pure white image and a target light image matching each structured light image, and the number of the camera picture sets is equal to the number of the projectors.

[0034] Specifically, in this embodiment, each structured light image is a light pattern image matching the pure white image generated according to the three-frequency four-step method. The three frequencies refer to three specific frequency combinations used to generate a pure white image. These three frequencies may be specific value ranges on the wavelength of light or other related physical quantities, such as the frequencies of three specific wavelengths of light. For example, they may be the frequencies corresponding to red light, green light, and blue light. The effect of a pure white image is achieved by a reasonable combination of these three frequencies of light. The four-step rule represents four specific steps for generating such a pure white image result light image. The first step is to determine the required three-frequency parameters, the second step is to perform preliminary modulation and combination of frequencies, the third step is to optimize and adjust the generated preliminary image, and the fourth step is to perform quality inspection and evaluation on the final generated pure white image result light image to ensure that it meets the expected pure white standard and quality requirements. Through the three-frequency four-step method, the required pure white image result light image can be generated more accurately and efficiently to meet specific application requirements.

[0035] Further, in this embodiment, after obtaining the preset projection picture set, each projector plays the projection picture set in sequence according to the preset projector projection order. In the process of the projector playing each projection picture in the projection picture set, a camera fixed at a specific position is used to collect the projection results of each projection picture, so as to obtain a camera picture set matching the projector. It should be noted that the number of camera pictures contained in the camera picture set here is equal to the number of projection pictures in the projection picture set, including a standard camera image matching the pure white image and a target light image matching each structured light image. And the number of camera picture sets is equal to the number of projectors. For example, assuming that there are three projectors and the projection picture set can only contain 25 projection pictures, when a certain projector is projecting a certain projection picture, the camera will capture the current situation of the projection screen corresponding to the projector as the projection result matching the projection picture, and finally three corresponding camera picture sets will be obtained, each camera picture set containing 25 projection results. In this way, a rich and accurate data basis can be provided for subsequent geometric correction.

[0036] S120, calculating the image splitting coefficient and geometric deformation parameter of each projector according to each camera image set.

[0037] Among them, the image splitting coefficient is a quantitative indicator used to determine the image splitting method and degree during the image processing process. For example, in this embodiment, the image splitting coefficient can be expressed as the number of pixels to be split per row or column. For example, the image splitting coefficient may stipulate that each row is split into units of 100 pixels, or image segmentation is performed when the brightness difference exceeds 50.

[0038] Furthermore, the geometric deformation parameters are related values ​​used to describe and quantify changes in the geometric shape of an image or object, including translation, rotation angle, scaling ratio, and distortion degree.

[0039] S130, acquiring an image to be processed, and processing the image to be processed according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction.

[0040] Among them, the image to be processed is processed according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction, including: obtaining the image splitting coefficient of the projector, and splitting the image to be processed based on the image splitting coefficient to obtain an image splitting result matching the projector; obtaining the geometric deformation parameters of the projector, using a thin plate spline interpolation method to perform a geometric deformation operation on the image splitting result of the projector based on the geometric deformation parameters to obtain a geometrically deformed image matching the projector; using each projector to project the geometrically deformed image matching the current projector to obtain a target projection image matching the image to be processed.

[0041] The thin plate spline interpolation method is a mathematical interpolation method used to estimate the value of an unknown position based on known discrete data points to achieve smooth interpolation and surface fitting of data. The thin plate means that mathematically, the surface fitted by this method has physical properties similar to those of a bent thin plate. Thin plate spline interpolation is achieved by minimizing an energy-based function that comprehensively considers the position of the data points and the degree of curvature of the interpolation function.

[0042] In a specific implementation scenario of this embodiment, exemplarily, the image splitting coefficient of the projector is obtained, and the image to be processed is split based on this coefficient to obtain an image splitting result that matches the projector. For example, for an image to be processed of 800x600 pixels, if the image splitting coefficient stipulates that each line is split in units of 200 pixels, then a splitting result of 3 lines will be obtained. Then the geometric deformation parameters of the projector are obtained, and the image splitting result of the projector is geometrically deformed based on these parameters using the thin plate spline interpolation method to obtain a geometrically deformed image that matches the projector. For example, if the geometric deformation parameters are a horizontal translation of 50 pixels and a clockwise rotation of 30 degrees, then the processed image will change position and angle accordingly. Finally, each projector is used to project the geometrically deformed image that matches the current projector, and the final projection result obtained is a target projection image that matches the image to be processed.

[0043] It should be noted that, in this embodiment, since there are multiple projectors that simultaneously project the geometrically deformed images of the image to be processed, the target projection image finally obtained is a result of superimposing the multiple geometrically deformed images.

[0044] The technical solution of the embodiment of the present invention obtains a preset projection picture set, uses multiple projectors to play the projection picture set according to a preset projection order, and obtains each camera picture set matching each projector respectively through a camera acquisition at a fixed position, then calculates the image splitting coefficient and geometric deformation parameters of each projector according to each camera picture set, finally obtains a to-be-processed image, and processes the to-be-processed image according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction, thereby realizing geometric correction of the projection picture, improving the efficiency of the projection picture geometric correction operation, the correction precision and the accuracy of the correction result, and reducing the difficulty of the geometric correction operation.

[0045] Embodiment 2

[0046] Figure 2 A flowchart of a method for geometric correction of a projected image provided in Embodiment 2 of the present invention. This embodiment is refined based on the above-mentioned embodiment. In this embodiment, the method for calculating the image splitting coefficient and geometric deformation parameters of each projector based on each camera image set is specifically refined.

[0047] like Figure 2 As shown, the method includes:

[0048] S210, obtaining a preset projection picture set, using multiple projectors to play the projection picture set according to a preset projection order, and acquiring camera picture sets respectively matched with each projector through cameras at fixed positions.

[0049] S220, calculating the image splitting coefficient and the maximum projection area of ​​each projector according to each camera picture set, and calculating the image mapping relationship between each projector and the camera based on the projection picture set and each camera picture set matched with each projector.

[0050] Among them, the image splitting coefficient and the maximum projection area of ​​each projector are calculated according to each camera picture set, including: obtaining each camera picture in the camera picture set, calculating the effective display area of ​​each camera picture respectively, and calculating the geometric splitting coefficient of the projector according to the position of the effective display area of ​​each camera picture in the camera picture; calculating the maximum inscribed rectangle of each camera picture according to the effective display area of ​​each camera picture, and adjusting the size of the maximum inscribed rectangle of each camera picture based on a preset size adjustment rule to obtain target inscribed rectangles of uniform size; merging the target inscribed rectangles, and obtaining the maximum effective display area according to the merging result.

[0051] Among them, the "effective display area" refers to the area in the camera image that can display the projected content normally and clearly, such as the actual effective projection imaging area in a projection image taken by a camera that is not affected by shadows, blur or other interference factors. The maximum inscribed rectangle refers to the largest rectangle that can be contained inside an irregular figure, such as the largest rectangular part that can be found in an irregular effective display area.

[0052] Specifically, first obtain each camera image in the camera image set, calculate the effective display area of ​​each camera image respectively, and calculate the geometric splitting coefficient of the projector according to the position of the effective display area of ​​each camera image in the camera image. For example, by analyzing multiple camera images, it is found that the effective display area is mainly concentrated in the upper left corner and the lower right corner of the image, then the geometric splitting coefficient may determine the corresponding segmentation method accordingly, and then calculate the maximum inscribed rectangle of each camera image according to the effective display area of ​​each camera image, and adjust the size of the maximum inscribed rectangle of each camera image based on the preset size adjustment rule to obtain target inscribed rectangles of uniform size. For example, the original maximum inscribed rectangles in different camera images are of different sizes. Through the size adjustment rule, they are all adjusted to a uniform size of 800x600. Finally, the target inscribed rectangles are merged, and the maximum effective display area is obtained according to the merged result.

[0053] Furthermore, based on the projection picture set and the camera picture sets respectively matched with the projectors, the image mapping relationship between the projectors and the cameras is calculated, including: obtaining each structured light image in the projection picture set, and performing phase unwrapping processing on each structured light image to obtain a standard phase unwrapping map respectively matched with each structured light image; obtaining a camera picture set matched with the projector, and performing phase unwrapping processing on each target light image in the camera picture set to obtain a target phase unwrapping map matched with each target light image; and calculating the image mapping relationship between the projector and the camera based on each standard phase unwrapping map and each target phase unwrapping map.

[0054] Among them, the phase unwrapping is a processing method used to restore the wrapped or truncated phase information in optical measurement or image processing; further, the image mapping relationship between the projector and the camera is calculated based on each standard phase unwrapping image and each target phase unwrapping image. Specifically, by comparing the position difference of the same feature point in the standard phase unwrapping image and the target phase unwrapping image, the mapping relationship parameters such as the scaling ratio and rotation angle between the projector image and the camera image are determined. It should be noted that the method for calculating the mapping relationship of multiple phase images is a mature prior art, which will not be described in detail in this embodiment.

[0055] S230 , based on the maximum projection area, the camera picture sets respectively matched with the projectors, and the image mapping relationship between the projectors and the cameras, respectively calculate and obtain the geometric deformation parameters of the projectors.

[0056] Among them, based on the maximum projection area, the camera picture sets respectively matched with each projector, and the image mapping relationship between each projector and the camera, the geometric deformation parameters of each projector are calculated respectively, including: generating multiple grid points at the maximum projection area based on a preset grid point generation rule, and establishing a coordinate system with the center point of the maximum projection area as the origin; obtaining the grid coordinates of each grid point in the coordinate system according to the coordinate system, and obtaining the starting parameters and the ending parameters of the maximum inscribed rectangle of the projector; using the starting parameters and the ending parameters to normalize the grid coordinates of each grid point to obtain the normalized coordinates matching each grid point; based on the image mapping relationship between the projector and the camera, respectively calculating the target coordinates corresponding to each grid point, wherein the normalized coordinates are input as camera coordinates; the normalized coordinates of each grid point and the target coordinates are aggregated and stored as the geometric deformation parameters of the projector.

[0057] Among them, the grid points are discrete points distributed according to certain rules in a specified area. In this embodiment, they can be specifically points uniformly distributed at fixed intervals in the maximum projection area; further, the normalization process is to map the data to a specific range or standard according to a certain rule to eliminate the difference in dimension or numerical range, so as to facilitate unified processing and comparison. For example, the coordinate value is mapped to the interval of 0 to 1. In this embodiment, the normalization operation can be specifically to normalize the coordinates of the upper left corner pixel point of the maximum projection area to (0,0), and the coordinate position of the lower right corner pixel of the rectangle to (1,1), and then obtain the normalized coordinates matching each grid point based on the above settings.

[0058] Specifically, multiple grid points are generated at the maximum projection area based on a preset grid point generation rule, and a coordinate system is established with the center point of the maximum projection area as the origin. For example, in the maximum projection area of ​​a rectangle, a grid point is generated every 10 units, and a rectangular coordinate system is established with the center of the area as the origin. According to the coordinate system, the grid coordinates of each grid point in the coordinate system are obtained, and the starting parameters (such as the coordinates of the starting point) and the ending parameters (such as the coordinates of the ending point) of the maximum inscribed rectangle of the projector are obtained. The grid coordinates of each grid point are normalized using the starting parameters and the ending parameters to obtain the normalized coordinates matching each grid point, and then the target coordinates corresponding to each grid point are calculated based on the image mapping relationship between the projector and the camera, wherein the normalized coordinates are input as the camera coordinates, and the calculation results calculated by the image mapping relationship are the target coordinates. Finally, the normalized coordinates of each grid point and the target coordinates are aggregated and stored as the geometric deformation parameters of the projector. Through such a calculation process, the key parameters for correcting the geometric deformation of the projector can be accurately obtained, thereby improving the accuracy and consistency of the projection effect.

[0059] S240, acquiring an image to be processed, and processing the image to be processed according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction.

[0060] The technical solution of the embodiment of the present invention obtains a preset projection picture set, uses multiple projectors to play the projection picture set according to a preset projection order, and obtains each camera picture set matching each projector through a camera acquisition at a fixed position, then calculates the image splitting coefficient and the maximum projection area of ​​each projector according to each camera picture set, and calculates the image mapping relationship between each projector and the camera based on the projection picture set and the camera picture sets matching each projector, then calculates the geometric deformation parameters of each projector based on the maximum projection area, the camera picture sets matching each projector, and the image mapping relationship between each projector and the camera, finally obtains the image to be processed, and processes the image to be processed according to the image splitting coefficient and the geometric deformation parameters of each projector to obtain the target projection image after geometric correction, thereby realizing the geometric correction of the projection picture, improving the efficiency of the geometric correction operation of the projection picture, the correction precision and the accuracy of the correction result, and reducing the difficulty of the geometric correction operation.

[0061] Embodiment 3

[0062] Figure 3 This is a schematic diagram of the structure of a geometric correction device for a projection image provided by Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0063] The camera picture acquisition module 310 is used to acquire a preset projection picture set, use multiple projectors to play the projection picture set according to a preset projection order, and acquire each camera picture set matching each projector respectively through a camera at a fixed position;

[0064] A data calculation module 320 is used to calculate the image splitting coefficient and geometric deformation parameters of each projector according to each camera image set;

[0065] The image correction module 330 is used to obtain the image to be processed, and process the image to be processed according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction.

[0066] The technical solution of the embodiment of the present invention obtains a preset projection picture set, uses multiple projectors to play the projection picture set according to a preset projection order, and obtains each camera picture set matching each projector respectively through a camera acquisition at a fixed position, then calculates the image splitting coefficient and geometric deformation parameters of each projector according to each camera picture set, finally obtains a to-be-processed image, and processes the to-be-processed image according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction, thereby realizing geometric correction of the projection picture, improving the efficiency of the projection picture geometric correction operation, the correction precision and the accuracy of the correction result, and reducing the difficulty of the geometric correction operation.

[0067] Based on the above embodiment, the camera image acquisition module 310 includes:

[0068] An image acquisition unit, configured to acquire the projection image set, wherein the projection image set includes a pure white image and structured light images matching the pure white image;

[0069] An image playing unit, used to make each projector play the projection picture set in sequence according to a preset projection order of the projectors;

[0070] The projection result acquisition unit is used to collect the projection results of each projection picture using a camera at a fixed position during the process of the projector playing each projection picture in the projection picture set, so as to obtain a camera picture set matching the projector.

[0071] Based on the above embodiment, the data calculation module 320 includes:

[0072] A mapping calculation unit, used to calculate the image splitting coefficient and the maximum projection area of ​​each projector according to each camera picture set, and calculate the image mapping relationship between each projector and the camera based on the projection picture set and each camera picture set matched with each projector;

[0073] The geometric deformation parameter calculation unit is used to calculate the geometric deformation parameters of each projector based on the maximum projection area, the camera picture sets matched with each projector, and the image mapping relationship between each projector and the camera.

[0074] Based on the above embodiment, the mapping calculation unit further includes:

[0075] A geometric splitting coefficient calculation unit, used to obtain each camera picture in the camera picture set, calculate the effective display area of ​​each camera picture respectively, and calculate the geometric splitting coefficient of the projector according to the position of the effective display area of ​​each camera picture in the camera picture;

[0076] A size adjustment unit, used to calculate the maximum inscribed rectangle of each camera picture according to the effective display area of ​​each camera picture, and adjust the size of the maximum inscribed rectangle of each camera picture based on a preset size adjustment rule to obtain target inscribed rectangles of uniform size;

[0077] The merging unit is used to merge the inscribed rectangles of each target and obtain the maximum effective display area according to the merging result.

[0078] Based on the above embodiment, the mapping calculation unit further includes:

[0079] A first phase unwrapping unit is used to acquire each structured light image in the projection picture set, and perform phase unwrapping processing on each structured light image to obtain a standard phase unwrapping image that matches each structured light image;

[0080] A second phase unwrapping unit is used to obtain a camera picture set matching the projector, and perform phase unwrapping processing on each target light image in the camera picture set to obtain a target phase unwrapping image matching each target light image;

[0081] The image mapping unit is used to calculate the image mapping relationship between the projector and the camera based on each standard phase unwrapping image and each target phase unwrapping image.

[0082] Based on the above embodiment, the geometric deformation parameter calculation unit further includes:

[0083] A coordinate establishing unit, configured to generate a plurality of grid points at the maximum projection area based on a preset grid point generating rule, and to establish a coordinate system with the center point of the maximum projection area as the origin;

[0084] A grid coordinate confirmation unit, used to obtain the grid coordinates of each grid point in the coordinate system according to the coordinate system, and simultaneously obtain the starting parameters and the ending parameters of the maximum inscribed rectangle of the projector;

[0085] A normalization unit, configured to perform normalization processing on the grid coordinates of each grid point using the start parameter and the end parameter to obtain normalized coordinates matching each grid point;

[0086] A target coordinate calculation unit, used to calculate the target coordinates corresponding to each grid point based on the image mapping relationship between the projector and the camera, wherein the normalized coordinates are input as camera coordinates;

[0087] The aggregation unit is used to aggregate and store the normalized coordinates of each grid point and the target coordinates as geometric deformation parameters of the projector.

[0088] Based on the above embodiment, the image correction module 330 includes:

[0089] An image splitting unit, used for acquiring an image splitting coefficient of the projector, and splitting the image to be processed based on the image splitting coefficient to obtain an image splitting result matching the projector;

[0090] A geometric deformation unit, used to obtain geometric deformation parameters of the projector, and use a thin plate spline interpolation method to perform a geometric deformation operation on the image segmentation result of the projector based on the geometric deformation parameters to obtain a geometrically deformed image matching the projector;

[0091] The projection unit is used to use each projector to project the geometrically deformed image that matches the current projector to obtain a target projection image that matches the image to be processed.

[0092] A geometric correction device for a projection image provided by an embodiment of the present invention can execute a geometric correction method for a projection image provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0093] Embodiment 4

[0094] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0095] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0096] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0097] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a method for geometric correction of a projected image.

[0098] Accordingly, the method comprises:

[0099] Obtaining a preset projection picture set, using multiple projectors to play the projection picture set according to a preset projection order, and acquiring camera picture sets respectively matched with each projector through cameras at fixed positions;

[0100] The image splitting coefficient and geometric deformation parameter of each projector are calculated based on the image sets of each camera;

[0101] An image to be processed is acquired, and the image to be processed is processed according to the image splitting coefficient and the geometric deformation parameter of each projector to obtain a target projection image after geometric correction.

[0102] In some embodiments, a method for geometric correction of a projection image may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for geometric correction of a projection image described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a method for geometric correction of a projection image in any other appropriate manner (e.g., by means of firmware).

[0103] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0105] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

Claims

1. A method for geometric correction of a projected image, characterized in that: include: Obtaining a preset projection picture set, using multiple projectors to play the projection picture set according to a preset projection order, and acquiring camera picture sets respectively matched with each projector through cameras at fixed positions; The image splitting coefficient and geometric deformation parameter of each projector are calculated based on the image sets of each camera; An image to be processed is acquired, and the image to be processed is processed according to the image splitting coefficient and the geometric deformation parameter of each projector to obtain a target projection image after geometric correction.

2. The method according to claim 1, characterized in that Obtaining a preset projection picture set, using multiple projectors to play the projection picture set in a preset play order, and acquiring camera picture sets respectively matched with each projector through cameras at fixed positions, including: Acquire the projection picture set, wherein the projection picture set includes a pure white image and structured light images matching the pure white image; Make each projector play the projection picture set in sequence according to a preset projection order of the projectors; In the process of the projector playing each projection picture in the projection picture set, using a camera at a fixed position to collect the projection results of each projection picture, and obtaining a camera picture set matching the projector; Among them, the number of camera pictures contained in the camera picture set is equal to the number of projection pictures in the projection picture set, including a standard camera image matching the pure white image and a target light image matching each structured light image, and the number of the camera picture sets is equal to the number of the projectors.

3. The method according to claim 1, characterized in that The image splitting coefficient and geometric deformation parameters of each projector are calculated based on each camera image set, including: The image splitting coefficient and the maximum projection area of ​​each projector are calculated according to each camera picture set, and the image mapping relationship between each projector and the camera is calculated based on the projection picture set and each camera picture set matched with each projector; Based on the maximum projection area, the camera picture sets respectively matched with the projectors, and the image mapping relationship between the projectors and the cameras, the geometric deformation parameters of the projectors are respectively calculated.

4. The method according to claim 3, characterized in that The image splitting coefficient and the maximum projection area of ​​each projector are calculated based on the image sets of each camera, including: Obtaining each camera picture in the camera picture set, respectively calculating the effective display area of ​​each camera picture, and calculating the geometric splitting coefficient of the projector according to the position of the effective display area of ​​each camera picture in the camera picture; Calculate the maximum inscribed rectangle of each camera image according to the effective display area of ​​each camera image, and resize the maximum inscribed rectangle of each camera image based on a preset resize rule to obtain target inscribed rectangles of uniform sizes; The inscribed rectangles of each target are merged, and the maximum effective display area is obtained according to the merged result.

5. The method according to any one of claims 1 to 3, characterized in that: Based on the projection picture set and the camera picture sets respectively matched with the projectors, the image mapping relationship between the projectors and the cameras is calculated, including: Acquire each structured light image in the projection picture set, and perform phase unwrapping processing on each structured light image to obtain a standard phase unwrapping image that matches each structured light image; Acquire a camera picture set that matches the projector, and perform phase unwrapping processing on each target light image in the camera picture set to obtain a target phase unwrapping image that matches each target light image; The image mapping relationship between the projector and the camera is calculated based on each standard phase unwrapping image and each target phase unwrapping image.

6. The method according to any one of claims 1 to 4, characterized in that: Based on the maximum projection area, the camera picture sets respectively matched with the projectors, and the image mapping relationship between the projectors and the cameras, the geometric deformation parameters of the projectors are respectively calculated, including: Generate a plurality of grid points at the maximum projection area based on a preset grid point generation rule, and establish a coordinate system with the center point of the maximum projection area as the origin; According to the coordinate system, the grid coordinates of each grid point in the coordinate system are obtained, and at the same time, the starting parameters and the ending parameters of the maximum inscribed rectangle of the projector are obtained; Normalizing the grid coordinates of each grid point using the starting parameter and the ending parameter to obtain normalized coordinates matching each grid point; Based on the image mapping relationship between the projector and the camera, the target coordinates corresponding to each grid point are calculated respectively, wherein the normalized coordinates are input as camera coordinates; The normalized coordinates of each grid point and the target coordinates are aggregated and stored as geometric deformation parameters of the projector.

7. The method according to claim 1, characterized in that The image to be processed is processed according to the image splitting coefficient and the geometric deformation parameter of each projector to obtain a target projection image after geometric correction, including: Acquiring an image splitting coefficient of the projector, and splitting the image to be processed based on the image splitting coefficient to obtain an image splitting result matching the projector; Acquire geometric deformation parameters of the projector, and use a thin plate spline interpolation method to perform a geometric deformation operation on an image segmentation result of the projector based on the geometric deformation parameters to obtain a geometrically deformed image matching the projector; Each projector is used to project a geometrically deformed image that matches the current projector to obtain a target projection image that matches the image to be processed.

8. A geometric correction device for a projected image, characterized in that: include: A camera image acquisition module is used to acquire a preset projection image set, use multiple projectors to play the projection image set according to a preset projection order, and acquire camera image sets that match each projector respectively through cameras at fixed positions; A data calculation module is used to calculate the image splitting coefficient and geometric deformation parameters of each projector according to each camera image set; The image correction module is used to obtain the image to be processed, and process the image to be processed according to the image splitting coefficient and geometric deformation parameters of each projector to obtain a target projection image after geometric correction.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform a method for geometric correction of a projection image according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for geometric correction of a projection image according to any one of claims 1 to 7 when executed.

Images