Multi-projector picture automatic fusion method, device, equipment and medium

Abstract

The invention provides a multi-projector picture automatic fusion method. The method comprises the following steps: acquiring a checkerboard characteristic graph which is shot by an auxiliary camera and comprises each checkerboard characteristic sub-graph synchronously projected by a plurality of projectors; and obtaining an angular point coordinate data set of the checkerboard characteristic sub-graph under a preset coordinate system, and obtaining a transformed checkerboard characteristic graph of the checkerboard characteristic graph under a first ray machine coordinate system of the reference projector according to the first perspective transformation matrix of the reference projector and the plurality of angular point coordinate data sets. And obtaining an intersection coordinate data set between the maximum inscribed rectangle corresponding to the plurality of transformed checkerboard characteristic sub-graphs and each transformed checkerboard characteristic sub-graph. And according to the second perspective transformation matrix and the intersection coordinate data set corresponding to each projector, obtaining a target angular point coordinate data set of each intersection coordinate data set under a second ray machine coordinate system corresponding to each projector, so as to fuse the pictures of the plurality of projectors. And the efficiency and the accuracy of picture fusion of the plurality of projectors are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of picture fusion, in particular to a multi-projector picture automatic fusion method, device, equipment and medium. BACKGROUND

[0002] With the wide application of large-screen display technology in the fields of traffic transportation dispatching, large-scale conference implementation, flight simulation training, and combat command, the performance limitations of a single projector and the contradiction with the scene-based large-screen demand are becoming increasingly prominent. In view of this, the projector fusion and splicing technology emerges as the times require and gradually develops maturely under this background, becoming a key solution to meet the demand for ultra-large size and high-quality display.

[0003] In the prior art, the fusion of multiple projector pictures can be realized by using a traditional splicing and fusion method. The specific steps are as follows: first, a plurality of feature points or feature lines are defined on the pictures corresponding to the projectors; further, a debugging personnel manually adjusts the feature points or feature lines to target positions and adjusts additional parameters such as fusion curves; finally, a fixed strategy fusion algorithm is used to adjust the projector pictures in real time, so as to obtain a whole coordinated picture and realize the fusion of multiple projector pictures.

[0004] However, using the prior art, the operation of the debugging personnel is too dependent, which reduces the efficiency and accuracy of the fusion of multiple projector pictures. SUMMARY

[0005] Embodiments of the present application provide a multi-projector picture automatic fusion method, device, equipment and medium, which can avoid the problem of too much dependence on the debugging of the debugging personnel in the prior art, and improve the efficiency and accuracy of the fusion of multiple projector pictures.

[0006] To achieve the above-mentioned purpose, in a first aspect, embodiments of the present application provide a multi-projector picture automatic fusion method, comprising: obtaining a checkerboard feature map photographed by an auxiliary camera, wherein the checkerboard feature map comprises a plurality of checkerboard feature sub-pictures synchronously projected by a plurality of projectors; obtaining a plurality of corner point coordinate data sets of the plurality of checkerboard feature sub-pictures in a preset coordinate system, wherein the preset coordinate system is a coordinate system established according to a vertex of the checkerboard feature map as an origin, and the corner point coordinate data set comprises corner point coordinate data of four corner points of each of the checkerboard feature sub-pictures; obtaining a transformed checkerboard feature map of the checkerboard feature map in a first optical-mechanical coordinate system of a reference projector in the plurality of projectors according to a first perspective transformation matrix corresponding to the reference projector and a plurality of corner point coordinate data sets, wherein the transformed checkerboard feature map comprises a transformed checkerboard feature sub-picture corresponding to each of the checkerboard feature sub-pictures; obtaining a maximum inscribed rectangle corresponding to each of the plurality of transformed checkerboard feature subgraphs and an intersection coordinate data set between each of the plurality of transformed checkerboard feature subgraphs; obtaining a target corner point coordinate data set of each of the intersection coordinate data sets in a second optical-mechanical coordinate system corresponding to each of the projectors according to a second perspective transformation matrix corresponding to each of the projectors and the plurality of intersection coordinate data sets, wherein the target corner point coordinate data set is used to fuse the plurality of projector images.

[0007] In an embodiment, before the obtaining the transformed checkerboard feature subgraph of the checkerboard feature graph in the first optical-mechanical coordinate system of the reference projector according to the first perspective transformation matrix corresponding to the reference projector in the plurality of projectors and the plurality of corner point coordinate data sets, the method further comprises: determining a reference corner point coordinate data set corresponding to the reference projector in the plurality of corner point coordinate data sets; establishing the first perspective transformation matrix according to the reference corner point coordinate data set and a first preset corner point coordinate data set of the reference projector in the first optical-mechanical coordinate system, wherein the first preset corner point coordinate data set is a corner point coordinate data of four corners of the transformed checkerboard feature subgraph corresponding to the reference projector in the first optical-mechanical coordinate system.

[0008] In an embodiment, the reference corner point coordinate data set comprises four reference corner point coordinate data of the checkerboard feature subgraph corresponding to the reference projector, the first preset corner point coordinate data set comprises four first preset corner point coordinate data of the transformed checkerboard feature subgraph corresponding to the reference projector, and the establishing the first perspective transformation matrix according to the reference corner point coordinate data set and the first preset corner point coordinate data set of the reference projector in the first optical-mechanical coordinate system comprises: obtaining intermediate reference corner point coordinate data corresponding to each of the reference corner point coordinate data according to the preset first perspective transformation matrix and the four reference corner point coordinate data; obtaining the first perspective transformation matrix according to the four intermediate reference corner point coordinate data and the four first preset corner point coordinate data.

[0009] In an embodiment, the obtaining the transformed checkerboard feature subgraph of the checkerboard feature graph in the first optical-mechanical coordinate system of the reference projector according to the first perspective transformation matrix corresponding to the reference projector in the plurality of projectors and the plurality of corner point coordinate data sets comprises: transforming the plurality of corner point coordinate data sets according to the first perspective transformation matrix corresponding to the reference projector in the plurality of projectors to obtain a first transformed corner point coordinate data set corresponding to each of the plurality of corner point coordinate data sets in the first optical-mechanical coordinate system; obtaining the transformed checkerboard feature subgraph according to the plurality of first transformed corner point coordinate data sets.

[0010] In one embodiment, before the acquiring the intersection coordinate dataset between each transformed checkerboard feature subgraph and the maximum inscribed rectangle corresponding to the plurality of transformed checkerboard feature subgraphs, the method further comprises: According to a dynamic programming method, acquiring a maximum inscribed rectangle coordinate dataset corresponding to the plurality of transformed checkerboard feature subgraphs in the first optical-mechanical coordinate system; According to the maximum inscribed rectangle coordinate dataset, determining the maximum inscribed rectangle in the transformed checkerboard feature graph.

[0011] In one embodiment, before the acquiring the target corner point coordinate dataset of each intersection coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector respectively according to the second perspective transformation matrix corresponding to each projector respectively and the plurality of intersection coordinate datasets, the method further comprises: According to the first transformed corner point coordinate dataset corresponding to each projector in the first optical-mechanical coordinate system and the second preset corner point coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector respectively, establishing the second perspective transformation matrix corresponding to each projector respectively.

[0012] In one embodiment, the method further comprises: After obtaining the target corner point coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector respectively, sending the target corner point coordinate dataset to each projector, so that each projector fuses the plurality of projector pictures according to the target corner point coordinate dataset.

[0013] In a second aspect, an embodiment of the present application provides a multi-projector picture automatic fusion device, comprising: A checkerboard feature graph acquisition module is configured to acquire a checkerboard feature graph captured by an auxiliary camera, wherein the checkerboard feature graph comprises a plurality of checkerboard feature subgraphs synchronously projected by a plurality of projectors; A corner point coordinate dataset acquisition module is configured to acquire a corner point coordinate dataset of the plurality of checkerboard feature subgraphs in a preset coordinate system, wherein the preset coordinate system is a coordinate system established according to a vertex of the checkerboard feature graph as an origin, and the corner point coordinate dataset comprises corner point coordinate data of four corner points of each checkerboard feature subgraph; A transformed checkerboard feature graph acquisition module is configured to acquire a transformed checkerboard feature graph of the checkerboard feature graph in a first optical-mechanical coordinate system of a reference projector in the plurality of projectors according to a first perspective transformation matrix corresponding to the reference projector and the plurality of corner point coordinate datasets, wherein the transformed checkerboard feature graph comprises a transformed checkerboard feature subgraph corresponding to each checkerboard feature subgraph. An intersection coordinate dataset obtaining module is configured to obtain a maximum inscribed rectangle corresponding to the plurality of transformed checkerboard feature subgraphs and an intersection coordinate dataset between each transformed checkerboard feature subgraph. A target corner point coordinate dataset obtaining module is configured to obtain a target corner point coordinate dataset of each intersection coordinate dataset in a second optical-mechanical coordinate system corresponding to each projector according to a second perspective transformation matrix corresponding to each projector and the plurality of intersection coordinate datasets, wherein the target corner point coordinate dataset is used to fuse the plurality of projector images.

[0014] In a third aspect, an electronic device is provided, which includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the multi-projector image automatic fusion method in the first aspect are implemented.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores at least one program. When the at least one program is executed by a processor, the steps of the multi-projector image automatic fusion method in the first aspect are implemented.

[0016] The above technical solution has the following technical effects: In this way, the multi-projector image automatic fusion method provided by the embodiment can obtain a checkerboard feature graph shot by an auxiliary camera, wherein the checkerboard feature graph includes a plurality of checkerboard feature subgraphs synchronously projected by the plurality of projectors. A corner point coordinate dataset of the plurality of checkerboard feature subgraphs in a preset coordinate system is obtained, wherein the preset coordinate system is a coordinate system established according to a vertex of the checkerboard feature graph as an origin, and the corner point coordinate dataset includes corner point coordinate data of four corner points of each checkerboard feature subgraph. A transformed checkerboard feature graph of the checkerboard feature graph in a first optical-mechanical coordinate system of a reference projector in the plurality of projectors is obtained according to a first perspective transformation matrix corresponding to the reference projector and the plurality of corner point coordinate datasets, wherein the transformed checkerboard feature graph includes a transformed checkerboard feature subgraph corresponding to each checkerboard feature subgraph. A maximum inscribed rectangle corresponding to the plurality of transformed checkerboard feature subgraphs and an intersection coordinate dataset between each transformed checkerboard feature subgraph are obtained. A target corner point coordinate dataset of each intersection coordinate dataset in a second optical-mechanical coordinate system corresponding to each projector is obtained according to a second perspective transformation matrix corresponding to each projector and the plurality of intersection coordinate datasets, wherein the target corner point coordinate dataset is used to fuse the plurality of projector images. In this way, the problem that the prior art excessively relies on debugging by a debugging personnel can be avoided, and the efficiency and accuracy of the fusion of the plurality of projector images are improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1A flowchart of a multi-projector picture automatic fusion method provided by an embodiment of the present application is shown in the figure. Figure 2 A schematic diagram of a chessboard feature map photographed by an auxiliary camera is shown in the figure. Figure 3 A schematic diagram of a chessboard feature map transformed in a first optical machine coordinate system of a reference projector is shown in the figure. Figure 4 A schematic diagram of a multi-projector picture automatic fusion device provided by an embodiment of the present application is shown in the figure. Figure 5 A structural schematic diagram of an electronic device of an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0018] To further illustrate the embodiments, the present application provides accompanying drawings. These accompanying drawings are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can be used to explain the operating principles of the embodiments in conjunction with the relevant description of the specification. Those of ordinary skill in the art should be able to understand other possible implementations and advantages of the present application in conjunction with these contents. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0019] The present application will be further described in conjunction with the accompanying drawings and specific embodiments.

[0020] Embodiment one: Figure 1 A flowchart of a multi-projector picture automatic fusion method provided by an embodiment of the present application is shown in the figure. In this embodiment, the following steps are specifically included: S10: Obtain a chessboard feature map photographed by an auxiliary camera.

[0021] The chessboard feature map includes: a plurality of chessboard feature sub-maps synchronously projected by the plurality of projectors. For example, Figure 2 A schematic diagram of a chessboard feature map photographed by an auxiliary camera provided by an embodiment of the present application is shown in the figure. Figure 2 As shown in the figure, when there are three projectors, the three projectors synchronously project chessboard feature sub-maps to a projection surface, and the auxiliary camera photographs the chessboard feature sub-maps synchronously projected by the plurality of projectors to the projection surface, so as to obtain a chessboard feature map including the plurality of chessboard feature sub-maps synchronously projected by the three projectors, i.e., a chessboard feature sub-map 20 corresponding to a projector 1, a chessboard feature sub-map 21 corresponding to a projector 2, and a chessboard feature sub-map 22 corresponding to a projector 3. The auxiliary camera may, for example, be an electronic device such as a mobile phone or a tablet computer, but is not limited thereto. The present application does not specifically limit this, and those skilled in the art can set it according to the actual situation.

[0022] Specifically, the auxiliary camera is used to capture the checkerboard feature subgraphs synchronously projected by the plurality of projectors on the projection surface, so as to obtain a checkerboard feature graph including the plurality of checkerboard feature subgraphs.

[0023] S11: Obtain a corner point coordinate dataset of the plurality of checkerboard feature subgraphs in a preset coordinate system.

[0024] The preset coordinate system is a coordinate system established with a vertex of the checkerboard feature graph as an origin. The corner point coordinate dataset includes corner point coordinate data of four corner points of each checkerboard feature subgraph.

[0025] For example, based on the above embodiment, continuing to refer to Figure 2 A coordinate system is established with a vertex of the checkerboard feature graph as an origin. For the checkerboard feature subgraph 21, the corner point coordinate dataset in the preset coordinate system is obtained, that is, the corner point coordinate data of the corner point A, the corner point B, the corner point C and the corner point D of the checkerboard feature subgraph 21 in the preset coordinate system, but the present application is not limited thereto, and the person skilled in the art can set it according to the actual situation.

[0026] Specifically, after obtaining the checkerboard feature graph including the plurality of checkerboard feature subgraphs synchronously projected by the plurality of projectors, for each checkerboard feature subgraph, the corner point coordinate data corresponding to the four corner points of each checkerboard feature subgraph in the preset coordinate system is obtained.

[0027] Optionally, the findChessboardCorners() function of OpenCV can be called to calculate the corner point coordinate data corresponding to the four corner points of each checkerboard feature subgraph, but the present application is not limited thereto, and the person skilled in the art can set it according to the actual situation.

[0028] S12: Obtain a transformed checkerboard feature graph of the checkerboard feature graph in a first optical-mechanical coordinate system of a reference projector according to a first perspective transformation matrix corresponding to the reference projector and the plurality of corner point coordinate datasets.

[0029] The first perspective transformation matrix is used to transform the plurality of checkerboard feature subgraphs synchronously projected by the plurality of projectors in the preset coordinate system to the first optical-mechanical coordinate system of the reference projector.

[0030] The above reference projector refers to the projector corresponding to the rectangular shape of the checkerboard feature subgraph after the plurality of projectors project on the projection surface.

[0031] The first optical-mechanical coordinate system is a key reference coordinate system inside the reference projector, which is used to describe the position of a point in the space of the optical-mechanical device (i.e. the projection unit). The transformed checkerboard feature graph includes a transformed checkerboard feature subgraph corresponding to each checkerboard feature subgraph.

[0032] For example, in the above embodiment, with reference to Figure 3 as shown, Figure 3 The schematic diagram for transforming the checkerboard feature map in the first optical machine coordinate system of the reference projector provided by an embodiment of the present application includes: the transformed checkerboard feature submap 30 corresponding to the checkerboard feature submap 20 of the projector 1, the transformed checkerboard feature submap 31 corresponding to the checkerboard feature submap 21 of the projector 2, and the transformed checkerboard feature submap 32 corresponding to the checkerboard feature submap 22 of the projector 3. Further, assuming that the projector 2 is the reference projector, the first optical machine coordinate system is established according to one corner point of the transformed checkerboard feature submap 31 as the origin 01. However, the present application is not specifically limited, and a person skilled in the art can set it according to the actual situation.

[0033] Optionally, in the above embodiment, before S12 is performed, the method further includes: S20: determining a reference corner point coordinate data set corresponding to the reference projector in the plurality of corner point coordinate data sets.

[0034] The reference corner point coordinate data set includes four reference corner point coordinate data of the checkerboard feature submap corresponding to the reference projector.

[0035] Specifically, the reference projector is determined in the plurality of projectors, and thus the reference corner point coordinate data set corresponding to the reference projector is determined in the plurality of corner point coordinate data sets.

[0036] S21: establishing a first perspective transformation matrix according to the reference corner point coordinate data set and a first preset corner point coordinate data set of the reference projector in the first optical machine coordinate system.

[0037] The first preset corner point coordinate data set is the corner point coordinate data of four corner points of the transformed checkerboard feature submap of the reference projector in the first optical machine coordinate system. The first preset corner point coordinate data set includes four first preset corner point coordinate data of the transformed checkerboard feature submap corresponding to the reference projector.

[0038] For example, in the above embodiment, with reference to Figures 2-3 as shown, the four reference corner point coordinate data of the checkerboard feature submap 21 corresponding to the reference projector are the corner point coordinate data of the corner point A, the corner point B, the corner point C, and the corner point D in the preset coordinate system, and the four first preset corner point coordinate data of the transformed checkerboard feature submap 31 corresponding to the reference projector are the corner point A1 coordinate data, the corner point B1 coordinate data, the corner point C1 coordinate data, and the corner point D1 coordinate data in the first optical machine coordinate system, but the present application is not specifically limited, and a person skilled in the art can set it according to the actual situation.

[0039] Optionally, based on the above-mentioned embodiments, in some embodiments of the present application, one implementation of S21 can be: S211: obtaining intermediate reference corner coordinate data corresponding to each reference corner coordinate data according to the preset first perspective transformation matrix and the four reference corner coordinate data.

[0040] wherein the preset first perspective transformation matrix is a matrix set in advance, each parameter included in the preset first perspective transformation matrix is an unknown variable, and the preset first perspective transformation matrix is used to transform the four reference corner coordinate data. may be represented as:

[0041] Specifically, the preset first perspective transformation matrix is used to transform the four reference corner coordinate data, and intermediate reference corner coordinate data corresponding to each reference corner coordinate data is obtained.

[0042] Optionally, based on the above-mentioned embodiments, in some embodiments of the present application, the intermediate reference corner coordinate data is determined according to the formula

[0043] wherein, ， represents the i-th reference corner coordinate data in the four reference corner coordinate data, i represents the first perspective transformation matrix, ， ， represents the i-th intermediate reference corner coordinate data corresponding to the i-th reference corner coordinate data, that is, the following can be determined: i i

[0044]

[0045]

[0046] S212: obtaining the first perspective transformation matrix according to the intermediate reference corner coordinate data and the four first preset corner coordinate data.

[0047] Optionally, based on the above-mentioned embodiments, in some embodiments of the present application, one implementation of S212 can be: ​​​​An equivalent perspective transformation matrix of the preset first perspective transformation matrix is determined, and the first perspective transformation matrix is solved according to the equivalent perspective transformation matrix, the four intermediate reference corner point coordinate data, the four first preset corner point coordinate data, and a preset transformation calculation formula.

[0048] Optionally, on the basis of the above embodiment, in some embodiments of the present application, the equivalent perspective transformation matrix of the preset first perspective transformation matrix can be defined by the following expression:

[0049] wherein, represents the equivalent perspective transformation matrix, .

[0050] Optionally, on the basis of the above embodiment, in some embodiments of the present application, the preset transformation calculation formula can be defined by the following expression:

[0051]

[0052] wherein, ， represents the i-th first preset corner point coordinate data corresponding to the i-th reference corner point coordinate data in the four reference corner point coordinate data. i i

[0053] Optionally, on the basis of the above embodiment, in some embodiments of the present application, one implementation of S12 can be: S121: transforming the plurality of corner point coordinate data sets according to the first perspective transformation matrix corresponding to the reference projector in the plurality of projectors, to obtain a plurality of first transformed corner point coordinate data sets corresponding to the plurality of corner point coordinate data sets in the first optical-mechanical coordinate system.

[0054] S122: obtaining a transformed checkerboard feature map according to the plurality of first transformed corner point coordinate data sets.

[0055] Specifically, after obtaining the first perspective transformation matrix corresponding to the reference projector, the plurality of corner point coordinate data sets are transformed by using the first perspective transformation matrix corresponding to the reference projector, so that the plurality of corner point coordinate data sets in the preset coordinate system are transformed to the first optical-mechanical coordinate system, to obtain a plurality of first transformed corner point coordinate data sets corresponding to the plurality of corner point coordinate data sets in the first optical-mechanical coordinate system. After obtaining the first transformed corner point coordinate data sets of the plurality of corner point coordinate data sets in the first optical-mechanical coordinate system, visualization is performed by using the plurality of first transformed corner point coordinate data sets, so as to determine the transformed checkerboard feature map.

[0056] ​​S13: Obtain the intersection coordinate dataset between the maximum inscribed rectangle corresponding to the plurality of transformed checkerboard feature subgraphs and each transformed checkerboard feature subgraph.

[0057] Optionally, on the basis of the above-mentioned embodiments, in some embodiments of the present application, before S13 is executed, it further comprises: S30: Obtain the maximum inscribed rectangle coordinate dataset corresponding to the plurality of transformed checkerboard feature subgraphs in the first optical-mechanical coordinate system according to the dynamic programming method.

[0058] S31: Determine the maximum inscribed rectangle in the transformed checkerboard feature graph according to the maximum inscribed rectangle coordinate dataset.

[0059] Specifically, the maximum inscribed rectangle coordinate dataset corresponding to the plurality of transformed checkerboard feature subgraphs is obtained in the first optical-mechanical coordinate system corresponding to the reference projector by the dynamic programming method, and the maximum inscribed rectangle is determined by visualizing the transformed checkerboard feature graph using the maximum inscribed rectangle coordinate dataset.

[0060] For example, continuing to refer to Figure 3 As shown, the maximum inscribed rectangle coordinate dataset comprises the coordinate data of the vertex a of the maximum inscribed rectangle, the coordinate data of the vertex b, the coordinate data of the vertex c, and the coordinate data of the vertex d, based on which the maximum inscribed rectangle 00 is determined in the transformed checkerboard feature graph according to the maximum inscribed rectangle coordinate dataset, but the present application is not limited thereto, and the person skilled in the art can set it according to the actual situation.

[0061] Specifically, after obtaining the maximum inscribed rectangle corresponding to the plurality of transformed checkerboard feature subgraphs, the intersection coordinate dataset between the maximum inscribed rectangle and each transformed checkerboard feature subgraph is obtained in the first optical-mechanical coordinate system.

[0062] For example, continuing to refer to Figure 3 As shown, for the transformed checkerboard feature subgraph 30, the intersection coordinate dataset between the maximum inscribed rectangle 00 and the transformed checkerboard feature subgraph 30 is the intersection coordinate data e, the intersection coordinate data f, the intersection coordinate data g, and the intersection coordinate data h, but the present application is not limited thereto, and the person skilled in the art can set it according to the actual situation.

[0063] S14: Obtain the target corner point coordinate dataset of each intersection coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector according to the second perspective transformation matrix corresponding to each projector and the plurality of intersection coordinate datasets.

[0064] The target corner point coordinate data set is used for fusing multiple projector pictures. The second perspective transformation matrix is used for transforming the intersection coordinate data set between the maximum inscribed rectangle in the first optical machine coordinate system and each transformed checkerboard feature subgraph to the second optical machine coordinate system corresponding to each projector respectively.

[0065] Optionally, on the basis of the above-mentioned embodiments, in some embodiments of the present application, before S14 is executed, the following steps are further included: S40: establishing a second perspective transformation matrix corresponding to each projector respectively according to the first transformed corner point coordinate data set corresponding to each projector in the first optical machine coordinate system and the second preset corner point coordinate data set in the second optical machine coordinate system corresponding to each projector respectively.

[0066] For the specific implementation of S40, refer to the specific implementation of S21, which will not be described in detail here.

[0067] Specifically, after obtaining the intersection coordinate data set between the maximum inscribed rectangle and each transformed checkerboard feature subgraph, each intersection coordinate data set is transformed by the second perspective transformation matrix corresponding to each projector respectively to obtain a target corner point coordinate data set of each intersection coordinate data set in the second optical machine coordinate system corresponding to each projector respectively, and the target corner point coordinate data set is used for fusing multiple projector pictures.

[0068] In this way, the multi-projector picture automatic fusion method provided by the embodiment can obtain a checkerboard feature graph shot by an auxiliary camera, wherein the checkerboard feature graph includes: multiple checkerboard feature subgraphs synchronously projected by multiple projectors. Corner point coordinate data sets of the multiple checkerboard feature subgraphs in a preset coordinate system are obtained, wherein the preset coordinate system is a coordinate system established according to a vertex of the checkerboard feature graph as an origin, and the corner point coordinate data set includes: corner point coordinate data of four corners of each checkerboard feature subgraph. A transformed checkerboard feature graph of the checkerboard feature graph in a first optical machine coordinate system of a reference projector in the multiple projectors is obtained according to a first perspective transformation matrix corresponding to the reference projector and the multiple corner point coordinate data sets, wherein the transformed checkerboard feature graph includes: a transformed checkerboard feature subgraph corresponding to each checkerboard feature subgraph. A maximum inscribed rectangle corresponding to the multiple transformed checkerboard feature subgraphs and an intersection coordinate data set between the maximum inscribed rectangle and each transformed checkerboard feature subgraph are obtained. Target corner point coordinate data sets of each intersection coordinate data set in a second optical machine coordinate system corresponding to each projector respectively are obtained according to a second perspective transformation matrix corresponding to each projector respectively and the multiple intersection coordinate data sets, wherein the target corner point coordinate data set is used for fusing multiple projector pictures. In this way, the problem that the prior art excessively relies on debugging personnel for debugging can be avoided, and the efficiency and accuracy of the fusion of multiple projector pictures are improved.

[0069] Optionally, on the basis of the above embodiments, in some embodiments of the present application, after obtaining the target corner point coordinate data set in the second optical-mechanical coordinate system corresponding to each projector, the target corner point coordinate data set is sent to each projector, so that each projector fuses multiple projector pictures according to the target corner point coordinate data set.

[0070] Embodiment two: Figure 4 The schematic diagram of the multi-projector picture automatic fusion device provided by an embodiment of the present application is shown in FIG. 1, which includes a checkerboard feature map acquisition module 10, a corner point coordinate data set acquisition module 11, a transformed checkerboard feature map acquisition module 12, an intersection coordinate data set acquisition module 13, and a target corner point coordinate data set acquisition module 14. Figure 4

[0071] The checkerboard feature map acquisition module 10 is configured to acquire a checkerboard feature map captured by an auxiliary camera, wherein the checkerboard feature map includes multiple checkerboard feature sub-maps synchronously projected by multiple projectors.

[0072] The corner point coordinate data set acquisition module 11 is configured to acquire a corner point coordinate data set of multiple checkerboard feature sub-maps in a preset coordinate system, wherein the preset coordinate system is a coordinate system established according to a vertex of the checkerboard feature map as an origin, and the corner point coordinate data set includes corner point coordinate data of four corner points of each checkerboard feature sub-map.

[0073] The transformed checkerboard feature map acquisition module 12 is configured to acquire a transformed checkerboard feature map of the checkerboard feature map in a first optical-mechanical coordinate system of a reference projector in the multiple projectors according to a first perspective transformation matrix corresponding to the reference projector and the multiple corner point coordinate data sets, wherein the transformed checkerboard feature map includes a transformed checkerboard feature sub-map corresponding to each of the checkerboard feature sub-maps.

[0074] The intersection coordinate data set acquisition module 13 is configured to acquire an intersection coordinate data set between a maximum inscribed rectangle corresponding to multiple transformed checkerboard feature sub-maps and each transformed checkerboard feature sub-map.

[0075] The target corner point coordinate data set acquisition module 14 is configured to acquire a target corner point coordinate data set of each intersection coordinate data set in a second optical-mechanical coordinate system corresponding to each projector according to a second perspective transformation matrix corresponding to each projector and the multiple intersection coordinate data sets, wherein the target corner point coordinate data set is used for fusing multiple projector pictures.

[0076] ​Thus, in the embodiment, the chessboard feature map acquisition module acquires the chessboard feature map captured by the auxiliary camera, wherein the chessboard feature map comprises a plurality of chessboard feature subgraphs synchronously projected by the plurality of projectors. The corner point coordinate dataset acquisition module acquires a plurality of corner point coordinate datasets of the plurality of chessboard feature subgraphs in a preset coordinate system, wherein the preset coordinate system is a coordinate system established with a vertex of the chessboard feature map as an origin, and each corner point coordinate dataset comprises corner point coordinate data of four corner points of each chessboard feature subgraph. The transformed chessboard feature map acquisition module acquires a transformed chessboard feature map of the chessboard feature map in a first optical-mechanical coordinate system of a reference projector in the plurality of projectors according to a first perspective transformation matrix corresponding to the reference projector and the plurality of corner point coordinate datasets, wherein the transformed chessboard feature map comprises a transformed chessboard feature subgraph corresponding to each of the chessboard feature subgraphs. The intersection coordinate dataset acquisition module acquires intersection coordinate datasets between a maximum inscribed rectangle corresponding to the plurality of transformed chessboard feature subgraphs and each transformed chessboard feature subgraph. The target corner point coordinate dataset acquisition module acquires target corner point coordinate datasets of each intersection coordinate dataset in a second optical-mechanical coordinate system corresponding to each projector according to a second perspective transformation matrix corresponding to each projector and the plurality of intersection coordinate datasets, wherein the target corner point coordinate datasets are used for fusing the plurality of projector images. Thus, the problem that the prior art excessively relies on debugging personnel for debugging can be avoided, and the efficiency and accuracy of fusing the plurality of projector images are improved.

[0077] Embodiment three The application further provides an electronic device, such as Figure 5 As shown in the figure, the electronic device comprises a processor 1101, a memory 1102, a bus 1103, and a computer program stored in the memory 1102 and executable on the processor 1101, the processor 1101 comprising one or more processing cores, the memory 1102 being connected to the processor 1101 through the bus 1103, the memory 1102 being used for storing program instructions, and the processor implementing the steps in the above method embodiments of the embodiment one of the application when executing the computer program.

[0078] Further, as an executable solution, the electronic device can be a computer unit, which can be a desktop computer, a notebook, a palm computer, a cloud server, and the like. The computer unit can comprise, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned component structure of the computer unit is merely an example of the computer unit, and does not constitute a limitation on the computer unit, and can comprise more or fewer components than the above, or combine certain components, or different components. For example, the computer unit can also comprise an input / output device, a network access device, a bus, and the like, which are not limited by the embodiments of the application.

[0079] Further, as an executable solution, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like, which is a control center of the computer unit, and connects all parts of the computer unit through various interfaces and lines.

[0080] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer unit by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; and the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0081] Embodiment Four The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the method in the above embodiments of the application.

[0082] The modules / units integrated by the computer units, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM) and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0083] Although the present application has been specifically shown and described with respect to preferred embodiments, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A method for automatic blending of images from multiple projectors, characterized in that, The method includes: Acquire a chessboard feature map captured by an auxiliary camera, wherein the chessboard feature map includes: each chessboard feature sub-map projected synchronously by multiple projectors; Obtain a dataset of corner coordinates of multiple chessboard feature sub-images in a preset coordinate system, wherein the preset coordinate system is a coordinate system established with one vertex of the chessboard feature image as the origin, and the corner coordinate dataset includes: corner coordinate data of the four corner points of each chessboard feature sub-image; Based on the first perspective transformation matrix corresponding to the reference projector among multiple projectors and multiple corner point coordinate datasets, the transformed checkerboard feature map under the first optical-mechanical coordinate system of the reference projector is obtained, wherein the transformed checkerboard feature map includes: the transformed checkerboard feature map corresponding to each of the checkerboard feature sub-maps. Obtain the dataset of intersection coordinates between the largest inscribed rectangle corresponding to multiple transformed chessboard feature subgraphs and each transformed chessboard feature subgraph; Based on the second perspective transformation matrix corresponding to each projector and multiple intersection coordinate datasets, the target corner coordinate datasets in the second optical-mechanical coordinate system corresponding to each projector are obtained. The target corner coordinate datasets are used to fuse the images of multiple projectors.

2. The method according to claim 1, characterized in that, Before obtaining the transformed checkerboard feature map in the first optical-mechanical coordinate system of the reference projector based on the first perspective transformation matrix corresponding to the reference projector among multiple projectors and multiple corner point coordinate datasets, the method further includes: Determine the reference corner coordinate dataset corresponding to the reference projector from multiple corner coordinate datasets; Based on the reference corner point coordinate dataset and the first preset corner point coordinate dataset of the reference projector in the first optical-mechanical coordinate system, a first perspective transformation matrix is ​​established, wherein the first preset corner point coordinate dataset is the corner point coordinate data of the four corner points corresponding to the transformed checkerboard feature sub-map of the reference projector in the first optical-mechanical coordinate system.

3. The method according to claim 2, characterized in that, The reference corner coordinate dataset includes: the coordinate data of four reference corner points of the checkerboard feature sub-image corresponding to the reference projector; the first preset corner coordinate dataset includes: the coordinate data of four first preset corner points of the transformed checkerboard feature sub-image corresponding to the reference projector; the step of establishing a first perspective transformation matrix based on the reference corner coordinate dataset and the first preset corner coordinate dataset of the reference projector in the first optical-mechanical coordinate system includes: Based on the preset first perspective transformation matrix and the coordinate data of the four reference corner points, obtain the coordinate data of the intermediate reference corner points corresponding to the coordinate data of each reference corner point; Based on the coordinate data of the four intermediate reference corner points and the coordinate data of the four first preset corner points, the first perspective transformation matrix is ​​obtained.

4. The method according to claim 3, characterized in that, The step of obtaining the transformed checkerboard feature map of the checkerboard feature map in the first optical-mechanical coordinate system of the reference projector based on the first perspective transformation matrix corresponding to the reference projector among multiple projectors and multiple corner point coordinate datasets includes: Based on the first perspective transformation matrix corresponding to the reference projector among multiple projectors, the multiple corner point coordinate datasets are transformed to obtain the first transformed corner point coordinate datasets corresponding to the multiple corner point coordinate datasets in the first optical-mechanical coordinate system. The transformed chessboard feature map is obtained based on multiple first transformed corner point coordinate datasets.

5. The method according to claim 4, characterized in that, Before obtaining the intersection coordinate dataset between the largest inscribed rectangle corresponding to multiple transformed chessboard feature subgraphs and each transformed chessboard feature subgraph, the method further includes: Based on the dynamic programming method, the maximum inscribed rectangle coordinate dataset corresponding to multiple transformed chessboard feature subgraphs is obtained in the first optomechanical coordinate system. The maximum inscribed rectangle is determined in the transformed checkerboard feature map based on the maximum inscribed rectangle coordinate dataset.

6. The method according to claim 5, characterized in that, Before obtaining the target corner coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector based on the second perspective transformation matrix corresponding to each projector and multiple intersection coordinate datasets, the method further includes: Based on the first transformation corner point coordinate dataset corresponding to each projector in the first optical-mechanical coordinate system and the second preset corner point coordinate dataset corresponding to each projector in the second optical-mechanical coordinate system, a second perspective transformation matrix corresponding to each projector is established.

7. The method according to claim 6, characterized in that, The method further includes: After obtaining the target corner coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector, the target corner coordinate dataset is sent to each projector so that each projector can fuse multiple projector images according to the target corner coordinate dataset.

8. An automatic image blending device for multiple projectors, characterized in that, include: The checkerboard feature map acquisition module is used to acquire checkerboard feature maps captured by an auxiliary camera, wherein the checkerboard feature map includes: each checkerboard feature sub-map projected synchronously by multiple projectors; The corner coordinate dataset acquisition module is used to acquire the corner coordinate dataset of multiple chessboard feature sub-images in a preset coordinate system. The preset coordinate system is a coordinate system established with one vertex of the chessboard feature image as the origin. The corner coordinate dataset includes: the corner coordinate data of the four corner points of each chessboard feature sub-image. The transformation checkerboard feature map acquisition module is used to acquire the transformation checkerboard feature map of the checkerboard feature map in the first optical-mechanical coordinate system of the reference projector based on the first perspective transformation matrix corresponding to the reference projector among multiple projectors and multiple corner point coordinate datasets. The transformation checkerboard feature map includes: the transformation checkerboard feature map corresponding to each checkerboard feature sub-map. The intersection coordinate dataset acquisition module is used to obtain the intersection coordinate dataset between the largest inscribed rectangle corresponding to multiple transformed chessboard feature subgraphs and each transformed chessboard feature subgraph. The target corner coordinate dataset acquisition module is used to acquire the target corner coordinate dataset in the second optical-mechanical coordinate system corresponding to each projector based on the second perspective transformation matrix corresponding to each projector and multiple intersection coordinate datasets. The target corner coordinate dataset is used to fuse the images of multiple projectors.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the automatic fusion method for multi-projector images according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is executed by a processor to implement the steps of the automatic fusion method for multi-projector images as described in any one of claims 1 to 7.

Images