A fusion registration method for projection images of dual scene generation devices

Abstract

The invention proposes a fusion registration method for projection images of a dual-scene generation device, belongs to the technical field of image registration, and is used to solve the mismatch problem of a dual-scene generation device in a dual-channel common-aperture parallel projection optical path. After the dual-scene generation device is mechanically debugged to achieve preliminary registration, a comprehensive image registration algorithm is used for precise registration: first, a scale-invariant feature transformation algorithm is used to extract feature points, and then a matching mechanism of a fast nearest neighbor approximation search is used to perform preliminary matching on the feature points, and a random sampling consistency algorithm is used to eliminate erroneous matching pairs to calculate optimal homography matrix parameters; finally, an affine transformation is performed on one source image, and the other remains unchanged, and the two images are uploaded to two scene generation devices respectively, so that high-precision real-time fusion of the projection images can be achieved, and the mismatch error is about 1 pixel, which greatly improves the projection registration accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of image registration, and in particular, relates to a fusion registration method for projection images of a dual-view generating device. Background Art

[0002] Image registration methods based on image vision aim to achieve pixel alignment by detecting relevant feature points in the image, matching and iteratively finding the optimal transformation model, so as to intuitively obtain fusion information. This method is applicable to acquired image data. The goal of this method is to achieve pixel alignment so as to intuitively fuse information between different images. This technology has a wide range of applications in computer vision, medical image processing, material mechanics, remote sensing and other fields.

[0003] The basic principle of image registration is to detect feature points in images, which have corresponding relationships between different images. By matching these feature points and iteratively finding the optimal transformation model, pixel-level alignment of images can be achieved. This process can be used to accurately superimpose images taken from different perspectives or at different times, making the fused image clearer and more accurate.

[0004] Existing technologies are mostly used for image registration of imaging systems, such as eliminating artifacts generated when imaging a moving target. The artifact of a moving target is caused by the difference between the motion state at the previous moment and the motion state at the next moment, that is, the inconsistency in the image shooting time and the movement of dynamic objects in the scene. Eliminating artifacts caused by target motion can greatly improve the imaging quality, but these technologies are only aimed at improving the performance of the imaging system and cannot be applied to the projection display field.

[0005] In the field of projection, by using a dual-channel common aperture projection optical system, the modulation layers of the dual scene generation devices that are synchronously controlled can be optically superimposed, so that the pixels overlap one by one. The joint spatial light modulation of the overlapping pixels can greatly increase the bit depth or frame rate of the projected image and improve the performance of the projection device. However, it is impossible to achieve pixel-level registration of the two scene generation devices through traditional mechanical adjustment methods, and the artifacts caused by the mismatch will also reduce the projection quality. Therefore, the present invention proposes an image fusion registration method for the projection field, which is suitable for solving the mismatch problem of the two scene generation devices in the dual-channel common aperture parallel optical path. Summary of the invention

[0006] The present invention proposes a fusion registration method for projection images of a dual scene generating device to solve the technical problem of projection image mismatch of the dual scene generating device in a dual-channel common aperture optical system, and can realize pixel-level registration of the projection images of the dual scene generating device.

[0007] The present invention is achieved through the following technical solutions:

[0008] A fusion registration method for projection images of a dual scene generation device: the method specifically comprises the following steps:

[0009] Step 1: Preliminary registration: The camera receiving module collects the projection of the dual scene generator, collects the horizontal and vertical axis errors of the dual crosshairs, and performs preliminary mechanical debugging;

[0010] Step 2, preprocessing: upload the source image with complex features to the projection module, the dual scene generation device projects respectively, the camera receiving module performs format conversion, and the obtained images are used as reference images and images to be registered respectively to complete the preprocessing;

[0011] Step 3: Accurate registration: Use the image registration module to accurately register the reference image and the image to be registered.

[0012] Step 4, affine transformation: use the affine transformation module to perform affine transformation on the source image uploaded to one of the scene generation devices according to the homography matrix, and the source image uploaded to the other scene generation device remains unchanged, and is projected again through the projection module, so that the artifact disappears;

[0013] Step 5: Verify the registration accuracy: Repeat steps 2 to 4 to verify the registration accuracy and determine whether the homography matrix is ​​valid.

[0014] Step 6, continuous multi-frame projection: When the subsequent source images are projected, before being uploaded to the two scene generation devices, one of them needs to be affine transformed using the homography matrix first, and then the source image and the transformed source image are uploaded to the two scene generation devices respectively and projected, so as to realize the fusion and registration of the projection images of the two scene generation devices.

[0015] Furthermore, in step one, a cross-line source image is uploaded to the projection module so that the dual scene generation devices are projected simultaneously, and the camera receiving module is fixed for acquisition; the horizontal and vertical axis errors of the acquired dual cross-lines are observed, and preliminary mechanical debugging is performed, and finally the average errors of the horizontal and vertical axes are controlled within about 50 pixels to complete the preliminary alignment.

[0016] A source image with complex features is uploaded to the projection module, and the dual scene generation devices are projected separately. The camera receiving module collects two images and stores them in the host computer; the pixel size of the two collected images is converted into the pixel size that conforms to the scene generation device; the obtained images are used as reference images and images to be registered to complete the preprocessing.

[0017] Furthermore, in step three, the precise registration process is as follows:

[0018] Step 31: firstly extract feature points of the reference image and the image to be registered according to the scale-invariant feature transformation algorithm;

[0019] Step 32: Use the matching mechanism of fast nearest neighbor approximation search to perform initial matching on the extracted feature points;

[0020] Step 33: Random sampling consistency algorithm, iteratively calculate the optimal homography matrix parameters to complete the precise matching of feature points.

[0021] Furthermore, in step 31, specifically:

[0022] Firstly, the Gaussian difference method is used to construct the scale space extreme value; secondly, the extreme value points detected in the Gaussian scale space are fitted to determine the exact position and scale of the feature points;

[0023] Then the feature point direction is assigned according to the local gradient of the image; finally, according to the image gradient size and direction, a descriptor generator is obtained to calculate the feature descriptor of each feature point.

[0024] Furthermore, in step 32, a fast nearest neighbor approximation search matching mechanism is established

[0025] For the feature points of the reference image and the image to be registered extracted in step 3.1, a set is established, and the feature points in the reference image are: , Feature points in the image to be registered: ,

[0026] The Euclidean distance is used to measure the similarity of feature points with 128 dimensions. The similarity measurement of any feature point pair is: ,To get the paired feature point pairs, Need to meet: In the reference image, the distance R i The nearest point S j , and the next closest point S p The ratio of the Euclidean distance of is less than a certain set threshold T;

[0027] The range of T is 0-1, which is an empirical value. When it is close to 0, the total number of matches decreases and the matching accuracy increases; when it is close to 1, the total number of matches increases, the number of false matches also increases, and the matching accuracy decreases.

[0028] Further, in step three-three, a random sampling consensus algorithm is used;

[0029] For the feature points that are initially matched successfully in step 3.2, 4 pairs are randomly selected as samples, and the homography matrix parameters are calculated iteratively for multiple times to eliminate incorrect matching pairs and improve the matching accuracy, so that the corresponding cost function is minimized, and the final precise matching and optimal homography matrix parameters are obtained to complete the precise matching of the feature points of the reference image and the image to be registered;

[0030] The homography matrix is ​​expressed as:

[0031]

[0032] in( μ 1, ν 1, 1) is the homogeneous coordinates of the feature points of the reference image, ( μ 2, ν 2, 1) are the homogeneous coordinates of the feature points of the image to be registered, h 11 - h 33 are the parameters of the homography matrix;

[0033] The cost function is expressed as:

[0034] .

[0035] Furthermore, in step 4, an affine transformation module is used to perform an affine transformation on the source image uploaded to one of the scene generation devices according to the homography matrix, and the source image and the image after the affine transformation of the source image are uploaded to the projection module, and are projected simultaneously by the first scene generation device and the second scene generation device, respectively, and the camera receiving module collects the images again; finally, the projected images of the two scene generation devices are fused, and the artifacts disappear;

[0036] In step five, in order to verify the final registration accuracy, start again from step two, collect the projection images of the dual-view generation device separately, and after size conversion, perform feature point extraction, preliminary matching, and fine matching again, and record the coordinates of 20 pairs of matched feature points for comparison. When the image registration accuracy is less than 1 pixel, the required homography matrix is ​​determined to be valid.

[0037] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0038] Beneficial effects of the present invention

[0039] The present invention is innovatively used in the field of dual-scene generation device projection image fusion. Compared with traditional mechanical adjustment methods, the pixel-level alignment of the dual-scene generation device has significant advantages. Traditional mechanical adjustment methods are difficult to achieve pixel-level alignment because they may be limited by mechanical precision. Mismatches may cause artifacts, which inevitably lead to a reduction in the quality of the projected image. In addition, mismatches may also reduce the joint spatial optical capacity of the synchronously controlled dual-scene generation devices, affecting the overall system performance. Therefore, the present invention solves the problem of image quality degradation caused by mismatches, and effectively guarantees the joint spatial optical capacity of the parallel dual-scene generation devices.

[0040] The present invention achieves high-precision image registration by adopting a scale-invariant feature transformation algorithm, a fast nearest neighbor approximation search matching mechanism, and a random sampling consistency algorithm. Through final verification, the average pixel error of the projected image on the X-axis and Y-axis is within 1 pixel, indicating that the registration accuracy is quite high.

[0041] The present invention adopts a random sampling consistency algorithm to perform precise matching of feature points, which can eliminate erroneous matching pairs, improve the robustness of the algorithm, and make the final alignment result more reliable.

[0042] The present invention introduces a matching mechanism of fast nearest neighbor approximation search, which accelerates the initial matching process of feature points, thereby improving the efficiency of overall registration.

[0043] The present invention can be applicable to different types of source images, including images with complex features, through an affine transformation module.

[0044] The two scene generating devices used in the present invention can be any type of scene generating devices as long as the product models are exactly the same, including but not limited to digital micromirror devices, liquid crystal displays, liquid crystal on silicon, resistor arrays, etc.

[0045] This versatility makes the scheme have application potential in a variety of scenarios. Through the final affine transformation, the artifacts caused by the mismatch between the two scene generation devices are successfully eliminated, greatly improving the quality of the fused projection image. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a device for executing a method for fusion of projection images of a dual scene generation device according to the present invention;

[0047] Figure 2 This is the preliminary registration effect diagram;

[0048] Figure 3 Flowchart for accurate registration;

[0049] Figure 4 It is the flow chart of scale-invariant feature transformation algorithm;

[0050] Figure 5 This is the effect diagram of the scale-invariant feature transformation algorithm;

[0051] Figure 6 Flowchart of the matching mechanism for fast nearest neighbor search;

[0052] Figure 7 The effect diagram of the matching mechanism for fast nearest neighbor approximation search;

[0053] Figure 8 This is the flow chart of the random sampling consensus algorithm;

[0054] Fig. 9 This is the effect diagram of the random sampling consensus algorithm;

[0055] Fig.10 To accurately align the renderings;

[0056] Fig.11 This is the average pixel error map of the XY axis after precise alignment. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Combination Figures 1 to 11 .

[0059] The present invention is directed to a projection display device having a dual-channel common-aperture optical system for registering two projection images. Therefore, the dual-scene generator described in the present invention is not limited to a digital micromirror device, but can also be a liquid crystal display, a silicon-based liquid crystal, a resistor array, etc., as long as the model of the dual-scene generating device used for the parallel projection optical path is consistent and the resolution of the camera used to receive the projection image is higher than that of the dual-scene generating device.

[0060] The present invention proposes a fusion registration method for projection images of a dual scene generation device;

[0061] The device of the fusion registration method includes a projection module, a camera receiving module, an image registration module and an affine transformation module;

[0062] The projection module is composed of a dual-image generating device. After the cross-line source image is uploaded to the projection module, the dual-image generating device projects simultaneously, and the camera receiving module is kept fixed to receive the image projected by the dual-image generating device.

[0063] In this embodiment, two digital micromirror devices (DMDs) with a size of 1024×768 pixels are used as dual-image generation devices, and the image resolution collected by the camera receiving module is 2448×2048 pixels.

[0064] The image registration module is used to accurately register the reference image and the image to be registered. The overall flow chart of accurate registration is as follows: Figure 3 As shown, it consists of three parts: feature point extraction by scale-invariant feature transformation algorithm, initial matching by fast nearest neighbor approximation search matching mechanism, and precise matching by random sampling consistency algorithm.

[0065] The affine transformation module is used for affine transformation of the source image. The source image and the image after affine transformation of the source image are uploaded to the projection module and projected simultaneously. The camera receiving module collects again, and the projection images of the dual scene generation device are fused, and the artifacts disappear.

[0066] In the case where the structure of the projection module composed of the dual scene generation device does not change, the affine transformation module can be applied to the projection module in real time so that the fusion effect is maintained, whether it is a single frame projection of the image or a continuous multi-frame projection;

[0067] The fusion registration method of the dual scene generation device projection images specifically comprises the following steps:

[0068] Step 1: Preliminary registration: The camera receiving module collects the projection of the dual scene generator, collects the horizontal and vertical axis errors of the dual crosshairs, and performs preliminary mechanical debugging;

[0069] Upload a cross-line source image to the projection module, so that the dual scene generation devices are projected simultaneously, and the camera receiving module is fixed for acquisition; observe the horizontal and vertical axis errors of the acquired dual cross-lines, perform preliminary mechanical debugging, and finally control the average errors of the horizontal and vertical axes within about 50 pixels to complete the preliminary alignment; the effect is as follows Figure 2 shown.

[0070] Step 2, preprocessing: upload the source image with complex features to the projection module, the dual scene generation device projects respectively, the camera receiving module performs format conversion, and the obtained images are used as reference images and images to be registered respectively to complete the preprocessing;

[0071] In the embodiment, a source image with complex features is uploaded to the projection module, the dual scene generation devices are projected respectively, and the camera receiving module collects two 2448×2048 pixel size images and stores them in the host computer; the pixel size of the two collected images is converted to 1024×768 that conforms to the DMD pixel size; the obtained images are used as reference images and images to be registered respectively to complete the preprocessing.

[0072] Step 3: Accurate registration: Use the image registration module to accurately register the reference image and the image to be registered. The overall flow chart of accurate registration is as follows: Figure 3 As shown,

[0073] The precise registration process is as follows:

[0074] Step 31: firstly extract feature points of the reference image and the image to be registered according to the scale-invariant feature transformation algorithm; Figure 4 It is the flow chart of scale-invariant feature transformation algorithm;

[0075] Firstly, the Gaussian difference method is used to construct the scale space extreme value. Secondly, the extreme value points detected in the Gaussian scale space are fitted to determine the accurate position and scale of the feature points. At the same time, the feature points with low contrast and unstable edge response are removed to enhance the matching stability and improve the anti-noise ability.

[0076] Then, the direction of the feature points is assigned according to the local gradient of the image; finally, a descriptor generator is obtained according to the size and direction of the image gradient to calculate the feature descriptor of each feature point. Through the above steps, the feature point extraction of the reference image and the image to be registered is completed; Figure 5 This is the effect diagram of the scale-invariant feature transformation algorithm.

[0077] Step 32: Use the matching mechanism of fast nearest neighbor approximation search to perform initial matching on the extracted feature points;

[0078] Establish a fast nearest neighbor approximation search matching mechanism, Figure 6 Flowchart of the matching mechanism for fast nearest neighbor search;

[0079] For the feature points of the reference image and the image to be registered extracted in step 3.1, a set is established, and the feature points in the reference image are: , Feature points in the image to be registered: ,

[0080] The Euclidean distance is used to measure the similarity of feature points with 128 dimensions. The similarity measurement of any feature point pair is: ,To get the paired feature point pairs, Need to meet: In the reference image, the distance R i The nearest point S j , and the next closest point S p The ratio of the Euclidean distance of is less than a certain set threshold T;

[0081] The range of T is 0-1, which is an empirical value. When it is close to 0, the total matching amount decreases and the matching accuracy increases; when it is close to 1, the total matching amount increases, the number of mismatches also increases, and the matching accuracy decreases. In the embodiment, T is selected as 0.9 to ensure that a sufficient matching amount is obtained; after the above steps, the initial matching of the feature points of the reference image and the image to be registered is completed; Figure 7 A diagram showing the matching mechanism for fast nearest neighbor search.

[0082] Step 33: Random sampling consistency algorithm, iteratively calculate the optimal homography matrix parameters to complete the precise matching of feature points.

[0083] Using a random sampling consensus algorithm, Figure 8 This is the flow chart of the random sampling consensus algorithm;

[0084] For the feature points that are initially matched successfully in step 3.2, 4 pairs are randomly selected as samples, and the homography matrix parameters are calculated iteratively for multiple times to eliminate incorrect matching pairs and improve the matching accuracy, so that the corresponding cost function is minimized, and the final precise matching and optimal homography matrix parameters are obtained to complete the precise matching of the feature points of the reference image and the image to be registered; Fig. 9 This is the effect diagram of the random sampling consensus algorithm;

[0085] The homography matrix is ​​expressed as:

[0086]

[0087] in( μ 1, ν 1, 1) is the homogeneous coordinates of the feature points of the reference image, ( μ 2, ν 2, 1) are the homogeneous coordinates of the feature points of the image to be registered, h 11 - h 33 are the parameters of the homography matrix;

[0088] The cost function is expressed as:

[0089] .

[0090] Step 4, affine transformation: using an affine transformation module to perform affine transformation on the source image uploaded to one of the scene generation devices according to the homography matrix, and the source image uploaded to the other scene generation device remains unchanged, and is projected again by the projection module;

[0091] In step 4, the source image uploaded to one of the scene generation devices is affine transformed according to the homography matrix using the affine transformation module, and the source image and the image after the affine transformation of the source image are uploaded to the projection module, and are projected simultaneously by the first scene generation device and the second scene generation device, respectively, and the camera receiving module collects again; Fig.10 To accurately register the renderings, eventually, the artifacts disappear;

[0092] Step 5: Verify the registration accuracy: Repeat steps 2 to 4 to determine whether the homography matrix is ​​valid.

[0093] In step five, in order to verify the final registration accuracy, we start from step two again, collect the projection images of the dual-view generation device, and after size conversion, perform feature point extraction, preliminary matching, and fine matching again, and record the coordinates of 20 pairs of matched feature points for comparison. When the image registration accuracy is less than 1 pixel, the homography matrix is ​​determined to be valid, such as Fig.11As shown in the figure, it is the XY axis pixel error map after accurate registration. The average pixel error of the X axis is calculated to be 0.23, and the average pixel error of the Y axis is 0.21, both within 1 pixel, achieving high registration accuracy, and judging that the homography matrix is ​​valid;

[0094] Step 6, continuous multi-frame projection: When the subsequent source images are projected, before being uploaded to the two scene generation devices, one of them needs to be affine transformed using the homography matrix first, and then the source image and the transformed source image are uploaded to the two scene generation devices respectively and projected, so as to realize the fusion and registration of the projection images of the two scene generation devices.

[0095] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0096] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, or a flash memory. The volatile memory may be a random access memory, RAM, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory static RAM, SRAM, dynamic random access memory dynamic RAM, DRAM, synchronous dynamic random access memory synchronous DRAM, SDRAM, double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM, enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM, synchronous connection dynamic random access memory synchlink DRAM, SLDRAM and direct memory bus random access memory direct rambus RAM, DR RAM. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0097] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center through a wired method such as coaxial cable, optical fiber, digital subscriber line digital subscriber line, DSL or wireless such as infrared, wireless, microwave, etc. to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium such as a floppy disk, a hard disk, a tape, an optical medium such as a high-density digital video disc digital video disc, DVD, or a semiconductor medium such as a solid state hard disk solid state disc, SSD, etc.

[0098] The above is a detailed introduction to the fusion registration method of the projection images of a dual-scene generating device proposed in the present invention, and the principles and implementation methods of the present invention are explained. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A fusion registration method for projection images of a dual scene generation device, characterized in that: The method specifically comprises the following steps: Step 1: Preliminary registration: The camera receiving module collects the projection of the dual scene generator, collects the horizontal and vertical axis errors of the dual crosshairs, and performs preliminary mechanical debugging; Step 2, preprocessing: upload the source image with complex features to the projection module, the dual scene generation device projects respectively, the camera receiving module performs format conversion, and the obtained images are used as reference images and images to be registered respectively to complete the preprocessing; Step 3: Accurate registration: Use the image registration module to accurately register the reference image and the image to be registered. The accurate registration process is as follows: Step 31: firstly extract feature points of the reference image and the image to be registered according to the scale-invariant feature transformation algorithm; Step 32: Use the matching mechanism of fast nearest neighbor approximation search to perform initial matching on the extracted feature points; Step 33: Random sampling consistency algorithm, iteratively calculate the optimal homography matrix parameters, and complete the precise matching of feature points; Step 4, affine transformation: use the affine transformation module to perform affine transformation on the source image uploaded to one of the scene generation devices according to the homography matrix, and the source image uploaded to the other scene generation device remains unchanged, and is projected again through the projection module, so that the artifact disappears; Step 5: Verify the registration accuracy: Repeat steps 2 to 4 to verify the registration accuracy and determine whether the homography matrix is ​​valid. Step 6, continuous multi-frame projection: When the subsequent source images are projected, before being uploaded to the two scene generation devices, one of them needs to be affine transformed using the homography matrix first, and then the source image and the transformed source image are uploaded to the two scene generation devices respectively and projected, so as to realize the fusion and registration of the projection images of the two scene generation devices.

2. The registration method according to claim 1, characterized in that: In step one, Upload a cross-line source image to the projection module, so that the dual scene generation devices are projected simultaneously, and the camera receiving module is fixed for acquisition; Observe the horizontal and vertical axis errors of the collected double cross lines, perform preliminary mechanical debugging, and finally control the average errors of the horizontal and vertical axes within 50 pixels to complete the preliminary alignment.

3. The registration method according to claim 2, characterized in that: In step 2, A source image with complex features is uploaded to the projection module, and the dual scene generation devices are projected separately. The camera receiving module collects two images and stores them in the host computer; the pixel size of the two collected images is converted into the pixel size that conforms to the scene generation device; the obtained images are used as reference images and images to be registered to complete the preprocessing.

4. The registration method according to claim 3, characterized in that: In step 31, specifically: Firstly, the Gaussian difference method is used to construct the scale space extreme value; secondly, the extreme value points detected in the Gaussian scale space are fitted to determine the exact position and scale of the feature points; Then the feature point direction is assigned according to the local gradient of the image; finally, according to the image gradient size and direction, a descriptor generator is obtained to calculate the feature descriptor of each feature point.

5. The registration method according to claim 4, characterized in that: In step 32, Establish a matching mechanism for fast nearest neighbor approximation search For the feature points of the reference image and the image to be registered extracted in step 3.1, a set is established, and the feature points in the reference image are: , Feature points in the image to be registered: , The Euclidean distance is used to measure the similarity of feature points with 128 dimensions. The similarity measurement of any feature point pair is: ,To get the paired feature point pairs, Need to meet: In the reference image, the distance R i The nearest point S j , and the next closest point S p The ratio of the Euclidean distance of is less than a certain set threshold T; The range of T is 0~1, which is an empirical value. When it is close to 0, the total number of matches decreases and the matching accuracy increases; when it is close to 1, the total number of matches increases, the number of false matches also increases, and the matching accuracy decreases.

6. The registration method according to claim 5, characterized in that: In step 33, A random sampling consensus algorithm is used; For the feature points that are initially matched successfully in step 3.2, 4 pairs are randomly selected as samples, and the homography matrix parameters are calculated iteratively for multiple times to eliminate incorrect matching pairs and improve the matching accuracy, so that the corresponding cost function is minimized, and the final precise matching and optimal homography matrix parameters are obtained to complete the precise matching of the feature points of the reference image and the image to be registered; The homography matrix is ​​expressed as: in( μ 1, ν 1, 1) is the homogeneous coordinates of the feature points of the reference image, ( μ 2, ν 2, 1) are the homogeneous coordinates of the feature points of the image to be registered, h 11 - h 33 are the parameters of the homography matrix; The cost function is expressed as: In every random sample of 4 pairs (n=4): is the horizontal X-axis coordinate of the feature point that is successfully matched in the reference image. The Y-axis coordinate of the feature point that is initially matched successfully in the reference image; is the horizontal X-axis coordinate of the feature point of the image to be registered. It is the Y-axis coordinate of the feature point of the image to be registered that is successfully matched.

7. The registration method according to claim 6, characterized in that: In step 4, the source image uploaded to one of the scene generation devices is affine transformed according to the homography matrix using the affine transformation module, and the source image and the image after the affine transformation of the source image are uploaded to the projection module, and are projected simultaneously by the first scene generation device and the second scene generation device, respectively, and the camera receiving module collects again; Finally, the projected images of the dual scene generation devices are fused and the artifacts disappear; In step five, in order to verify the final registration accuracy, start again from step two, collect the projection images of the dual-view generation device separately, and after size conversion, perform feature point extraction, preliminary matching, and fine matching again, and record the coordinates of 20 pairs of matched feature points for comparison. When the image registration accuracy is less than 1 pixel, the required homography matrix is ​​determined to be valid.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images