Image stitching method and system of camera and image stitching camera

Abstract

The invention discloses a camera image splicing method and system and an image splicing camera, and relates to the technical field of image data, and the method comprises the steps: receiving images shot by a lens module, and forming an image set; inputting the image set into a trained feature alignment model, and obtaining feature information of each target image; selecting adjacent feature information according to the target position and extracting intersecting feature coordinates, and repeating until all feature coordinates are extracted; sequentially selecting two adjacent images in the image set as a reference image and a target image, and calculating a feature offset according to feature coordinates of the adjacent images; performing projection transformation on the target image according to the feature offset to obtain an aligned target image; and inputting the aligned target image and reference image into an image splicing model for splicing. The method has the technical effects that the image splicing efficiency is improved, and splicing traces and image artifacts are reduced.

Description

Technical Field

[0001] This application relates to the field of image communication technology, and in particular to a camera image stitching method, system, and image stitching camera. Background Technology

[0002] In current technology, with the rapid development of science and technology, image stitching cameras are playing an increasingly important role in many fields. In the field of security monitoring, traditional single cameras, due to their limited field of view, are unable to provide comprehensive coverage of large areas. Image stitching cameras, however, can capture images from different angles using multiple lenses and stitch them together into a complete panoramic image, effectively eliminating blind spots and greatly improving monitoring efficiency and security. For example, in places requiring panoramic monitoring, such as underground coal mines, image stitching cameras can monitor personnel movement, machine operation, and other situations in real time from all directions, providing strong support for safety management.

[0003] With the deepening development of security management, the requirements for high-precision and high-pixel image stitching technology are becoming increasingly stringent. Existing image stitching technologies suffer from unstable feature extraction, which is prone to errors and affects the accuracy of image stitching. Furthermore, current deep learning stitching algorithms have long training times, and their deep learning stitching models can only stitch two images, not multiple images. Moreover, the stitched images cannot achieve realistic texture effects and exhibit artifacts and obvious stitching marks, which are urgent technical problems that need to be solved. Summary of the Invention

[0004] Therefore, the technical problem to be solved by this application is to improve the existing technology, which cannot stitch multiple images together, and the stitched image cannot achieve a realistic texture effect, and has artifacts and obvious stitching marks.

[0005] To address the aforementioned technical problems, in a first aspect, this application provides a camera image stitching method, comprising: The receiving lens module calibrates the object and captures images from different angles to form a target image set; The target image set is input into the trained feature alignment model to obtain the feature information of each target image in the target image set; In each target image, adjacent feature information is selected based on the target location and the intersecting feature coordinates are extracted. This step is repeated until all feature coordinates of the target image set are extracted. Two adjacent images from the target image set are selected sequentially as the reference image and the target image, and the feature offset between the reference image and the target image is calculated based on the feature coordinates of the adjacent reference image and the target image. The target image is projected and transformed according to the feature offset to obtain the aligned target image. This step is repeated until all target images are transformed into aligned target images. The aligned target image and the reference image are sequentially input into the image stitching model for stitching until all aligned target images are stitched together to obtain a stitched image.

[0006] The lens module calibrates the object and captures images from different angles, including: The lens module is moved to calibrate the object to be photographed. The lens module is controlled to take pictures of the calibrated object at a preset angle to obtain multi-angle images of the object.

[0007] Preferably, the training method for the feature alignment model includes: Construct a feature alignment model; Select a training image set, extract features from the training image set to obtain image pairs, and form dataset 1. Obtain image pairs with different proportions and disparity in the image set to form dataset 2. The image pairs include training reference images and training target images. The dataset 1 is used as a sample set and input into the feature alignment model; The images in the sample set are subjected to feature extraction, and the (a+1)×(b+1)×2 feature offset is calculated based on the extracted features. Based on the feature offset, construct a×b transformation matrices, evenly divide the training target image into a×b image blocks, use the a×b transformation matrices to perform projection transformation on the corresponding image blocks respectively, and then stitch them together to obtain the aligned training target image. Adjust the parameters of the feature alignment model by the similarity of the overlapping regions of the training target image and the training reference image. The dataset 2 is used as the sample set and input into the feature alignment model for training until the feature extraction model is trained.

[0008] Preferably, the feature alignment model includes a convolutional layer for extracting feature information from all images in the target image set; Two branches with the same structure receive feature information from the reference image and the target image, and extract the intersecting feature coordinates. Each branch includes multiple convolutional layers and multiple CSP modules. Each CSP module outputs a feature map and uses the feature map as input to the next CSP module. After the feature maps output by the CSP module of the same layer of the two branches are concatenated in the channel direction, the feature coordinates of the concatenated result are optimized by several convolutional layers. Then, a regression network composed of average pooling layer and fully connected layer is used to regress and calculate a feature offset based on the optimization result.

[0009] Preferably, the process of constructing the image stitching model includes: An image stitching model is established, wherein the image stitching model adopts a generative adversarial network, and the generative adversarial network includes a generator and a discriminator; A set of adversarial images is selected, and image pairs with different proportions and parallax are obtained from the set to form dataset 3. Dataset 3 is input into the trained feature alignment model to obtain the aligned target image. The real image in dataset 3 is set as label 1, and the stitched image generated during the generation period is set as label 2. The aligned target image and reference image are directly stacked, and the pixel value of the overlapping area is replaced by the average value of the pixels of the two images to obtain the superimposed image. The superimposed image is set as label 2, and thus the training dataset is obtained. The training dataset is input into the adversarial generative network, and the adversarial generative loss function is used to train the adversarial generative network until the training is completed. The generator in the trained adversarial generative network is used as the image stitching model.

[0010] Preferably, the generator adopts an encoder-decoder structure.

[0011] Preferably, the generator is formed by sequentially connecting multiple convolutional layers and deconvolutional layers of the same number.

[0012] Secondly, this application provides a camera image stitching system, the system comprising: Receiver module: Used to receive images of the object calibrated by the lens module and capture images from different angles to form a target image set; Feature information acquisition module: used to input the target image set into the trained feature alignment model to acquire feature information of each target image in the target image set; Feature coordinate extraction module: In each target image, it selects adjacent feature information based on the target location and extracts the intersecting feature coordinates, repeating this step until all feature coordinates of the target image set are extracted; Feature offset calculation module: used to sequentially select two adjacent images in the target image set as reference images and target images, and calculate the feature offset between the reference image and the target image based on the feature coordinates of the adjacent reference image and target image; Projection alignment image module: used to perform projection transformation on the target image according to the feature offset to obtain an aligned target image, and repeat this step until all target images are transformed into aligned target images; Image stitching module: used to sequentially input the aligned target image and the reference image into the image stitching model for stitching until all aligned target images are stitched together to obtain a stitched image.

[0013] Thirdly, this application also provides an image stitching camera, comprising: The lens module is used to calibrate the object to be photographed and capture images from different angles. A control module, which communicates with the lens module via wired and wireless communication, is used to execute any of the above-mentioned camera image stitching methods.

[0014] In summary, this application includes at least one of the following beneficial technical effects: The image stitching method described in this application optimizes the structure of the feature alignment model in the alignment stage of the image stitching algorithm, reduces the number of parameters in the feature alignment model, and makes the structure of the feature alignment model more suitable for image alignment tasks. This reduces the consumption of computing resources, improves the training speed, convergence speed, and inference speed of the network, and realizes image stitching of image sets, effectively avoiding the problem of insufficient geometric structure in scanned image sets. Furthermore, this application uses an image stitching model constructed with a generative adversarial network to realize the fusion step of image stitching. Compared with traditional stitching methods, this application can learn real-world image features, making the overlapping and transition areas of the final stitched image more natural, greatly reducing stitching marks and image artifacts, and making it more suitable for use in complex working conditions in coal mines. Attached Figure Description

[0015] To make the content of this application easier to understand, the following detailed description is provided based on specific embodiments and accompanying drawings, wherein: Figure 1 This is a flowchart of a preferred embodiment of the camera image stitching method of this application; Figure 2 This is a schematic diagram of the feature alignment model in this application; Figure 3 This is a schematic diagram of the image stitching model in this application; Figure 4 This is a schematic diagram of the camera image stitching system in this application. Detailed Implementation

[0016] The following is in conjunction with the appendix Figure 1-4 This application will be described in further detail.

[0017] After reading this specification, those skilled in the art may make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0020] This application proposes a camera image stitching method. This method optimizes the structure of the feature alignment model in the alignment stage of the image stitching algorithm, reduces the number of parameters of the feature alignment model, and makes the structure of the feature alignment model more suitable for image alignment tasks. It reduces the occupation of computing resources, improves the training speed, convergence speed and inference speed of the network, and improves the clarity of the stitched image.

[0021] like Figure 1 As shown, a method for stitching images from a camera includes the following steps: S101: Receives images from the lens module calibrating the object and captures images from different angles to form a target image set; S102: Input the target image set into the trained feature alignment model to obtain the feature information of each target image in the target image set; S103: In each target image, select adjacent feature information based on the target location and extract the intersecting feature coordinates. Repeat this step until all feature coordinates of the target image set are extracted. S104: Select two adjacent images from the target image set as the reference image and the target image in sequence, and calculate the feature offset between the reference image and the target image based on the feature coordinates of the adjacent reference image and the target image. S105: Perform a projection transformation on the target image based on the feature offset to obtain the aligned target image. Repeat this step until all target images are transformed into aligned target images. S106: Input the aligned target image and reference image into the image stitching model in sequence for stitching until all aligned target images are stitched together to obtain the stitched image.

[0022] The image stitching method described in this embodiment extracts features from all images in the image set captured by the camera. Then, based on the feature offsets of adjacent images, it performs projection transformations on the images. Finally, it stitches the images together using an image stitching model to obtain the stitched image. The method described in this embodiment reduces the consumption of computational resources, improves the training speed, convergence speed, and inference speed of the network, and achieves image stitching of the image set, effectively avoiding the problem of insufficient geometric structure in the scanned image set.

[0023] In this embodiment, preferably, step S101 specifically includes: S1011: Controls the movement of the lens module to calibrate the object being photographed.

[0024] The lens module includes laser 3D cameras, etc., and the object is calibrated by adjusting the shooting angle of the camera lens.

[0025] S1012: Controls the lens module to take pictures of the calibrated object according to the preset angle, and obtains multi-angle images of the object.

[0026] The preset lens module is controlled to take pictures of the calibrated object at preset angles. The preset angles are images taken from different angles in 3D space to ensure that the calibration block covers the entire space within the scanning area and completes the image taking according to the preset angle requirements, providing a basis for subsequent feature extraction and image stitching. The scanning area is the area where image stitching is required.

[0027] S1013: Receives multi-angle images of the object to be photographed, forming a target image set.

[0028] In this embodiment, preferably, the training method for the feature alignment model described in step S102 includes steps Sa-Sf, as follows: Sa: Construct a feature alignment model.

[0029] In this embodiment, as Figure 2As shown, the feature alignment model includes multiple convolutional layers for extracting image features and two branches with identical structures (feature offset calculation layers). The network parameters of the branches are consistent. The two branches are used to extract feature maps of the reference image and the target image, respectively. Each branch includes a convolutional layer and multiple CSP modules. Each CSP module outputs a feature map and inputs the feature map into the next CSP module. The feature maps output by the same CSP module of the two branches are concatenated in the channel direction. Several convolutional layers are used to extract and optimize the features of the concatenated result. Then, a regression network composed of average pooling layers and fully connected layers is used to regress and calculate a feature offset based on the optimization result. Figure 2 The example shows two CSP modules, which generate a total of two feature offsets.

[0030] Specifically, each CSP module also includes two sub-branches. Sub-branch one consists of a CBS module with a convolutional layer, a batch normalization layer, a SiLU activation layer, a ResNet residual unit, and a convolutional layer connected in sequence. It is used to extract feature maps. Sub-branch two has only one independent convolutional layer, which is used to extract feature maps. Then, the feature maps of the two sub-branches are concatenated and input into the batch normalization layer, the Leaky ReLU activation layer, and the CBS module. The feature maps are then calculated and output.

[0031] Sb: Select a training image set, extract features from the training image set to obtain image pairs, forming dataset 1, and obtain image pairs with different proportions and disparity in the image set to form dataset 2. The image pairs include training reference images and training target images.

[0032] In this embodiment, a publicly available image dataset is selected as the training image set, such as the MSCOCO dataset. Image pairs are obtained by cropping and transforming the images to form dataset 1. Specifically, any image in the MSCOCO dataset is selected and cropped to obtain image pairs of 64×64 size with different overlapping areas. A random projection transformation is applied to one of the images to obtain a set of image pairs to be stitched together.

[0033] In this embodiment, image pairs with varying overlap ratios and parallax, collected in the real world, are used to form dataset 2.

[0034] Sc: Input dataset 1 as the sample set into the feature alignment model.

[0035] Sd: Extract features from the images in the sample set, and calculate the (a+1)×(b+1)×2 feature offset based on the extracted features.

[0036] In this embodiment, as Figure 2As shown, after each input image passes through the CSP module, the feature maps of the two branches are concatenated and fed into a regression network consisting of an average pooling layer and a fully connected layer, outputting a feature offset of size (a+1)×(b+1)×2. Specifically, this application sets the image transformation to be represented by an a×b grid transformation, therefore the coordinate offset of the grid vertex can be represented by a tensor of size (a+1)×(b+1)×2.

[0037] Se: Construct a×b transformation matrices based on the feature offsets, evenly divide the training target image into a×b image blocks, use the a×b transformation matrices to perform projection transformations on the corresponding image blocks respectively, and then stitch them together to obtain the aligned training target image. Adjust the parameters of the feature alignment model by the similarity of the overlapping regions of the training target image and the training reference image.

[0038] Sf: Use dataset 2 as the sample set, input the feature alignment model for training, until the feature extraction model is trained.

[0039] In this embodiment, specifically, dataset 2 is used as the sample set, and the process jumps to step Sd, and steps Sd and Se are repeated to fine-tune the network parameters of the feature alignment model and obtain the trained feature alignment model.

[0040] In this embodiment, preferably, in step S103, a line feature extraction algorithm can be used to extract the intersecting feature coordinates. These feature coordinates will be used for feature offset acquisition in subsequent steps. Through the above settings, the system can acquire different feature information on each image and extract the corresponding feature coordinates, laying the foundation for subsequent image stitching and 3D reconstruction.

[0041] In this embodiment, preferably, the image stitching model employs an adversarial generative network structure, using least-squares loss to train the generator to automatically learn how to fuse aligned images to obtain the stitched image. For example... Figure 3 As shown, the image stitching model construction process described in step S106 includes steps one to three, as detailed below: Step 1: Establish an image stitching model. The image stitching model uses a generative adversarial network, which includes a generator and a discriminator.

[0042] In this embodiment, preferably, the generator adopts an encoder-decoder structure, which can be obtained by sequentially connecting multiple convolutional layers and deconvolutional layers of the same number, with each convolutional layer and deconvolutional layer followed by a batch normalization layer.

[0043] Step 2: Select an adversarial image set, obtain image pairs with different proportions and parallax from the adversarial image set, form dataset 3, input dataset 3 into the trained feature alignment model to obtain the aligned target image, set the real image in dataset 3 as label 1, set the stitched image generated during the generation period as label 2, directly stack the aligned target image and reference image, replace the pixel value of the overlapping area with the average value of the pixels of the two images to obtain the superimposed image, set the superimposed image as label 2, and thus obtain the training dataset; Step 3: Input the training dataset into the adversarial generative network (PGN), and train the GPN using the adversarial generative loss function until the GPN is trained. Use the generator in the trained GPN as the image stitching model.

[0044] After training, the trained generator is used as an image stitching model. When applied, the target image and reference image to be aligned are input into the image stitching model, and the stitched image is output after calculation.

[0045] The model is trained using unsupervised learning and an image stitching model is built using an adversarial generative network structure. This allows for rapid training of a high-precision model without the need for labeled data, thus improving the speed and accuracy of image stitching.

[0046] The image stitching method described in this application optimizes the structure of the feature alignment model in the alignment stage of the image stitching algorithm, reduces the number of parameters in the feature alignment model, and makes the structure of the feature alignment model more suitable for image alignment tasks. This reduces the consumption of computing resources, improves the training speed, convergence speed, and inference speed of the network, and realizes image stitching of image sets, effectively avoiding the problem of insufficient geometric structure in scanned image sets. Furthermore, this application uses an image stitching model constructed with a generative adversarial network to realize the fusion step of image stitching. Compared with traditional stitching methods, this application can learn real-world image features, making the overlapping and transition areas of the final stitched image more natural, greatly reducing stitching marks and image artifacts, and making it more suitable for use in complex working conditions in coal mines.

[0047] like Figure 4 As shown, this application also discloses a camera image stitching system, comprising: Receiver module 100: Used to receive images of the object calibrated by the lens module and capture images from different angles to form a target image set; Feature information acquisition module 200: used to input the target image set into the trained feature alignment model to acquire feature information of each target image in the target image set; Feature coordinate extraction module 300: In each target image, it selects adjacent feature information according to the target position and extracts the intersecting feature coordinates, repeating this step until all feature coordinates of the target image set are extracted; Feature offset calculation module 400: used to sequentially select two adjacent images in the target image set as reference images and target images, and calculate the feature offset between the reference images and target images based on the feature coordinates of the adjacent reference images and target images; Projection alignment image module 500: used to perform projection transformation on the target image according to the feature offset to obtain an aligned target image, and repeat this step until all target images are transformed into aligned target images; Image stitching module 600: used to sequentially input the aligned target image and the reference image into the image stitching model for stitching until all aligned target images are stitched together to obtain a stitched image.

[0048] This application also discloses an image stitching camera, including: Lens module: The lens module is used to calibrate the object to be photographed and capture images from different angles; Specifically, the lenses that can be used for shooting include: Hikvision KBA7.4 (A)-6E45-XZSW, Hengyu Automation KBA-12R, Langfang Uniview KBA12Y, Nanyang Zhongtong Intelligent TC-600, Infineon V1130 series, Changzhou Luming LM-KY003 EX, etc.

[0049] The control module communicates with the lens module via wired and wireless communication to execute the camera image stitching method described above.

[0050] In the description of this specification, any process or method described in the flowcharts or otherwise herein may be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0051] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0052] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0053] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0054] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0055] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0056] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0057] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for stitching images from a camera, characterized in that, include: The receiving lens module calibrates the object and captures images from different angles to form a target image set; The target image set is input into the trained feature alignment model to obtain the feature information of each target image in the target image set; In each target image, adjacent feature information is selected based on the target location and the intersecting feature coordinates are extracted. This step is repeated until all feature coordinates of the target image set are extracted. Two adjacent images from the target image set are selected sequentially as the reference image and the target image, and the feature offset between the reference image and the target image is calculated based on the feature coordinates of the adjacent reference image and the target image. The target image is projected and transformed according to the feature offset to obtain the aligned target image. This step is repeated until all target images are transformed into aligned target images. The aligned target image and the reference image are sequentially input into the image stitching model for stitching until all aligned target images are stitched together to obtain a stitched image.

2. The image stitching method for a camera according to claim 1, characterized in that, The lens module calibrates the object and captures images from different angles, including: The lens module is moved to calibrate the object to be photographed. The lens module is controlled to take pictures of the calibrated object at a preset angle to obtain multi-angle images of the object.

3. The image stitching method for a camera according to claim 1, characterized in that, The training method for the feature alignment model includes: Construct a feature alignment model; Select a training image set, extract features from the training image set to obtain image pairs, and form dataset 1. Obtain image pairs with different proportions and disparity in the image set to form dataset 2. The image pairs include training reference images and training target images. The dataset 1 is used as a sample set and input into the feature alignment model; The images in the sample set are subjected to feature extraction, and the (a+1)×(b+1)×2 feature offset is calculated based on the extracted features. Based on the feature offset, construct a×b transformation matrices, evenly divide the training target image into a×b image blocks, use the a×b transformation matrices to perform projection transformation on the corresponding image blocks respectively, and then stitch them together to obtain the aligned training target image. Adjust the parameters of the feature alignment model by the similarity of the overlapping regions of the training target image and the training reference image. The dataset 2 is used as the sample set and input into the feature alignment model for training until the feature extraction model is trained.

4. The image stitching method for a camera according to claim 3, characterized in that, The feature alignment model includes convolutional layers for extracting feature information from all images in the target image set; Two branches with the same structure receive feature information from the reference image and the target image, and extract the intersecting feature coordinates. Each branch includes multiple convolutional layers and multiple CSP modules. Each CSP module outputs a feature map and uses the feature map as input to the next CSP module. After the feature maps output by the CSP module of the same layer of the two branches are concatenated in the channel direction, the feature coordinates of the concatenated result are optimized by several convolutional layers. Then, a regression network composed of average pooling layer and fully connected layer is used to regress and calculate a feature offset based on the optimization result.

5. The image stitching method for a camera according to claim 1, characterized in that: The process of constructing the image stitching model includes: An image stitching model is established, wherein the image stitching model adopts a generative adversarial network, and the generative adversarial network includes a generator and a discriminator; A set of adversarial images is selected, and image pairs with different proportions and parallax are obtained from the set to form dataset 3. Dataset 3 is input into the trained feature alignment model to obtain the aligned target image. The real image in dataset 3 is set as label 1, and the stitched image generated during the generation period is set as label 2. The aligned target image and reference image are directly stacked, and the pixel value of the overlapping area is replaced by the average value of the pixels of the two images to obtain the superimposed image. The superimposed image is set as label 2, and thus the training dataset is obtained. The training dataset is input into the adversarial generative network, and the adversarial generative loss function is used to train the adversarial generative network until the training is completed. The generator in the trained adversarial generative network is used as the image stitching model.

6. The image stitching method for a camera according to claim 5, characterized in that: The generator adopts an encoder-decoder structure.

7. The image stitching method for a camera according to claim 6, characterized in that: The generator is formed by sequentially connecting multiple convolutional layers and deconvolutional layers of the same number.

8. A camera image stitching system, characterized in that, The system includes: Receiver module: Used to receive images of the object calibrated by the lens module and capture images from different angles to form a target image set; Feature information acquisition module: used to input the target image set into the trained feature alignment model to acquire feature information of each target image in the target image set; Feature coordinate extraction module: In each target image, it selects adjacent feature information based on the target location and extracts the intersecting feature coordinates, repeating this step until all feature coordinates of the target image set are extracted; Feature offset calculation module: used to sequentially select two adjacent images in the target image set as reference images and target images, and calculate the feature offset between the reference images and target images based on the feature coordinates of the adjacent reference images and target images; Projection alignment image module: used to perform projection transformation on the target image according to the feature offset to obtain an aligned target image, and repeat this step until all target images are transformed into aligned target images; Image stitching module: used to sequentially input the aligned target image and the reference image into the image stitching model for stitching until all aligned target images are stitched together to obtain a stitched image.

9. An image stitching camera, characterized in that, include: The lens module is used to calibrate the object to be photographed and capture images from different angles. A control module, which communicates with the lens module via wired and wireless communication, is used to execute the camera image stitching method as described in any one of claims 1-7.

Images