Four-camera panoramic stitching method

Through the four-eye camera, images are collected and SIFT algorithm and background modeling are used to realize real-time identification and tracking of mobile targets, solving the problem of insufficient splicing traces and dynamic matching in the existing technology, and improving the real-time performance of panoramic images and User experience.

CN119071412APending Publication Date: 2024-12-03JIANGSU RUISHILANG INTELLIGENT EQUIP CO LTD

Patent Information

Application Number
CN202411177672.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing four-eye camera panoramic stitching technology is difficult to update the status of the mobile target in real time, and there are splicing traces, so it is impossible to effectively track the mobile target.

Method used

Image information is collected through the quad-eye camera and input into the CMOS image sensor. Key points are extracted using SIFT algorithm for image matching, dynamic targets are identified and fused into panoramic images to achieve dynamic image stitching.

Benefits of technology

Real-time identification and tracking of mobile targets is achieved, scribing traces are eliminated, real-time and dynamic expressiveness of panoramic images are improved, and a natural and smooth visual experience is provided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071412A_ABST
    Figure CN119071412A_ABST
Patent Text Reader

Abstract

The invention relates to a four-eye camera panoramic stitching method, which comprises the following steps of: acquiring image information in the same space through a four-eye camera, and importing the image information into a CMOS (Complementary Metal-Oxide-Semiconductor Transistor) image sensor; the image information optical signal is converted into a digital image signal by using a CMOS image sensor, and then the digital image signal is input into a DSP digital signal processing chip for image information preprocessing; key point extraction is carried out on digital image information in a DSP digital signal processing chip collected by a four-eye camera at different angles through an SIFT algorithm, panoramic image information in the four-eye camera is matched based on key point extraction, and a first spliced panoramic image output in a matched mode is obtained; identifying a dynamic target by calculating a pixel difference between the first spliced panoramic image and the second spliced panoramic image of the adjacent frame; and constructing a panoramic image background by using a background modeling method, fusing the dynamic target as a foreground into the constructed panoramic image background, and splicing a dynamic image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a four-camera panoramic stitching method. Background Art

[0002] At present, electronic devices with image acquisition functions have limited shooting angles. For example, the main camera of a mobile phone has an angle of view of 80°-90°, while the wide-angle camera of a professional camera is less than 160°, and there are blind spots in the camera angle. In response to this, there are also methods of setting the camera to a fisheye lens to acquire panoramic images, which enhances the capture of the lens field of view and achieves the effect of all-round photography.

[0003] However, the current imaging solution of cameras is to present panoramic images in a circular form with a full 360° angle, which places high demands on the image stitching algorithm for panoramic imaging. Although there are technical processing methods such as overlapping area fusion and color correction in this direction, the distortion effect of the fisheye lens itself will cause image distortion, and the objects in the image will be stretched and deformed, which will be inconsistent with reality.

[0004] Based on this technology, CN112581371A - A panoramic real-time imaging stitching method based on a novel structure of four-way camera, which takes images through four wide-angle lenses and inputs the images to a MINI PC through a USB hub. According to the perspective transformation principle, the four-point coordinates of the rectangle to be measured in the source image are determined, and then the four-point coordinates of the rectangle in the target image are determined. The image information is transformed into a matrix through perspective transformation, and then the transformed matrix is ​​input into a perspective transformation function interface to obtain a corresponding transformed image; after processing the pictures taken by the four cameras in turn, all the transformed images are stitched together to obtain a complete panoramic picture; the invention solves the problem of blind spots in the current four-way input 360-degree panoramic picture shooting, and can display the picture in real time; but in terms of dynamic matching of objects, the scheme only captures and processes the first frame image of the object, and cannot update the state of the moving target in real time, and cannot eliminate the stitching traces, resulting in the inability to track the moving target. Summary of the invention

[0005] The main purpose of the present invention is to provide a four-camera panoramic stitching method, which aims to update the state of a moving target in real time, eliminate stitching traces, and achieve tracking of the moving target.

[0006] To achieve the above object, the present invention provides a four-camera panoramic stitching method, comprising the following steps: Collecting image information in the same space through four cameras, and importing the image information into a CMOS image sensor, wherein the four cameras include cameras corresponding to preset focal lengths; The received image information optical signal is converted into an image information electrical signal by using the pixel array of the photoelectric conversion area in the CMOS image sensor, and the readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; Using a CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; The SIFT algorithm is used to extract key points of the digitized image information in the DSP digital signal processing chip collected by the four cameras at different angles, and the panoramic image information in the four cameras is matched based on the key point extraction, and a first stitched panoramic image output by the matching is obtained, where the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; Identifying a dynamic target by calculating a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame; A panoramic image background is constructed by using a background modeling method, and the dynamic target is integrated into the constructed panoramic image background as a foreground to splice a dynamic image.

[0007] Furthermore, the pixel array in the photoelectric conversion area of ​​the CMOS image sensor converts the received image information optical signal into an image information electrical signal, including: The four-eye camera collects image light and transmits it to the CMOS image sensor array pixels. The image light passes through the color filter channel on the array pixels to fill the original black and white image to obtain a color image. The photodiode on each pixel converts the incident photons into electric charges. A single color filter is placed on each pixel of the pixel array, and is arranged in an alternating manner to combine the proportions of 1 / 4, 1 / 4 blue and 1 / 2 green.

[0008] Furthermore, the method of inputting the digitized image signal into the DSP digital signal processing chip to perform digitized image information preprocessing includes: Digital image information is processed using image grayscale processing and geometric transformation algorithms.

[0009] The color digitized image information is grayed out by weighted averaging, and the RGB value of each pixel is converted into a single gray value, highlighting the edge and texture of the image; The digital image information after grayscale processing is subjected to geometric transformation algorithm operation, the digital image pixel matrix of the four-eye camera at different camera angles is identified, and the overlapping area of ​​the digital images at different camera angles is obtained; through the geometric transformation operations of translation, scaling, rotation, and mirroring, the consistency of the size and angle of the digital images of the four-eye camera at different camera angles is constrained.

[0010] Furthermore, the SIFT algorithm extracts key points of the digitized image information, and matches the panoramic image information in the four-eye camera based on the key point extraction, and obtains the first stitched panoramic image output by the matching, including: The SIFT algorithm is used to extract feature points from the relationship between the digitized images preprocessed with the preset focal length composition; The digitized images at different camera angles are matched according to the feature points between the digitized images to obtain a first stitched panoramic image.

[0011] Further, calculating the pixel difference between the first stitched panoramic image and the second stitched panoramic image of the adjacent frame to identify the dynamic target includes: The pixel values ​​of similar parts between the first panoramic image and the second panoramic image of its adjacent frame are reduced, and the dynamic target to be identified is the area showing changes between the two images.

[0012] Furthermore, the method of constructing a panoramic image background by a background modeling method and integrating the dynamic target as a foreground into the constructed panoramic image background includes: Read each frame of the panoramic image from the four-eye camera, build a background model based on the pixels with the highest frequency in each frame, and mark the dynamic targets identified in each adjacent frame of the panoramic image as foreground in the background model.

[0013] The present invention also provides a four-camera panoramic stitching device, comprising: The acquisition unit is used to acquire image information of the four cameras in the same space and import the image information into the CMOS image sensor; A signal conversion unit, used to convert the received image information optical signal into an image information electrical signal by using the pixel array in the CMOS image sensor, and a readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; A signal processing unit, used for the CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; A computing unit is used to extract key points from digital image information in a DSP digital signal processing chip collected by four cameras at different angles using a SIFT algorithm, and to match panoramic image information in the four cameras based on the key point extraction, and obtain a first stitched panoramic image output by the matching, wherein the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; an identification unit, configured to calculate a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame, and identify a dynamic target; The model unit is used to construct a panoramic image background by a background modeling method, and the dynamic target is merged into the constructed panoramic image background as a foreground to splice a dynamic image.

[0014] The present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the above-mentioned four-camera panoramic stitching method are implemented.

[0015] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned four-camera panoramic stitching method are implemented.

[0016] The four-camera panoramic stitching method provided by the present invention has the following beneficial effects: (1) The light signal conversion in the CMOS image sensor achieves efficient image light collection and conversion through the combination of color filter channels and photodiodes, resulting in high-quality color images; the specific filter combination ratio (1 / 4 blue, 1 / 4 blue and 1 / 2 green) can balance the color performance and improve the visual effect of the image; (2) Using the SIFT algorithm for image matching can accurately extract feature points between images preprocessed with the preset focal length composition, thereby improving the accuracy and stability of image matching. Image matching through feature points can generate a high-quality first stitched panoramic image, reducing ghosting and blurring. (3) Dynamic target recognition: By calculating the pixel difference between adjacent panoramic image frames, dynamic targets can be recognized in real time, improving the real-time performance and dynamic expression of panoramic images. The recognized dynamic targets can be used for subsequent analysis, tracking and interaction, expanding the application scenarios of panoramic images. (4) Background modeling and dynamic target fusion By reading each frame of the panoramic image of the four-eye camera, a stable background model is constructed to provide a basis for the fusion of dynamic targets. By marking the dynamic targets as foreground in the background model, natural and smooth dynamic panoramic images can be generated, improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a flow chart of a method for panoramic stitching of four cameras in one embodiment of the present invention; Figure 2 This is a structural block diagram of a panoramic stitching device with four cameras in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0018] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] Reference Figure 1 , is a flow chart of a four-camera panoramic stitching method proposed in the present invention, comprising the following steps: S1, collecting image information in the same space through four cameras, and importing the image information into a CMOS image sensor; S2, using the pixel array of the photoelectric conversion area in the CMOS image sensor to convert the received image information optical signal into an image information electrical signal, and the readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; S3, using a CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; S4, using the SIFT algorithm to extract key points from the digitized image information in the DSP digital signal processing chip collected by the four cameras at different angles, and matching the panoramic image information in the four cameras based on the key point extraction, and obtaining a first stitched panoramic image output by the matching, wherein the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; S5, identifying a dynamic target by calculating a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame; S6, constructing a panoramic image background by using a background modeling method, integrating the dynamic target as a foreground into the constructed panoramic image background, and splicing a dynamic image.

[0021] For S2, specifically: The pixel array in the photoelectric conversion area of ​​the CMOS image sensor converts the received image information optical signal into an image information electrical signal, including: The four-eye camera collects image light and transmits it to the CMOS image sensor array pixels. The image light passes through the color filter channel on the array pixels to fill the original black and white image to obtain a color image. The photodiode on each pixel converts the incident photons into electric charges. A single color filter is placed on each pixel of the pixel array, and is arranged in an alternating manner to combine the proportions of 1 / 4, 1 / 4 blue and 1 / 2 green.

[0022] In the specific implementation process: the four-eye camera is responsible for collecting image light and transmitting it to the array pixels of the CMOS image sensor. These lights then pass through the color filter channel on the array pixels. The role of the color filter is to fill the original black and white image with color to obtain a color image. The photodiode on each pixel plays a core role. When photons enter the photodiode, they are converted into electric charges. This conversion process is based on the principle of the photoelectric effect, that is, when photons interact with matter, they can transfer the energy of photons to electrons in the matter, so that they can obtain enough energy to escape and form a current. Regarding the design of the pixel array, a single color filter is placed on each pixel. These filters are combined in a specific interval arrangement, specifically, they are arranged according to the proportion of 1 / 4 blue, 1 / 4 blue and 1 / 2 green, aiming to balance the color performance and ensure the accuracy and richness of the image in color. Through the combination of filters of different colors, more color information can be captured, and then a more realistic and vivid color image can be restored in subsequent image processing.

[0023] For S3, specifically: The method for preprocessing the digitized image information by inputting the digitized image signal into the DSP digital signal processing chip comprises: Digital image information is processed using image grayscale processing and geometric transformation algorithms.

[0024] The color digitized image information is grayed out by weighted averaging, and the RGB value of each pixel is converted into a single gray value, highlighting the edge and texture of the image; The digital image information after grayscale processing is subjected to geometric transformation algorithm operation, the digital image pixel matrix of the four-eye camera at different camera angles is identified, and the overlapping area of ​​the digital images at different camera angles is obtained; through the geometric transformation operations of translation, scaling, rotation, and mirroring, the consistency of the size and angle of the digital images of the four-eye camera at different camera angles is constrained.

[0025] In the specific implementation process: using image grayscale processing technology, the color digitized image information can be converted into a grayscale image. This process is achieved through the weighted average method, that is, the RGB value of each pixel (representing the three color channels of red, green, and blue) will be converted into a single grayscale value. Grayscale processing helps to highlight the edge and texture information of the image, which facilitates subsequent image matching and stitching. The geometric transformation algorithm is operated on the grayscaled digitized image information. The purpose of this step is to identify the overlapping areas between the digitized images taken from different camera angles by the four-eye camera. Through the geometric transformation operations of translation, scaling, rotation, and mirroring, the images from different camera angles can be adjusted to align them in the same coordinate system, ensuring the consistency of size and angle between different images, and providing a basis for subsequent image stitching.

[0026] For S4, specifically: The steps of extracting key points of digitized image information using a SIFT algorithm, matching panoramic image information in four cameras based on the key point extraction, and obtaining a first stitched panoramic image output by matching include: The SIFT algorithm is used to extract feature points from the relationship between the digitized images preprocessed with the preset focal length composition; The digitized images at different camera angles are matched according to the feature points between the digitized images to obtain a first stitched panoramic image.

[0027] In the specific implementation process: the SIFT algorithm is used to extract feature points from the relationship between the digitized images preprocessed with the corresponding preset focal length composition. The SIFT algorithm is an algorithm for extracting local features in an image. It can find key points in different scale spaces and calculate the direction of the key points. These key points and their descriptors are invariant to image rotation, scale scaling, brightness changes, etc., and are suitable for image matching. Matching of digitized images from different camera angles is performed based on the feature points between the digitized images. In this step, the algorithm compares the feature points between different images and finds matching point pairs. These matching point pairs will serve as the basis for image stitching to ensure that the stitched images remain visually coherent and consistent. Through the matching feature point pairs, the algorithm will perform image stitching operations to generate a first stitched panoramic image. During the stitching process, the algorithm will consider the geometric transformation relationships of image translation, scaling, rotation, and mirroring to ensure that the stitched images are also geometrically aligned.

[0028] For S5, specifically: Calculating the pixel difference between the first stitched panoramic image and the second stitched panoramic image of the adjacent frame, and identifying the dynamic target, including: The pixel values ​​of similar parts between the first panoramic image and the second panoramic image of its adjacent frame are reduced, and the dynamic target to be identified is the area showing changes between the two images.

[0029] In the specific implementation process: first, a static first stitched panoramic image is obtained through preprocessing, feature extraction and matching of multiple camera images, and then a second stitched panoramic image of adjacent frames to the first panoramic image is obtained. The second stitched panoramic image is a frame immediately following the first image on the time axis.

[0030] The pixel values ​​of similar parts of the two images are compared to identify dynamic targets between the two images. Specifically, the difference between the first panoramic image and the second panoramic image of its adjacent frame at the corresponding pixel position is calculated. This difference reflects the change in image content between the two time points. The dynamic target to be identified is the area where changes appear between the two images. These changes are caused by the movement of objects, changes in light, or any other factors that cause changes in image content. By calculating the pixel difference, these change areas can be accurately located and identified as dynamic targets.

[0031] For S6, specifically: The method of constructing a panoramic image background by a background modeling method and integrating the dynamic target as a foreground into the constructed panoramic image background comprises: Read each frame of the panoramic image from the four-eye camera, build a background model based on the pixels with the highest frequency in each frame, and mark the dynamic targets identified in each adjacent frame of the panoramic image as foreground in the background model.

[0032] In the specific implementation process: read each frame of the panoramic image taken by the four-eye camera and stitched to form a continuous frame of the video sequence. According to the frequency of the pixel points in each frame, a background model is constructed. This model represents the background state that the panoramic image should present without the interference of dynamic targets. Each pixel point in the background model corresponds to the pixel value with the highest frequency in the video sequence. After the background model is established, the dynamic target identified in each adjacent frame of the panoramic image is marked as the foreground in this background model.

[0033] Through this method, the natural fusion of dynamic targets and static backgrounds can be achieved, generating more vivid and realistic panoramic images and providing a richer and more intuitive visual experience.

[0034] refer to Figure 2 , a four-camera panoramic stitching device, comprising: The acquisition unit is used to acquire image information of the four cameras in the same space and import the image information into the CMOS image sensor; A signal conversion unit, used to convert the received image information optical signal into an image information electrical signal by using the pixel array in the CMOS image sensor, and a readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; A signal processing unit, used for the CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; A computing unit is used to extract key points from digital image information in a DSP digital signal processing chip collected by four cameras at different angles using a SIFT algorithm, and to match panoramic image information in the four cameras based on the key point extraction, and obtain a first stitched panoramic image output by the matching, wherein the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; an identification unit, configured to calculate a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame, and identify a dynamic target; The model unit is used to construct a panoramic image background by a background modeling method, and the dynamic target is merged into the constructed panoramic image background as a foreground to splice a dynamic image.

[0035] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0036] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0037] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0038] In summary, image information in the same space is collected by four cameras, and the image information is imported into a CMOS image sensor, wherein the four cameras include cameras corresponding to preset focal lengths; The received image information optical signal is converted into an image information electrical signal by using the pixel array of the photoelectric conversion area in the CMOS image sensor, and the readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; Using a CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; The SIFT algorithm is used to extract key points of the digitized image information in the DSP digital signal processing chip collected by the four cameras at different angles, and the panoramic image information in the four cameras is matched based on the key point extraction, and a first stitched panoramic image output by the matching is obtained, where the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; Identifying a dynamic target by calculating a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame; A panoramic image background is constructed using a background modeling method, and the dynamic target is integrated into the constructed panoramic image background as a foreground to splice a dynamic image; the state of the moving target is updated in real time, the splicing traces are eliminated, and the moving target is tracked.

[0039] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0040] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0041] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A four-camera panoramic stitching method, It is characterized in that The following steps are involved: Collecting image information in the same space through four cameras, and importing the image information into a CMOS image sensor, wherein the four cameras include cameras corresponding to preset focal lengths; The received image information optical signal is converted into an image information electrical signal by using the pixel array of the photoelectric conversion area in the CMOS image sensor, and the readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; Using a CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; The SIFT algorithm is used to extract key points of the digitized image information in the DSP digital signal processing chip collected by the four cameras at different angles, and the panoramic image information in the four cameras is matched based on the key point extraction, and a first stitched panoramic image output by the matching is obtained, where the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; Identifying a dynamic target by calculating a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame; A panoramic image background is constructed by using a background modeling method, and the dynamic target is integrated into the constructed panoramic image background as a foreground to splice a dynamic image.

2. The four-camera panoramic stitching method according to claim 1, It is characterized in that The pixel array in the photoelectric conversion area of ​​the CMOS image sensor converts the received image information optical signal into an image information electrical signal, including: The four-eye camera collects image light and transmits it to the CMOS image sensor array pixels. The image light passes through the color filter channel on the array pixels to fill the original black and white image to obtain a color image. The photodiode on each pixel converts the incident photons into electric charges. A single color filter is placed on each pixel of the pixel array, and is arranged in an alternating manner to combine the proportions of 1 / 4, 1 / 4 blue and 1 / 2 green.

3. The four-camera panoramic stitching method according to claim 1, It is characterized in that The method for preprocessing the digitized image information by inputting the digitized image signal into the DSP digital signal processing chip comprises: Digital image information is processed using image grayscale processing and geometric transformation algorithms.

4. The four-camera panoramic stitching method according to claim 3, Features: The color digitized image information is grayed out by weighted averaging, and the RGB value of each pixel is converted into a single gray value, highlighting the edge and texture of the image; The digital image information after grayscale processing is subjected to geometric transformation algorithm operation, the digital image pixel matrix of the four-eye camera at different camera angles is identified, and the overlapping area of ​​the digital images at different camera angles is obtained; through the geometric transformation operations of translation, scaling, rotation, and mirroring, the consistency of the size and angle of the digital images of the four-eye camera at different camera angles is constrained.

5. The four-camera panoramic stitching method according to claim 1, It is characterized in that The steps of extracting key points of digitized image information using a SIFT algorithm, matching panoramic image information in four cameras based on the key point extraction, and obtaining a first stitched panoramic image output by matching include: The SIFT algorithm is used to extract feature points from the relationship between the digitized images preprocessed with the preset focal length composition; The digitized images at different camera angles are matched according to the feature points between the digitized images to obtain a first stitched panoramic image.

6. The four-camera panoramic stitching method according to claim 1, It is characterized in that Calculating the pixel difference between the first stitched panoramic image and the second stitched panoramic image of the adjacent frame, and identifying the dynamic target, including: The pixel values ​​of similar parts between the first panoramic image and the second panoramic image of its adjacent frame are reduced, and the dynamic target to be identified is the area showing changes between the two images.

7. The four-camera panoramic stitching method according to claim 1, It is characterized in that The method of constructing a panoramic image background by a background modeling method and integrating the dynamic target as a foreground into the constructed panoramic image background comprises: Read each frame of the panoramic image from the four-eye camera, build a background model based on the pixels with the highest frequency in each frame, and mark the dynamic targets identified in each adjacent frame of the panoramic image as foreground in the background model.

8. A four-camera panoramic stitching device, It is characterized in that include: The acquisition unit is used to acquire image information of the four cameras in the same space and import the image information into the CMOS image sensor; A signal conversion unit, used to convert the received image information optical signal into an image information electrical signal by using the pixel array in the CMOS image sensor, and a readout circuit on the CMOS image sensor converts the image information electrical signal into a digital image signal; A signal processing unit, used for the CMOS image sensor to input the converted digital image signal into a DSP digital signal processing chip for image information preprocessing; A computing unit is used to extract key points from digital image information in a DSP digital signal processing chip collected by four cameras at different angles using a SIFT algorithm, and to match panoramic image information in the four cameras based on the key point extraction, and obtain a first stitched panoramic image output by the matching, wherein the first stitched panoramic image is a stitching of images collected by different cameras in the same time frame; an identification unit, configured to calculate a pixel difference between the first stitched panoramic image and a second stitched panoramic image of an adjacent frame, and identify a dynamic target; The model unit is used to construct a panoramic image background by a background modeling method, and the dynamic target is merged into the constructed panoramic image background as a foreground to splice a dynamic image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the four-camera panoramic stitching method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the four-camera panoramic stitching method described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Panoramic real-time imaging splicing method based on novel structure of four cameras

    CN112581371A

  • Real-time omnidirectional imaging system and method based on multipath camera

    CN108769578A

  • Automobile rearview panoramic image processing system

    CN115578710A

  • Panoramic image splicing method, electronic equipment and storage medium

    CN116883305A

  • Panorama composing method for motion video

    CN1756313A

Cited By

  • Visual image processing method and apparatus based on strain mechanism, device, and medium

    WO2026157073A1