Image splicing system and electronic equipment
By using a controller to synchronize signal output and data processing circuit in the image stitching system, the problems of space limitation and delay are solved, enabling low-latency image stitching in a small space and improving image stitching efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- NANCHANG O FILM OPTICAL ELECTRONICS TECH CO LTD
- Filing Date
- 2025-03-31
- Publication Date
- 2026-05-15
AI Technical Summary
In existing technologies, it is difficult to achieve 360-degree shooting without blind spots when connecting two cameras to a digital signal processing chip in a limited space, and the image stitching process needs to be cached in DDR SDRAM, which increases the latency.
The controller outputs a synchronization signal to multiple cameras to achieve synchronized data output. It also uses data stitching, synchronization, transposition and conversion circuits to stitch images together. Combined with a direct memory access controller and image processing chip, it reduces latency and space usage.
It enables low-latency image stitching in a smaller space, reduces data transmission and processing latency, and improves image stitching efficiency.
Smart Images
Figure CN224249778U_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image acquisition and processing technology, and in particular to an image stitching system and electronic device. Background Technology
[0002] In related technologies, a single Digital Signal Processing (DSP) chip connected to two cameras can meet the needs of most applications, but it cannot meet the requirements in some specific scenarios. For example, when 360-degree shooting without blind spots needs to be achieved within the limited space of an electronic device, if a single chip can only connect to two cameras, it is difficult to install more DSP chips and cameras due to the limitations of the internal structure size of the device. Moreover, in the image stitching process of related technologies, all complete image frames captured by multiple cameras need to be cached in Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) before the stitching operation can be performed. This process involves the processing of one frame buffer, which will correspondingly increase the latency. Utility Model Content
[0003] In view of this, this application provides an image stitching system and electronic device to achieve low-latency image stitching while occupying less space.
[0004] The first aspect of this application provides an image stitching system, comprising: a plurality of cameras, each camera including a synchronous input interface and a data output interface; a controller, the controller including a synchronous signal output circuit and a plurality of data receiving circuits, the first terminals of the plurality of data receiving circuits being connected one-to-one with all the data output interfaces, the synchronous signal output circuits being connected to each synchronous input interface, the controller being used to output a synchronous signal to each synchronous input interface through the synchronous signal output circuits so that all cameras synchronously output image data through the data output interfaces, and the controller being used to stitch together the image data received by each data receiving circuit through the data output interface.
[0005] In one embodiment, the controller further includes a data splicing circuit, a data synchronization circuit, a data transposition circuit, and a data conversion circuit; wherein, the second terminal of each data receiving circuit is connected to the input terminal of the data splicing circuit, the output terminal of the data splicing circuit is connected to the input terminal of the data synchronization circuit, the output terminal of the data synchronization circuit is connected to the input terminal of the data transposition circuit, and the output terminal of the data transposition circuit is connected to the input terminal of the data conversion circuit.
[0006] In one embodiment, the image stitching system further includes an image processing chip for processing the data output by the data conversion circuit.
[0007] In one embodiment, the image stitching system further includes a direct memory access controller, which is electrically connected to the data conversion circuit and the image processing chip. The direct memory access controller is used to realize data transmission between the data conversion circuit and the image processing chip.
[0008] In one embodiment, the image stitching system further includes a double data rate synchronous dynamic random access memory (DRAM), which is connected to the data stitching circuit, the data synchronization circuit, the data transpose circuit, and the data conversion circuit.
[0009] In one embodiment, the controller further includes an I2C signal configuration circuit, and each camera further includes an I2C interface, wherein each I2C interface is connected to the I2C signal configuration circuit.
[0010] In one embodiment, the controller is a field-programmable gate array (FPGA).
[0011] In one embodiment, the plurality of cameras includes four cameras.
[0012] A second aspect of this application provides an electronic device including an image stitching system as described in any of the preceding claims.
[0013] In one embodiment, the electronic device is a 360-degree panoramic camera.
[0014] The image stitching system provided in this application outputs a synchronization signal to each camera through the synchronization signal output circuit of the controller, thereby controlling all cameras to synchronously output image data through the data output interface. Then, the data receiving circuit receives the image data output from the corresponding data output interface, and stitches the images based on the synchronously acquired image data from all cameras. In this way, because the data acquired from each of the multiple connected cameras is received and processed synchronously, the latency in data transmission and processing can be effectively reduced, and the space occupied by the image stitching system can be reduced. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation on the scope of protection of this application. In the various drawings, similar components are numbered similarly.
[0016] Figure 1 This is a structural diagram of an image stitching system provided in an embodiment of this application.
[0017] Figure 2 This is a data diagram illustrating the image stitching process provided in an embodiment of this application.
[0018] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0020] It should be noted that when a component is considered to be "connected" to another component, it can be directly connected to the other component or may also have an intervening component. When a component is considered to be "placed" on another component, it can be directly placed on the other component or may also have an intervening component. The terms "top," "bottom," "upper," "lower," "left," "right," "front," "back," and similar expressions used in this article are for illustrative purposes only.
[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
[0022] Some embodiments will now be described with reference to the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0023] In related technologies, a single Digital Signal Processing (DSP) chip connected to two cameras can meet the needs of most applications, but it cannot meet the requirements in some specific scenarios. For example, when 360-degree shooting without blind spots needs to be achieved within the limited space of an electronic device, if a single chip can only connect to two cameras, it is difficult to install more DSP chips and cameras due to the limitations of the internal structure size of the device. Moreover, in the image stitching process of related technologies, all complete image frames captured by multiple cameras need to be cached in Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) before the stitching operation can be performed. This process involves the processing of one frame buffer, which will correspondingly increase the latency.
[0024] In view of this, this application provides an image stitching system and electronic device that can achieve low-latency image stitching while occupying less space.
[0025] Please seeFigure 1 , Figure 1 This is a schematic diagram of an image stitching system 100 provided in an embodiment of this application. The image stitching system 100 includes a plurality of cameras 10 and a controller 20.
[0026] Each camera 10 includes a synchronization input interface 11 and a data output interface 12. The controller 20 includes a synchronization signal output circuit 21 and several data receiving circuits 22. The synchronization signal output circuit 21 is connected to each synchronization input interface 11, and the first ends of the several data receiving circuits 22 are connected one-to-one with all the data output interfaces 12. The controller 20 is used to output a synchronization signal to each synchronization input interface 11 through the synchronization signal output circuit 21, so that all cameras 10 synchronously output image data through the data output interfaces 12. The controller 20 is also used to perform image stitching on the image data received by each data receiving circuit 22 through the data output interfaces 12. That is, in this application, the controller 20 can act as a master device, each camera 10 can act as a slave device, and each camera 10 synchronously outputs image data to the controller 20 under the control of the controller 20, so that the controller 20 can perform image stitching.
[0027] In some embodiments, the synchronization input interface may be a FSIN / SYNC (Frame Synchronization Input / Output) interface, and the data output interface 12 may be a MIPI (Mobile Industry Processor Interface) interface. This application does not limit this.
[0028] Thus, the image stitching system provided in this application outputs a synchronization signal to each camera 10 through the synchronization signal output circuit of the controller 20, thereby controlling all cameras to synchronously output image data through the data output interface 12. Then, the data receiving circuit 22 receives the image data output from the corresponding data output interface 12 respectively, and performs image stitching based on the image data synchronously acquired by all cameras. In this way, since the data acquired by each of the multiple connected cameras 10 is received and processed synchronously, the latency in data transmission and processing can be effectively reduced, and the space occupied by the image stitching system can be reduced.
[0029] Please continue reading. Figure 1In some embodiments, the controller 20 further includes a data splicing circuit 23, a data synchronization circuit 24, a data transposition circuit 25, and a data conversion circuit 26. The second terminal of each data receiving circuit 22 is connected to the input terminal of the data splicing circuit 23, the output terminal of the data splicing circuit 23 is connected to the input terminal of the data synchronization circuit 24, the output terminal of the data synchronization circuit 24 is connected to the input terminal of the data transposition circuit 25, and the output terminal of the data transposition circuit 25 is connected to the input terminal of the data conversion circuit 26.
[0030] Specifically, please refer to Figure 2 , Figure 2 This embodiment of the present application illustrates an image data processing flow. The data receiving circuit 22 processes the image data received through the data output interface 12 to convert the MIPI-compliant image data into AXIS-compliant data (a). Data (a) includes row pixel data obtained from data acquired by each camera 11 after protocol conversion. For example, Ch-A represents the row pixel data corresponding to the first camera 10; Ch-B represents the row pixel data corresponding to the second camera 10; Ch-C represents the row pixel data corresponding to the third camera 10; and Ch-D represents the row pixel data corresponding to the fourth camera 10. Understandably, the number of bytes in each row pixel data acquired by each camera 11 may differ due to different resolutions. Therefore, the data stitching circuit 23 fills the row pixel data acquired by each camera 11 in data (a) to obtain data (b). For example, the last column of each row pixel data in each data (a) can be filled to increase the number of pixels in the last column of each row pixel data to an integer multiple of a preset number. Next, the data synchronization circuit 24 buffers data (b) according to a preset order and time interval, and outputs the corresponding data (c) to synchronize the data in data (c). Then, the data transpose circuit 25 transposes data (c) according to a preset concatenation strategy and outputs data (d), so that each data matrix in data (d) is column data, and the column data is sorted according to a preset order. For example, the data transpose circuit 25 can transpose the data in data (c) into a 4×N matrix and output it. The data conversion circuit 26 packages consecutive column data in data (d) into block data and outputs data (e) to improve transmission efficiency.
[0031] In some embodiments, the image stitching system 100 further includes an image processing chip 30. The image processing chip 30 is used to process the data output by the data conversion circuit 26, such as noise reduction and color correction, to optimize image quality.
[0032] In some embodiments, the image stitching system 100 further includes a Direct Memory Access (DMA) controller 40. The DMA controller 40 is electrically connected to the data conversion circuit 26 and the image processing chip 30, and is used to realize data transmission between the data conversion circuit 26 and the image processing chip. Understandably, by using the DMA controller 40, this application can significantly improve the efficiency of data transmission and further reduce the latency of image stitching.
[0033] In some embodiments, the image stitching system further includes a Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) 50, which is connected to the data stitching circuit 23, the data synchronization circuit 24, the data transposition circuit 25, and the data conversion circuit 26. Thus, when the data stitching circuit 23, the data synchronization circuit 24, the data transposition circuit 25, and the data conversion circuit 26 perform corresponding data processing, they can retrieve data from the DDR 50 and cache the generated data in the DDR 50.
[0034] In some embodiments, the controller further includes an I2C signal configuration circuit 27, and each camera further includes an I2C interface 13. Each I2C interface 13 is connected to the I2C signal configuration circuit 27. The I2C signal configuration circuit 27 is used to configure the address, parameters, etc., of each camera 10.
[0035] In some embodiments, the controller 20 is a field-programmable gate array (FPGA). The plurality of cameras includes four cameras. In other embodiments, the controller 20 may be replaced with other processors, and the controller 20 may connect to three or more cameras 10. This application does not limit the specific type of controller 20 or the number of cameras 10 included in the image stitching system 100.
[0036] Please see Figure 3 This application also provides an electronic device 200, including an image stitching system as described in any of the above embodiments. In some embodiments, the electronic device 200 may be a 360-degree panoramic camera. In other embodiments, the electronic device 200 may also be an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality (MR) device, or an extended reality (XR) device, etc. This application does not limit the specific type of the electronic device 200.
[0037] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the electronic device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.
[0038] Furthermore, the functional modules in the various embodiments of this application can be integrated into the same processing module, or each module can exist physically separately, or two or more modules can be integrated into the same module. The integrated modules described above can be implemented in hardware or in the form of hardware plus software functional modules.
[0039] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or electronic devices recited in the electronic device claims may also be implemented by the same module or electronic device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.
[0040] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting.
[0041] This application is not limited to the specific embodiments described above. Those skilled in the art will readily understand that many alternative solutions exist for the test fixture without departing from the principles and scope of this application. The scope of protection of this application is determined by the claims.
Claims
1. An image stitching system, characterized in that, include: A plurality of cameras, each of which includes a synchronous input interface and a data output interface; The controller includes a synchronization signal output circuit and several data receiving circuits. The first ends of the several data receiving circuits are connected to all the data output interfaces one by one. The synchronization signal output circuit is connected to each of the synchronization input interfaces. The controller is used to output a synchronization signal to each of the synchronization input interfaces through the synchronization signal output circuit, so that all the cameras synchronously output image data through the data output interfaces. The controller is also used to perform image stitching on the image data received by each of the data receiving circuits through the data output interfaces.
2. The image stitching system according to claim 1, characterized in that, The controller further includes a data splicing circuit, a data synchronization circuit, a data transposition circuit, and a data conversion circuit; wherein... The second terminal of each of the data receiving circuits is connected to the input terminal of the data splicing circuit, the output terminal of the data splicing circuit is connected to the input terminal of the data synchronization circuit, the output terminal of the data synchronization circuit is connected to the input terminal of the data transpose circuit, and the output terminal of the data transpose circuit is connected to the input terminal of the data conversion circuit.
3. The image stitching system according to claim 2, characterized in that, The image stitching system also includes an image processing chip, which is used to process the data output by the data conversion circuit.
4. The image stitching system according to claim 3, characterized in that, The image stitching system also includes a direct memory access controller, which is electrically connected to the data conversion circuit and the image processing chip. The direct memory access controller is used to realize data transmission between the data conversion circuit and the image processing chip.
5. The image stitching system according to claim 2, characterized in that, The image stitching system also includes a double data rate synchronous dynamic random access memory, which is connected to the data stitching circuit, the data synchronization circuit, the data transpose circuit, and the data conversion circuit.
6. The image stitching system according to claim 1, characterized in that, The controller further includes an I2C signal configuration circuit, and each camera further includes an I2C interface, wherein each I2C interface is connected to the I2C signal configuration circuit.
7. The image stitching system according to claim 1, characterized in that, The controller is a field-programmable gate array (FPGA).
8. The image stitching system according to claim 1, characterized in that, The plurality of cameras includes four of the cameras.
9. An electronic device, characterized in that, The image stitching system includes any one of claims 1 to 8.
10. The electronic device according to claim 9, characterized in that, The electronic device is a 360-degree panoramic camera.