Image transmission and processing system and method based on MIPI CSI C-PHY protocol

Abstract

The application discloses an image transmission processing system and method based on a MIPI CSI C-PHY protocol, which comprises N low-pixel cameras connected through a communication interface and a first receiving device connected with the N low-pixel cameras through a same group of CSI interfaces, wherein N is 2 or 3. The C-PHY protocol is used to re-allocate the MIPI CSI interface channel, the multiple low-pixel cameras share one group of MIPI CSI interfaces, and the first receiving device is updated, so that the image data collected by the low-pixel cameras is segmented and processed in the first receiving device, the channel sharing of the low-pixel camera signal transmission is realized, and the channel utilization is improved. Meanwhile, multiple groups of MIPI CSI interfaces are connected to one high-pixel camera, and the high frame rate image collection function of the high-pixel camera is realized.

Description

Technical Field

[0001] This disclosure generally relates to the field of communication technology, and in particular to an image transmission and processing system and method based on the MIPI CSI C-PHY protocol. Background Technology

[0002] With the increasing demand for consumer electronics products such as mobile phones and the improved capabilities of manufacturers, mobile phone cameras are becoming increasingly powerful. Key camera parameters include aperture, angle of view, zoom, pixels, and resolution. Among these, pixel count is the biggest selling point for major manufacturers. Currently, the highest resolution for a single camera has reached over 100 million pixels. This improved image acquisition capability undoubtedly presents challenges for image data transmission.

[0003] The Mobile Industry Processor Interface (MIPI) is an open standard and specification developed by the MIPI Alliance for mobile application processors, used to standardize the interfaces inside mobile devices into standard internal interfaces. Standard internal interfaces include the Camera Serial Interface (CSI). MIPI CSI is a universal transmission interface for mobile consumer electronics cameras. Among the protocols currently published by MIPI, two types of camera-based physical layer protocols are commonly used: the once-dominant D-PHY protocol and the C-PHY protocol. Given the increasingly higher requirements for data transmission rates, the higher-speed C-PHY has gradually become the preferred choice. However, even the update capabilities of C-PHY transmission speed cannot keep up with the growth rate of camera pixels; the latest version of the C-PHY standard still cannot enable 100-megapixel cameras to achieve image acquisition at a regular frame rate. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an image transmission and processing system and method based on the MIPI CSI C-PHY protocol.

[0005] In a first aspect, an image transmission and processing system based on the MIPI CSI C-PHY protocol is provided, comprising: N low-pixel cameras and a first receiving device; wherein the N low-pixel cameras are connected to each other via a communication interface, and the N low-pixel cameras are connected to the first receiving device using the same set of MIPI CSI interfaces with the C-PHY protocol; N is 2 or 3.

[0006] Furthermore, the system also includes: at least one high-resolution camera and a second receiving device; wherein, one of the high-resolution cameras is connected to the second receiving device using at least two sets of MIPI CSI interfaces with C-PHY protocol.

[0007] Secondly, an image transmission processing method based on the MIPI CSI C-PHY protocol is provided, including:

[0008] Acquire image data from N low-resolution cameras, where N is 2 or 3;

[0009] The image data captured by N low-pixel cameras are transmitted to the first receiving device using the same set of MIPI CSI interfaces with C-PHY protocol;

[0010] The first receiving device processes the image data captured by N low-pixel cameras and transmits it to the upper layer to form an image.

[0011] Furthermore, the method also includes:

[0012] Acquire image data from each high-resolution camera;

[0013] The image data captured by each high-pixel camera is transmitted to the second receiving device using at least two sets of MIPI CSI interfaces with C-PHY protocol;

[0014] The second receiving device processes at least two sets of image data transmitted via the MIPI CSI interface with the C-PHY protocol and transmits them to the upper layer to form an image.

[0015] Thirdly, a mobile terminal device is provided, the mobile terminal device comprising:

[0016] One or more processors;

[0017] Memory, used to store one or more programs.

[0018] When one or more programs are executed by one or more processors, the one or more processors perform the image transmission processing method based on the MIPI CSI C-PHY protocol described above.

[0019] Fourthly, a computer-readable storage medium is provided that stores a computer program, which is executed by a processor to perform the aforementioned image transmission processing method based on the MIPI CSI C-PHY protocol.

[0020] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0021] Given the limited MIPI channel resources currently available from receiving devices (CPUs), some embodiments of this application utilize the C-PHY protocol, which offers higher transmission rates and is not constrained by independent synchronization clocks, to reallocate MIPI CSI interface channels. This involves multiple low-pixel cameras sharing a single MIPI CSI interface with the C-PHY protocol. Furthermore, by updating the CPU chip architecture and performing segmentation and other processing on the image data acquired by the low-pixel cameras within the CPU chip, shared channels for low-pixel camera signal transmission can be achieved, improving the overall utilization rate of channel transmission.

[0022] Furthermore, addressing the technical problem that the transmission capacity of a single MIPI interface is insufficient for existing high-pixel cameras, according to certain embodiments of this application, multiple MIPI CSI interfaces with C-PHY protocol are connected to a single high-pixel camera. Simultaneously, with updates to the CPU chip architecture, the image data acquired by the high-pixel camera is integrated within the CPU chip, and data stability is protected through processing such as image alignment. This enables high-frame-rate image acquisition by the high-pixel camera, significantly improving the camera performance of mobile phones and other products. Attached Figure Description

[0023] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0024] Figure 1 A diagram illustrating the architecture for image acquisition, transmission, and processing from cameras in existing mobile phones and other products.

[0025] Figure 2 A diagram of an image transmission and processing system provided for embodiments of this application, in which three low-pixel cameras share a single MIPICSI interface with C-PHY protocol;

[0026] Figure 3 A diagram illustrating an image transmission and processing system for a high-pixel camera sharing two sets of MIPICSI interfaces with C-PHY protocol, provided in an embodiment of this application.

[0027] Figure 4 An exemplary flowchart of an image transmission method for multiple low-pixel cameras sharing a set of MIPI CSI interfaces based on the MIPI CSI C-PHY protocol provided in this application embodiment;

[0028] Figure 5 An exemplary flowchart of an image transmission and processing method for a high-pixel camera sharing multiple MIPI CSI interfaces based on the MIPI CSI C-PHY protocol provided in this application embodiment;

[0029] Figure 6 This application provides an embodiment of a high-resolution camera that shares image data streams from two MIPI CSI interfaces.

[0030] Figure 7 This application provides an application layer processing flowchart for image re-stitching using a high-pixel camera that shares two sets of MIPI CSI interfaces.

[0031] Figure 8 (a) is a diagram of the existing camera image acquisition, transmission, and processing architecture; Figure 8 (b) A diagram of an image transmission system based on the MIPI CSI C-PHY protocol provided in an embodiment of this application;

[0032] Figure 9 This is a schematic diagram of the structure of a mobile terminal device provided in an embodiment of this application. Detailed Implementation

[0033] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0034] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] like Figure 1 This refers to the current image acquisition, transmission, and processing architecture of mobile phones and other products. Because the current design allows a single MIPI interface to transmit data from only one camera, some functional low-pixel cameras (such as TOF depth cameras, depth-of-field cameras, and macro cameras) may only utilize one or a portion of their MIPI interface data lines, or even only a fraction or less of the data line's transmission capacity. The specific calculations are as follows:

[0036] Assuming the macro camera used has a mainstream resolution of 2 megapixels, the amount of data transmitted at 30Hz in Raw10 RGB format is:

[0037] 2*10 6 Pixels × 30Hz × 10bit = 6 * 10 8 bps (1)

[0038] The lowest speed D-PHY 1.0 currently has a transmission capacity of 4Gbps, so the channel utilization rate is:

[0039] 6*10 8 / 4*10 9 =15% (2)

[0040] Therefore, it can be seen that in the existing technology, a set of MIPI interfaces only transmits the data acquired by a low-pixel camera, resulting in very low channel utilization.

[0041] To address this issue, this application provides an image transmission and processing system based on the MIPI CSI C-PHY protocol. This system employs multiple low-pixel cameras sharing a single MIPI CSI interface with the C-PHY protocol. Specifically, it includes N low-pixel cameras and a first receiving device. The N low-pixel cameras are connected via a communication interface, and all N low-pixel cameras are connected to the first receiving device using the same set of MIPI CSI interfaces with the C-PHY protocol. N is 2 or 3.

[0042] Specifically, the hardware architecture of multiple low-resolution cameras (not exceeding the maximum number of data pairs supported by MIPI) is modified. Communication interfaces such as I2C or a single-wire bus with fewer lines are added between the multiple low-resolution cameras for connection. These cameras connect to the first receiving device using the same set of MIPI CSI interfaces with the C-PHY protocol. C-PHY is a physical layer protocol of MIPI used for displays and cameras; literally, it's used for bandwidth-constrained display and camera signal transmission channels.

[0043] Specifically, each of the low-pixel cameras is provided with a first camera sensor and a first encoder; the first receiving device includes a first LP receiver, a first HS receiver, a splitter, N first decoders and a first image processing unit;

[0044] Each of the first camera sensors is used to collect its own camera parameters and transmit the collected camera parameters to one of the low-pixel cameras; the first encoder of the low-pixel camera is used to encode the camera parameters collected by the N first camera sensors and transmit the encoded camera parameters to the first LP receiver in the first receiving device in LP mode during the image data transmission gap.

[0045] Each of the first camera sensors is used to acquire image data in HS mode and transmit the acquired image data to the first HS receiver through the same set of MIPI CSI interfaces with C-PHY protocol;

[0046] The segmenter is used to group and segment the image data received by the first HS receiver according to the encoded camera parameters received by the first LP receiver, so as to obtain N groups of image data.

[0047] Each of the first decoders is used to decode each group of image data and transmit the decoded image data to the first image processing unit;

[0048] The first image processing unit is used to process the decoded image data and transmit the processed image data to the upper layer to form an image.

[0049] like Figure 2 As shown, three low-pixel cameras, designated Camera 1, 2, and 3, are used. Cameras 2 and 3 transmit their own camera parameters (such as operating status, pixel count, and acquisition frequency) to Camera 1, which is connected to MIPI Pair 0, in real time. In this setup, Camera 1 acts as a temporary host, organizing and encoding the camera parameters from the three cameras. The encoded camera parameter information is then transmitted via Lane 0 Data0 to the first LP receiver (low-power receiver) in the first receiving device (CPU) during image data transmission intervals using MIPI LP Mode (low-power mode). A Lane is a signal group, representing a complete MIPI transmission interface, comprising several signal lanes (where D-PHY has a maximum of 1 lane for clock and 4 lanes for data; C-PHY has 3 lanes for data). A lane represents a signal differential pair (D-PHY) or a three-phase signal group (C-PHY).

[0050] During image transmission (MIPI HS Mode, high-speed mode), image data captured by the three cameras is transmitted to the first HS receiver in the CPU via its respective MIPI Data Lane, allocated according to data size. For example, image data from camera 1 is transmitted to the first HS receiver via CSI0 Lane 0, image data from camera 2 via CSI0 Lane 1, and image data from camera 3 via CSI0 Lane 2. The splitter in the CPU, based on the encoded camera parameters previously received by the first LP receiver, groups and splits the image data received by the first HS receiver, performing clock recovery, data equalization, and other operations sequentially to obtain three sets of image data related to clock and data. These three sets of image data are then transmitted to their respective independent first decoders for decryption, decoding, deserialization, serial-to-parallel conversion, and demapping via a 7-bit to 16-bit binary demapper. The processed image data is then transmitted to the first image processing unit (GPU) for further processing. The GPU then transmits the processed image data to the upper layer to form an image. Specifically, the upper layer can be a display or memory.

[0051] By utilizing the above technical solution, the utilization rate of the MIPI interface can be effectively improved. For example, if the images captured by the currently mainstream 13-megapixel ultra-wide-angle lens, 2-megapixel depth-of-field lens, and 2-megapixel telephoto lens are transmitted using Lane 0, Lane 1, and Lane 2 of the MIPI CSI0 C-PHY respectively, previous calculations can predict that there is still a large surplus in the transmission volume. By adopting the technical solution of this application, not only is the overall transmission volume utilization rate of CSI0 improved, but more importantly, compared with the existing design, the technical solution of this application saves two sets of MIPI CSI channels. This can save CPU resources and also reserve more resources for more cameras, or high-resolution main cameras that may require multiple sets of MIPI CSI transmission.

[0052] On the other hand, in existing technologies, the data transmission capacity of one MIPI interface may simply be insufficient for some high-pixel cameras (such as rear main cameras). Taking a current flagship smartphone's main camera—108 megapixels—as an example, let's calculate the data volume:

[0053] 1.08*10 8 Pixels × 30Hz × 10bit = 3.24 * 10 10 bps (3)

[0054] The highest-speed C-PHY 1.2 currently has a transmission capacity of 20.13Gbps, so the channel utilization rate is:

[0055] 3.24*10 10 / 2.13*10 10 ≈147% (4)

[0056] One set of MIPI interfaces is simply not enough. Therefore, currently, mobile phones equipped with 100-megapixel cameras only support 10Hz image acquisition when using this camera, and the video recording function is far from being realized.

[0057] Therefore, in order to solve the technical problem that the transmission capacity of a single MIPI interface is insufficient for high-pixel cameras, the image transmission and processing system based on the MIPI CSI C-PHY protocol of this application further includes:

[0058] At least one high-resolution camera and a second receiving device; wherein, one of the high-resolution cameras is connected to the second receiving device via at least two sets of MIPI CSI interfaces with C-PHY protocol.

[0059] Specifically, each of the high-pixel cameras is equipped with a second camera sensor and a second encoder; the second receiving device includes a second LP receiver, a second HS receiver, a blender, a second decoder, and a second image processing unit;

[0060] The second camera sensor is used to collect its own camera parameters, and the second encoder is used to encode the camera parameters collected by the second camera sensor, and transmit the encoded camera parameter information to the second LP receiver in LP mode through two sets of MIPI CSI interfaces with C-PHY protocol respectively.

[0061] The second camera sensor is used to acquire image data information, and transmits the acquired image data information to the second HS receiver through two sets of MIPI CSI interfaces with C-PHY protocol in HS mode;

[0062] The blender is used to blend the image data information transmitted from the two sets of MIPI CSI interfaces with C-PHY protocol received by the second HS receiver according to the encoded camera parameter information received by the second LP receiver, and then transmit the blended image data information to the second decoder.

[0063] The second decoder is used to decode the merged image data information and transmit the decoded image data information to the second image processing unit;

[0064] The second image unit is used to process the decoded image data information and transmit the processed image data to the upper layer to form an image.

[0065] Specifically, such as Figure 3 As shown, two sets of MIPI CSI interfaces with C-PHY protocol are connected to a high-resolution camera. Each set of MIPI CSI interfaces is responsible for transmitting half of the images captured by the high-resolution camera. LP Mode data (or the same data simultaneously transmitted via CSI0 Lane0) is transmitted to the real-time parameters of the second LP receiver camera in the second receiving device (CPU).

[0066] During image transmission (MIPI HS Mode), the two MIPI CSI interfaces transmit their respective portions of data to the second HS receiver in the CPU via a Data Pair. The CPU's second LP receiver, based on the camera parameter information transmitted in LP mode, identifies the image data (frame count, resolution, etc.). Simultaneously, the second HS receiver rearranges, merges, restores the clock, and performs data equalization on the image data from the two MIPI CSI interfaces. The merged image data is then sent to the second decoder for routine processing such as decryption, decoding, deserialization, and demapping. The processed image data is then passed to the second image processing unit (GPU), which transmits it to the upper layer to form an image.

[0067] Using the aforementioned method of sharing two sets of MIPICSI interfaces with C-PHY protocol for a single high-resolution camera, let's calculate the data volume using a 108MP, 30Hz, Raw10 RGB camera as an example:

[0068] 1.08*10 8 Pixels × 30Hz × 10bit = 3.24 * 10 10 bps (5)

[0069] The highest-speed C-PHY 1.2 currently has a transmission capacity of 20.13Gbps, so the channel utilization rate is:

[0070] 3.24*10 10 / (2.13*10 10 ×2)≈73% (6)

[0071] The maximum number of frames that can be captured under this transmission volume is calculated as follows:

[0072] 2.13*10 10 ×2 / (1.08*10 8 (Pixels × 10bit) ≈ 39.4Hz (7)

[0073] This frame rate is more than sufficient for taking photos and videos at full resolution.

[0074] It should be noted that the image transmission and processing system based on the MIPI CSI C-PHY protocol provided in this application essentially includes two technical solutions: multiple low-pixel cameras sharing a single image transmission and processing architecture with a MIPI CSI interface using the C-PHY protocol, and a high-pixel camera sharing multiple image transmission and processing architectures with a MIPI CSI interface using the C-PHY protocol. Either of these technical solutions can constitute a complete technical solution on its own. That is, the image transmission and processing system based on the MIPI CSI C-PHY protocol in this application can contain only a high-pixel camera sharing multiple MIPI CSI interfaces with the C-PHY protocol. Of course, the above two technical solutions can also be combined to form a complete technical solution, and the order of combination is not strictly limited. For example, the image transmission and processing system based on the MIPI CSI C-PHY protocol in this application can first include an image transmission and processing architecture with a high-pixel camera sharing multiple MIPI CSI interfaces with the C-PHY protocol, and then include an image transmission and processing architecture with multiple low-pixel cameras sharing a single MIPI CSI interface with the C-PHY protocol.

[0075] Figure 4 An image transmission processing method based on the MIPI CSI C-PHY protocol according to an embodiment of this application is shown, including:

[0076] S101: Acquire image data from N low-pixel cameras, where N is 2 or 3;

[0077] S102: Use the same set of MIPI CSI interfaces with C-PHY protocol to transmit image data captured by N low-pixel cameras to the first receiving device;

[0078] S103: The first receiving device processes the image data collected by N low-pixel cameras and transmits it to the upper layer to form an image.

[0079] Specifically, the internal settings of the low-pixel camera and the first receiving device, as well as the processing flow of the image data acquired by the camera, are described in the corresponding system structure above, and will not be repeated here.

[0080] Figure 5 In addition to an exemplary flowchart of another preferred embodiment of the image transmission processing method based on the MIPI CSI C-PHY protocol provided in the embodiments of this application, it also includes:

[0081] S201: Acquire image data from each high-resolution camera;

[0082] S202: Use at least two sets of MIPI CSI interfaces with C-PHY protocol to transmit the image data captured by each high-pixel camera to the second receiving device;

[0083] S203: The second receiving device processes at least two sets of image data transmitted via the MIPI CSI interface with the C-PHY protocol and transmits them to the upper layer to form an image.

[0084] Specifically, step S203 includes: adding preset feature points before and after each group of image data transmitted via the MIPI CSI interface with the C-PHY protocol; determining whether the stitching is effective based on the repetition of the preset feature points of each group of image data transmitted via the MIPI CSI interface with the C-PHY protocol at the stitching point. If the repetition of the preset feature points of each group of image data transmitted via the MIPI CSI interface with the C-PHY protocol at the stitching point exceeds 95%, then the stitching is considered effective.

[0085] like Figure 6 As shown in the embodiment of this application, an image data stream from a high-pixel camera sharing two sets of MIPI CSI interfaces is provided, including:

[0086] High-resolution cameras acquire image data, which is then encoded and segmented before being transmitted via their respective MIPI CSI interfaces. To ensure effective image stitching and recognition, this application does not simply transmit exactly half of the image data per channel. Instead, it adds pre-defined feature points (such as 16 columns of redundant dummy pixel data) before and after the effective data. The 16 columns of pixels at the stitching position represent the 16 columns of effective data from the other half of the image to be stitched together. This allows for precise identification of the stitching position by recognizing data that overlaps with the other half, ensuring effective data alignment. At the stitching position, it is sufficient to ensure that the 16 columns of dummy pixel data from the two images transmitted from the two MIPI CSI interfaces have at least 95% overlap for effective stitching. The dummy data at non-stitching positions is used for spacing and clock alignment with data from other frames.

[0087] like Figure 7 As shown, this application embodiment provides an image data processing method for a high-pixel camera sharing two sets of MIPI CSI interfaces, including:

[0088] The second decoder takes the merged image data and performs routine processing such as decryption, decoding, deserialization, and demapping. The resulting two sets of MIPI CSI interface-processed data reach the GPU. After reading the data, the GPU reads feature points (such as dummy data, or other feature codes can be used) to find the alignment and stitching positions. Due to different transmission environments (characteristics of physical channels, etc.), differences in the data at the stitching positions are inevitable. The GPU compares and extracts the read feature point data, extracting appropriate parameters to compensate for brightness, micro-distortion, etc. For example, software algorithms can be used to compare and calculate the image data with adjacent data, filtering out abnormal data (such as misalignment, transmission damage, etc.) for modification and correction (e.g., modifying the image data to conform to the variation pattern of adjacent pixel data), reducing the abruptness of the image, thus making the synthesized image without obvious stitching marks.

[0089] It should be noted that although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be executed in a different order. For example, in the image transmission processing method based on the MIPI CSI C-PHY protocol of this application, steps S201-S203 can be executed first, followed by steps S101-S103.

[0090] Figure 8 (a) is a diagram of the existing camera image acquisition, transmission, and processing architecture; Figure 8(b) is an exemplary example of the structure diagram of the image transmission system based on the MIPICSI C-PHY protocol provided in this application. Figure 8 As can be seen, compared with existing designs, the image transmission system based on the MIPI CSI C-PHY protocol provided in this application embodiment only uses 3 sets of MIPI CSI interfaces, saving 1 set. This saves CPU resources and also reserves space for future upgrades to camera functionality (adding cameras with other functions or further increasing camera pixels). Most importantly, this application improves the image acquisition capability of the main camera, enabling it to provide images and videos with higher frame rates, providing a material foundation for achieving high-quality functions (HDR, etc.). HDR stands for High-Dynamic Range, which, compared to ordinary images, can provide more dynamic range and image detail. It synthesizes the final HDR image by using LDR (Low-Dynamic Range) images with different exposure times and the LDR images with the best detail corresponding to each exposure time, which can better reflect the visual effects in the real environment.

[0091] Figure 9 A schematic diagram of the structure of a mobile terminal device according to an embodiment of this application is shown.

[0092] like Figure 9 As shown, in another aspect, this application also provides a mobile terminal device 300, including one or more central processing units (CPUs) 301, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 302 or programs loaded from storage portion 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the system 300. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0093] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0094] In particular, according to embodiments of this disclosure, the above references Figure 4-5 The described process can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing a page generation method. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] In another aspect, this application also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the page generation method described in this application.

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0098] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor; for example, each unit can be a software program located in a computer or mobile smart device, or a separately configured hardware device. The names of these units or modules do not, in some cases, constitute a limitation on the unit or module itself.

[0099] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An image transmission processing system based on MIPI CSI C-PHY protocol, characterized in that, include: N low-pixel cameras, a first receiving device, at least one high-pixel camera, and a second receiving device; wherein the N low-pixel cameras are connected to each other via a communication interface, and the N low-pixel cameras are connected to the first receiving device using the same set of MIPICSI interfaces with C-PHY protocol; N is 2 or 3; Each of the low-pixel cameras is provided with a first camera sensor and a first encoder; the first receiving device includes a first LP receiver, a first HS receiver, a splitter, N first decoders and a first image processing unit; Each of the first camera sensors is used to collect its own camera parameters and transmit the collected camera parameters to one of the low-pixel cameras; the first encoder of the low-pixel camera is used to encode the camera parameters collected by the N first camera sensors and transmit the encoded camera parameters to the first LP receiver in the first receiving device in LP mode during the image data transmission gap. Each of the first camera sensors is used to acquire image data in HS mode and transmit the acquired image data to the first HS receiver through the same set of MIPICSI interfaces with C-PHY protocol, wherein each of the first camera sensors uses a different Lane to transmit data to the first HS receiver. The segmenter is used to group and segment the image data received by the first HS receiver according to the encoded camera parameters received by the first LP receiver, so as to obtain N groups of image data. Each of the first decoders is used to decode each group of image data and transmit the decoded image data to the first image processing unit; The first image processing unit is used to process the decoded image data and transmit the processed image data to the upper layer to form an image; One of the high-pixel cameras uses at least two sets of MIPICSI interfaces with C-PHY protocol to connect to the second receiving device; the second receiving device processes the image data transmitted through the at least two sets of MIPICSI interfaces with C-PHY protocol and transmits it to the upper layer to form an image, specifically including: Preset feature points are added before and after each set of image data transmitted via the MIPICSI interface with C-PHY protocol; Whether the stitching is effective is determined by the repetition of preset feature points of the image data transmitted by the MIPICSI interface with C-PHY protocol at each stitching point. When the repetition of preset feature points of the image data transmitted by the MIPICSI interface with C-PHY protocol at each stitching point exceeds 95%, it is considered an effective stitching.

2. The MIPI CSI C-PHY protocol based image transmission processing system according to claim 1, characterized in that, Each of the high-pixel cameras is equipped with a second camera sensor and a second encoder; the second receiving device includes a second LP receiver, a second HS receiver, a blender, a second decoder, and a second image processing unit; The second camera sensor is configured to collect camera parameters of itself, and the second encoder is configured to encode the camera parameters collected by the second camera sensor and transmit the encoded camera parameter information to the second LP receiver through at least two groups of MIPI CSI interfaces with C-PHY protocol in the LP mode. The second camera sensor is configured to collect image data information, and transmit the collected image data information to the second HS receiver through at least two groups of MIPI CSI interfaces with C-PHY protocol in the HS mode. The blender is configured to blend the image data information transmitted by the second HS receiver through the at least two groups of MIPI CSI interfaces with C-PHY protocol according to the encoded camera parameter information received by the second LP receiver, and deliver the blended image data information to the second decoder. The second decoder is configured to decode the blended image data information and deliver the decoded image data information to the second image processing unit. The second image processing unit is configured to process the decoded image data information and transmit the processed image data to an upper layer to form an image.

3. The image transmission processing method based on the MIPI CSI C-PHY protocol, characterized in that, The processing method is based on the MIPI CSI C-PHY protocol-based image transmission processing system of claim 1 or 2, and the processing method comprises: acquiring image data collected by N low-pixel cameras, N being 2 or 3; transmitting the image data collected by the N low-pixel cameras to a first receiving device through the same group of MIPI CSI interfaces with C-PHY protocol; processing the image data collected by the N low-pixel cameras by the first receiving device and transmitting the processed image data to an upper layer to form an image.

4. The MIPI CSI C-PHY protocol based image transmission processing method according to claim 3, characterized in that, Further comprising: acquiring image data collected by each high-pixel camera; transmitting the image data collected by each high-pixel camera to a second receiving device through at least two groups of MIPI CSI interfaces with C-PHY protocol; processing the image data transmitted by the at least two groups of MIPI CSI interfaces with C-PHY protocol by the second receiving device and transmitting the processed image data to an upper layer to form an image.

5. The MIPI CSI C-PHY protocol based image transmission processing method according to claim 4, characterized in that, The processing of the image data transmitted by the at least two groups of MIPI CSI interfaces with C-PHY protocol by the second receiving device and transmitting the processed image data to an upper layer to form an image specifically comprises: adding preset feature points before and after the image data transmitted by each group of MIPI CSI interfaces with C-PHY protocol; determining whether the alignment and splicing are effective according to the repetition degree of the preset feature points of the image data transmitted by each group of MIPI CSI interfaces with C-PHY protocol at the splicing position.

6. The MIPI CSI C-PHY protocol based image transmission processing method according to claim 5, characterized in that, When the repetition degree of the preset feature points of the image data transmitted by each group of MIPI CSI interfaces with C-PHY protocol at the splicing position exceeds 95%, the alignment and splicing are effective.

7. A mobile terminal device, characterized by The mobile terminal device comprises: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the image transmission processing method based on the MIPI CSI C-PHY protocol according to any one of claims 3-6.

8. A computer readable storage medium storing a computer program, characterized in that, The program is executed by the processor to implement the image transmission processing method based on the MIPI CSI C-PHY protocol according to any one of claims 3-6.

Images