Image processing circuit, image processing method, and electronic device

Abstract

The application discloses an image processing circuit, an image processing method and an electronic device, and belongs to the technical field of electronic products. The image processing circuit comprises a main control chip and an image processing chip. A first PCIE interface of the main control chip is connected with a second PCIE interface of the image processing chip to form a first channel. The main control chip is used for sending camera data to be processed to the image processing chip through the first channel. The image processing chip is used for receiving the camera data to be processed sent by the main control chip through the first channel, and performing image data processing on the camera data to be processed. The image processing chip is also used for sending the processed camera data to a corresponding display screen through a first target channel. The first target channel is at least one of a second channel and a third channel. The second channel corresponds to a first display screen, and the third channel corresponds to a second display screen.

Description

Technical Field

[0001] This application belongs to the field of electronic product technology, specifically relating to an image processing circuit, an image processing method, and an electronic device. Background Technology

[0002] When a standalone image chip is used in a foldable phone, if it is to achieve simultaneous display of the main and secondary screens or support image data processing functions (such as frame interpolation on the secondary screen), it is necessary to add a Mobile Industry Processor Interface (MIPI) switch and a Tear Effect (TE) switch for design, which results in a large motherboard wiring area. Summary of the Invention

[0003] The purpose of this application is to provide an image processing circuit, image processing method, and electronic device that can solve the problem of large motherboard wiring area when existing independent image chips support image data processing functions.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide an image processing circuit, including:

[0006] The system includes a main control chip and an image processing chip. The main control chip includes a first external device interconnect bus interface (Mobile Industry Processor Interface, PCIe) interface. The image processing chip includes a second PCIe interface, a first display serial interface (DSI) transmitting interface, and a second DSI transmitting interface. The first PCIe interface and the second PCIe interface are connected to form a first path, the first DSI transmitting interface is connected to a first display screen through a second path, and the second DSI transmitting interface is connected to a second display screen through a third path.

[0007] The main control chip is used to send the camera data to be processed to the image processing chip through the first channel;

[0008] The image processing chip is used to receive camera data to be processed sent by the main control chip through the first channel, and to perform image data processing on the camera data to be processed to obtain processed camera data.

[0009] The image processing chip is also used to send the processed camera data to a corresponding display screen through a first target path, wherein the first target path is at least one of a second path and a third path, the second path corresponds to the first display screen, and the third path corresponds to the second display screen.

[0010] Secondly, embodiments of this application provide an electronic device, including: an image processing circuit as described in one aspect.

[0011] Thirdly, embodiments of this application provide an image processing method applied to the electronic device described in the second aspect, comprising:

[0012] When the main control chip obtains the camera data to be processed, it sends the camera data to be processed to the image processing chip through the first channel;

[0013] The image processing chip performs image data processing on the camera data to be processed to obtain processed camera data.

[0014] The processed camera data is sent to the corresponding display screen through a first target path, wherein the first target path is at least one of a second path and a third path, the second path corresponds to the first display screen, and the third path corresponds to the second display screen;

[0015] The first PCIe interface of the main control chip is connected to the second PCIe interface of the image processing chip to form the first path, the first DSI transmission interface of the image processing chip is connected to the first display screen to form the second path, and the second DSI transmission interface of the image processing chip is connected to the second display screen to form the third path.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the third aspect.

[0017] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the third aspect.

[0018] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the third aspect.

[0019] In this embodiment, the image processing circuit includes a main control chip and an image processing chip. The main control chip is used to send camera data to be processed to the image processing chip via a first path formed by connecting a first PCIe interface and a second PCIe interface. The image processing chip is used to receive the camera data to be processed sent by the main control chip via the first path, and to perform image data processing on the camera data to be processed to obtain processed camera data. The image processing chip is also used to send the processed camera data to a corresponding display screen via a first target path, wherein the first target path is at least one of a second path and a third path, the second path corresponds to the first display screen, and the third path corresponds to the second display screen. Thus, when the first display screen is a foldable main screen and the second display screen is a foldable secondary screen, the foldable main screen and the foldable secondary screen can support simultaneous display and image data processing functions. Furthermore, this embodiment does not require additional MIPI switches and signal switches, avoiding the problem of a large motherboard wiring area. Compared to designs that require additional MIPI switches and signal switches, the software implementation complexity of this embodiment is low, and it can avoid risks such as screen flickering. Furthermore, since it is not limited by the MIPI specification supported by DDIC, it can support data processing from higher resolution cameras, which can improve the user experience in various scenarios and for taking photos. Attached Figure Description

[0020] Figure 1 This diagram illustrates the architecture of an existing standalone image chip.

[0021] Figure 2 This represents one of the working principle diagrams of existing standalone image processing chips;

[0022] Figure 3 This is the second diagram illustrating the working principle of an existing standalone image chip.

[0023] Figure 4 This represents one of the image processing circuits according to an embodiment of the present invention;

[0024] Figure 5 This represents a second image processing circuit according to an embodiment of the present invention;

[0025] Figure 6 This represents the third image processing circuit according to an embodiment of the present invention;

[0026] Figure 7 A flowchart illustrating the image processing method according to an embodiment of the present invention;

[0027] Figure 8 A structural block diagram illustrating an embodiment of the electronic device of the present invention;

[0028] Figure 9 This is a schematic diagram illustrating the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] The following is in conjunction with the appendix Figures 1 to 3 This section introduces existing standalone image processing chips.

[0032] I. Independent Image Processing Chip Architecture

[0033] like Figure 1 The diagram shows an independent image processing chip architecture. The camera captures RAW data (raw R / G / B data) images and videos, which are transmitted from the camera to the main control chip (Application Processor, AP) via Camera serial interface Transsimition 2 (CSI-TX2). The main control chip converts the RAW data (raw R / G / B data) images or videos into YUV domain images or videos (Y represents luminance, and U and V represent chrominance). The images or videos are then transmitted to the image processing chip via DSI-TX3 (display serial interface 3). The image processing chip then performs noise reduction, super-resolution, frame interpolation, and High Dynamic Range (HDR) rendering. The processed images and videos are sent back to the main control chip via the CSI-TX3 interface. The main control chip stores the processed images or content in its internal storage space and transmits them to the image processing chip DSI-RX1 (display serial transmit interface 31) via DSI-TX1 (display serial receive interface 31). Then, through the chip's internal Analog bypass and DSI-TX2, the data is transmitted to the display screen for real-time preview.

[0034] When a mobile phone plays videos, plays games, or browses pictures, the game, video, and image data are transmitted to the image processing chip through the DSI-TX1 interface. After super-resolution, frame interpolation, and HDR processing within the image processing chip, the data is transmitted to the display screen through the DSI-TX2 interface for display, providing users with a better experience in movies, games, and picture browsing.

[0035] II. Application Schemes of Existing Independent Image Processing Chips

[0036] Option 1: As Figure 2 In the main control chip, DSI0 (Display serial interface 0) is mainly used for the display path, while DS1 (Display serial interface 1) is mainly used for the camera path. Whether the foldable screen uses the main screen or the secondary screen for display needs to be selected using MIPI_SW (Mobile Industry Processor Interface switch) and TE Switch (Tear effect signal switch).

[0037] In other words, the existing main control chip only has two sets of MIPI DSI (DSI0 and DS1) for data transmission. One set of MIPI DSI can only be used for data transmission from the discrete graphics chip's camera. The other set of MIPI DSI needs to support data transmission between the main screen and the secondary screen. This requires adding the two switches mentioned above to select whether to use the main screen or the secondary screen for display. Both the folding main screen and the secondary screen are connected to the image processing chip's DSI TX0 via the MIPI selection switch. The MIPI switch can only select either the main screen or the secondary screen at a time, therefore the main screen and the secondary screen cannot be displayed simultaneously.

[0038] As can be seen, the disadvantages of the above-mentioned Solution 1 are: it does not support simultaneous display on the main and secondary screens, nor does it support noise reduction for simultaneous display on the main and secondary screens, which affects the user's experience in scenarios where dual screens are displayed simultaneously. Adding two switches, the MIPI Switch (Mobile Industry Processor Interface Switch) and the TE Switch (Tear effect Switch), increases the motherboard wiring area, increases costs, and makes the system software implementation scheme switching more complex.

[0039] Option 2: Figure 3In this configuration, the main control chip's DSI0 is primarily used for the display path, while the MIPI 2 (Mobile Industry Processor Interface 2) in the DSI1 (Display serial interface 1) split path is mainly used for the camera path. The MIPI 1 in the DSI1 split path is primarily used for the folding secondary screen display. Here, "split" refers to the separate use of the MIPI transmission channels. MIPI_SW_1, MIPI_SW_2, TE_SW_1 (Tear effect 1), and TE_SW_2 (Tear effect 2) mainly enable the folding secondary screen to utilize the image processing chip's noise reduction, super-resolution, frame interpolation, and HDR processing functions.

[0040] In other words, existing image processing chips only have two data channels. The main control chip and the image processing chip use either DSI0 or MIPI2 for data transmission. MIPI2 is used for camera data transmission, and DSI0 is used for main screen display data transmission. If the foldable secondary screen is to support discrete graphics functionality (referring to the use of discrete graphics' super-resolution, frame interpolation, and HDR processing functions), a selection switch needs to be added. The secondary screen data is transmitted from DSI0 to the discrete graphics, and after the discrete graphics completes its processing, the secondary screen is selected via the switch.

[0041] Specifically, the display data path for the main screen is as follows: data is transmitted from DSI0 of the main control chip to DSI RX0 of the image processing chip, from DSI RX0 to DSI TX0, from DSI TX0 to MIPI_SW_1, and from MIPI_SW_1 to Main LCM (folding main screen). The TE path for the folding main screen is as follows: the folding main screen sends Main LCM TE to TE_SW_1, TE_SW_1 sends TE0 to DSI TX0, and TE0 is transmitted from DSI RX0 to DSI0.

[0042] The secondary display path that does not go through the discrete graphics card is as follows: from the main control chip's DSI1 split MIPI 1 to MIPI_SW_2, and through MIPI_SW_2 to Sub LCM (folding secondary screen); the TE path is as follows: the folding secondary screen sends Sub lcmTE to TE SW2, and TE SW2 sends TE4 to DSI1 split.

[0043] The display path of the foldable secondary screen via the discrete graphics card is as follows: data is transmitted from the main control chip's DSI0 to the image processing chip's DSI RX0, from the image processing chip's DSI RX0 to DSI TX0, from DSI TX0 to MIPI_SW_1, from MIPI_SW_1 to MIPI_SW_2, and from MIPI_SW_2 to the Sub LCM (secondary screen). The TE path: the foldable secondary screen sends Sub LCM TE to TE_SW_2, TE_SW_2 transmits Sub LCM TE to TE_SW_1, TE_SW_1 sends TE3 to DSI TX0, and TE3 is transmitted from DSI RX0 to DSI0.

[0044] As can be seen from the above, the existing discrete graphics architecture only has two MIPI data transmission channels, and MIPI CS1 supports camera data but does not support display data transmission.

[0045] The disadvantages of this second option are: the addition of two MIPI switches and two signal switches increases the motherboard wiring area and cost; the use of super-resolution and frame interpolation for the foldable secondary screen adds two MIPI selection switches, making the entire MIP trace longer, and the display signal attenuates more significantly over the longer trace and MIPI selection switches; the system software implementation is more complex, and there are risks such as screen flickering during screen switching. DSI1 supports split (separate MIPI transmission channels) and requires MIPI 1 and MIPI 2 rates to be the same. However, due to limitations in the MIPI specifications supported by DDIC, the maximum supported camera resolution is limited, affecting the user's photography experience.

[0046] The following is in conjunction with the appendix Figures 4 to 6 The prominent processing circuits provided in the embodiments of this application will be described in detail through specific implementation methods and application scenarios.

[0047] Please refer to Figure 4 This invention provides an image processing circuit, including a main control chip and an image processing chip; the main control chip includes a first external device interconnect bus interface (PCIE); the image processing chip includes a second PCIE interface, a first display serial interface (DSI) transmitting interface, and a second DSI transmitting interface; wherein the first PCIE interface and the second PCIE interface are connected to form a first path, the first DSI transmitting interface is connected to a first display screen through a second path, and the second DSI transmitting interface is connected to a second display screen through a third path.

[0048] The main control chip is used to send the camera data to be processed to the image processing chip through the first channel;

[0049] The image processing chip is used to receive camera data to be processed sent by the main control chip through the first channel, and to perform image data processing on the camera data to be processed to obtain processed camera data.

[0050] The image processing chip is also used to send the processed camera data to a corresponding display screen through a first target path, wherein the first target path is at least one of a second path and a third path, the second path corresponds to the first display screen, and the third path corresponds to the second display screen.

[0051] In this embodiment, a PCIe interface is added to the main control chip and the image processing chip. The camera module can transmit camera data to be processed to the main control chip via the Camera Serial Interface (CSI). The main control chip and the image processing chip interact with each other via the PCIe interface. The image processing chip performs image data processing on the camera data and transmits the processed image data to the first display screen via the first Display Serial Interface (DSI) and to the second display screen via the second DSI. Thus, when the first display screen is a foldable main screen and the second display screen is a foldable secondary screen, both screens can support simultaneous display and image data processing. Furthermore, this embodiment does not require additional MIPI switches and signal switches, avoiding the problem of large motherboard wiring area. Compared to designs that require additional MIPI switches and signal switches, the software implementation complexity of this embodiment is low, avoiding risks such as screen flicker. In addition, since it is not limited by the MIPI specifications supported by DDIC, it can support data processing from higher resolution cameras, improving the user experience in various scenarios and for taking photos.

[0052] Specifically, the main control chip is further configured to receive processed camera data sent by the image processing chip through the first channel; the image processing chip is further configured to send the processed camera data to the main control chip through the first channel. In this embodiment, the main control chip further includes a third DSI transmitting interface and a fourth DSI transmitting interface; the image processing chip further includes a first DSI receiving interface and a second DSI receiving interface; wherein, the third DSI transmitting interface is connected to the first DSI receiving interface to form a fourth channel, and the fourth DSI transmitting interface is connected to the second DSI receiving interface to form a fifth channel;

[0053] The main control chip is also used to interact with the image quantity chip via the second target path, wherein the second target path is at least one of the fourth path and the fifth path, the fourth path is applied to the first data displayed on the first display screen, and the fifth path is applied to the second data displayed on the second display screen.

[0054] See Figure 5 The DSI0 interface (third DSI transmitting interface) of the main control chip is connected to the DSI RX0 (first DSI receiving interface) of the image processing chip to transmit display data of the first display screen; the DSI1 (fourth DSI transmitting interface) of the main control chip and the DSI RX1 (second DSI receiving interface) of the image processing chip are connected to transmit display data of the second display screen.

[0055] in, Figure 5 The TE0 (Tear Effect 0) signal is used for data exchange between the image processing chip and the main control chip. When the main control chip receives the TE0 signal from the image processing chip, it begins transmitting data to the image processing chip. The image processing chip's DSI TX0 (first DSI transmitting interface) is connected to the first display screen for data transmission. Figure 5 The Main_lcm_TE (folding main screen) is used for data exchange between the image processing chip and the first display screen. When the image processing chip receives the Main_lcm_TE signal from the first display screen, it begins data transmission.

[0056] in, Figure 5 The TE1 (Tear Effect 01) signal is used for data exchange between the image processing chip and the main control chip. When the main control chip receives the TE1 signal from the image processing chip, it begins data transmission. The image processing chip's DSI TX1 (first DSI transmitting interface) is connected to the second display screen for data transmission. Figure 5 The Sub_lcm_TE (folding sub-screen) is used for data exchange between the image processing chip and the second display screen. When the image processing chip receives the Sub_lcm_TE signal from the folding sub-screen, it begins data transmission.

[0057] Specifically, the image processing chip is also used for at least one of the following:

[0058] Upon receiving the first data stream, image data processing is performed on the first data stream, and the processed first data stream is sent to the first display screen through the second channel;

[0059] Upon receiving the second data stream, image data processing is performed on the second data stream, and the processed second data stream is sent to the second display screen through the third channel;

[0060] Upon receiving the first data stream and the second data stream, where the first data stream and the second data stream are identical and the resolution of the first display screen is greater than that of the second display screen, the data stream is processed according to the resolution of the second display screen. The second data stream is processed using the image data processing described above, and the processed first data stream is sent to the second display screen via the third channel. The processed first data stream is then magnified to the resolution of the first display screen and sent to the first display screen via the second channel.

[0061] In this embodiment, when the first data and the second data are received, and the first data and the second data are the same, and the resolution of the first display screen is greater than the resolution of the second display screen, the memory usage of the image processing chip can be reduced by processing according to the resolution of the second display screen. Moreover, only one set of image processing chips is needed to enable the first display screen and the second display screen to support image processing functions simultaneously.

[0062] It should be noted that, when receiving both the first and second data streams, and the first and second data streams are identical, and the resolution of the first display screen is greater than that of the second display screen, the data can be processed according to the resolution of the first screen. After image data processing within the image, it can be directly transmitted to the main screen, then scaled down to the resolution of the second display screen before being transmitted to the second display screen. Alternatively, two sets of image processing chips can be used to process the data according to the resolutions of the first and second display screens respectively, and then output to the first and second display screens respectively.

[0063] In one specific embodiment, the image processing chip further includes: an amplification module, used to amplify the image data obtained after the image data processing.

[0064] like Figure 6 As shown, the amplification module is connected between the HDR processing module and the first DSI transmission interface, and is used to amplify the data of the image data processed and output to the first DSI transmission interface.

[0065] In the embodiments of this application, the image data processing includes at least one of the following: noise reduction processing, super-resolution and frame interpolation processing, and high dynamic range (HDR) processing.

[0066] In specific implementation, such as Figure 6As shown in the schematic diagram of the internal structure of the image processing chip, when processing camera data, image data processing includes: noise reduction processing, super-resolution and frame interpolation processing, and HDR processing; when processing display data other than that from the camera, image data processing includes: super-resolution and frame interpolation processing, and HDR processing.

[0067] See Figure 6 In this embodiment of the application, the first DSI receiving interface (DSI-RX4) is also connected to the first DSI transmitting interface (DSI-TX6) through the first analog bypass path (Analog bypass 1) in the image processing chip, and the second DSI receiving interface (DSI-RX5) is also connected to the second DSI transmitting interface (DSI-TX7) through the second analog bypass path (Analogbypass 1) in the image processing chip.

[0068] In this way, by using the simulated bypass path in this embodiment, the image data processing process can be skipped, thus meeting the display requirements of image data that do not require the use of image data processing functions.

[0069] In one specific embodiment, the image processing chip is further configured to transmit the processed camera data back to the main control chip via the first channel.

[0070] Specifically, the main control chip is also used to: transmit corresponding data to the image quantity chip through a third target path, wherein the third target path is at least one of a fourth path and a fifth path, the fourth path corresponds to the amplified processed camera data, and the fifth path corresponds to the processed camera data;

[0071] The image processing chip is also used for at least one of the following: transmitting the magnified camera data to the first display screen via a first analog bypass path and a second path within the image processing chip; and directly transmitting the processed camera data to the second display screen via a second analog bypass path and a third path within the image processing chip.

[0072] In this embodiment, as another method of transmitting camera data, the camera data after image data processing can first be transmitted back to the main control chip through the first path formed by the PCIe interface. The main control chip then transmits the data to the first display screen through the fourth path (the path formed between the third DSI transmitting interface and the first DSI receiving interface), the first analog bypass path, and the first DSI transmitting interface, and / or through the fifth path (the path formed between the fourth DSI transmitting interface and the second DSI receiving interface), the second analog bypass path, and the second DSI transmitting interface.

[0073] This invention also provides an electronic device, including the image processing circuit described above.

[0074] In this embodiment, the electronic device can achieve simultaneous display on the main and secondary screens, noise reduction for simultaneous display on the main and secondary screens, frame interpolation / super-resolution for simultaneous display on the main and secondary screens, and HDR processing for simultaneous display on the main and secondary screens through the image enhancement circuit. This supports higher resolution camera data processing, improving the user experience in various scenarios and for taking photos. Furthermore, it eliminates the need for additional MIPI switches and signal switches, reducing motherboard wiring area, lowering costs, and reducing the complexity of system software implementation.

[0075] Please refer to Figure 7 This invention provides an image processing method applied to the electronic device described above, the method comprising:

[0076] Step 101: When the main control chip obtains the camera data to be processed, it sends the camera data to be processed to the image processing chip through the first channel.

[0077] Step 102: The image processing chip performs image data processing on the camera data to be processed to obtain processed camera data;

[0078] Step 103: Send the processed camera data to the corresponding display screen through the first target channel, wherein the first target channel is at least one of the second channel and the third channel, the second channel corresponds to the first display screen, and the third channel corresponds to the second display screen;

[0079] The first PCIe interface of the main control chip is connected to the second PCIe interface of the image processing chip to form the first path, the first DSI transmission interface of the image processing chip is connected to the first display screen to form the second path, and the second DSI transmission interface of the image processing chip is connected to the second display screen to form the third path.

[0080] The following is an explanation using specific examples.

[0081] Example 1: The foldable screen is in a folded state, and only the secondary screen is working, taking pictures with the camera.

[0082] In this example, such as Figure 6As shown, the data captured by the camera is transmitted to the main control chip via CSI-TX1 and CSI-RX1. The main control chip transmits the camera data to the image processing chip through a first path formed by connecting the first and second PCIe interfaces. The image processing chip processes the camera data through its internal noise reduction module, super-resolution module, frame interpolation module, and HDR processing module. The processed camera data can then be transmitted to the DSI-RX7 interface of the foldable secondary screen via the image processing chip DSI-TX7 (the second DSI transmitting module) for display and preview on the foldable secondary screen. If the processed camera data needs to be saved, it can also be sent back to the main control chip through the first path, where it is stored in the internal storage space for viewing in the photo album.

[0083] Example 2: The foldable screen is in the unfolded state. At this time, only the main screen is working and the camera is being used to take pictures.

[0084] In this example, such as Figure 6 As shown, the data captured by the camera is transmitted to the main control chip via CSI-TX1 and CSI-RX1. The main control chip transmits the camera data to the image processing chip through a first path formed by connecting the first and second PCIe interfaces. The image processing chip processes the camera data through its internal noise reduction module, super-resolution module, frame interpolation module, and HDR processing module. The processed camera data is then transmitted to the DSI-RX6 interface of the foldable main screen via the image processing chip's DSI-TX6 (first DSI transmit interface) for display preview on the foldable main screen. If the processed camera data needs to be saved, it can also be sent back to the main control chip through the first path, where it is stored in the internal storage space for viewing in the photo album.

[0085] Example 3: The foldable screen is in the unfolded state. At this time, the main screen and the secondary screen work simultaneously, and the camera is used to take pictures.

[0086] In this example, such as Figure 6As shown, the data captured by the camera is transmitted to the main control chip via CSI-TX1 and CSI-RX1 of the main control chip. The main control chip transmits the camera data to the image processing chip through a first path formed by connecting the first PCIe interface and the second PCIe interface. The image processing chip processes the camera data through its internal noise reduction module, super-resolution module, frame interpolation module, and HDR processing module. The processed camera data is then transmitted to the DSI-RX6 interface of the foldable main screen via the image processing chip DSI-TX6 (first DSI transmission interface) for display preview on the foldable main screen. Simultaneously, the processed camera data can be transmitted to the DSI-RX7 interface of the foldable secondary screen via the image processing chip DSI-TX7 (second DSI transmission interface) for display preview on the foldable secondary screen.

[0087] It should be noted that since the main screen's resolution is higher than the secondary screen's resolution, image data processing can be performed according to the secondary screen's resolution to reduce memory usage. The camera data can then be magnified before being transmitted to the main screen. Specifically, the magnification module within the image processing chip is connected between the HDR processing module and the first DSI transmission interface, and is used to magnify the image data processed and output to the first DSI transmission interface.

[0088] In this embodiment of the application, the method further includes:

[0089] The image quantity chip interacts with the image quantity chip through a second target path, wherein the second target path is at least one of the fourth path and the fifth path, the fourth path is applied to the first data displayed on the first display screen, and the fifth path is applied to the second data displayed on the second display screen.

[0090] The third DSI transmitting interface of the main control chip is connected to the first DSI receiving interface of the image processing chip to form the fourth channel, and the fourth DSI transmitting interface of the main control chip is connected to the second DSI receiving interface of the image processing chip to form the fifth channel; the first channel data is used to be displayed on the first display screen, and the second channel data is used to be displayed on the second display screen.

[0091] See Figure 5 The DSI0 interface (third DSI transmitting interface) of the main control chip is connected to the DSI RX0 (first DSI receiving interface) of the image processing chip to transmit display data of the first display screen; the DSI1 (fourth DSI transmitting interface) of the main control chip and the DSI RX1 (second DSI receiving interface) of the image processing chip are connected to transmit display data of the second display screen.

[0092] In this embodiment of the application, the method further includes at least one of the following:

[0093] When the image processing chip receives the first channel data, it performs image data processing on the first channel data and sends the processed first channel data to the first display screen through the second channel.

[0094] When the image processing chip receives the second data, it performs image data processing on the second data and sends the processed second data to the second display screen through the third channel.

[0095] When the image processing chip receives the first data and the second data, and the first data is the same as the second data, the first data is deleted, and the second data is processed by the image processing chip. The processed first data is then sent to the second display screen through the third channel. After the processed first data is magnified by the image processing chip, it is sent to the first display screen through the second channel.

[0096] In this embodiment, when the first data and the second data are received, and the first data and the second data are the same, and the resolution of the first display screen is greater than the resolution of the second display screen, the memory usage of the image processing chip can be reduced by processing according to the resolution of the second display screen. Moreover, only one set of image processing chips is needed to enable the first display screen and the second display screen to support image processing functions simultaneously.

[0097] In this embodiment of the application, the image data processing includes at least one of the following: noise reduction processing, super-resolution and frame interpolation processing modules, and HDR processing.

[0098] In specific implementation, such as Figure 6 As shown in the schematic diagram of the internal structure of the image processing chip, when processing camera data, image data processing includes: noise reduction processing, super-resolution and frame interpolation processing, and HDR processing; when processing display data other than that from the camera, image data processing includes: super-resolution and frame interpolation processing, and HDR processing.

[0099] The following section describes the image processing method for taking photos when the camera is not enabled, based on Example 5.

[0100] Example 5: The foldable screen is in the unfolded state, both the main screen and the secondary screen are working, and the discrete graphics function is being used (referring to the use of the discrete graphics' super-resolution and frame interpolation, HDR processing functions, the same below), but the camera is not being used to take pictures.

[0101] In this example, such as Figure 6 As shown, both the main and secondary screens are working, and the content displayed on both screens is exactly the same, only the resolution is different. The main control chip transmits the main screen display data to the image processing chip DSI-RX4 (first DSI receiving interface) through DSI-TX4 (third DSI transmitting interface), and transmits the secondary screen display data to the image processing chip DSI-RX5 (second DSI receiving interface) through DSI-TX5 (fourth DSI transmitting interface). Here, the image processing chip can discard the main screen display data and process the display data according to the resolution of the secondary screen. The data is then processed by the super-resolution and frame interpolation module and the HDR module within the image processing chip.

[0102] The processed display data is transmitted to the DSI-RX7 interface of the foldable secondary screen via the image processing chip DSI-TX7 (second DSI transmission interface) for display on the foldable secondary screen; and the processed display data is magnified to the resolution of the main screen via the upscaling module (UP-Scaling module), and transmitted to the DSI-RX6 of the foldable main screen via the image processing chip DSI-TX6 (first DSI transmission interface) for display on the foldable main screen.

[0103] In one specific embodiment, the above method further includes:

[0104] When the image processing chip receives the first data, it sends the first data to the first display screen through the first analog bypass path and the second path inside the image processing chip.

[0105] When the image processing chip receives the second data, it sends the second data to the first display screen through the second analog bypass path and the third path inside the image processing chip.

[0106] In this way, by using the simulated bypass path in this embodiment, the image data processing process can be skipped, thus meeting the display requirements of image data that do not require the use of image data processing functions.

[0107] The following example 6 describes the image processing methods for images taken without using a camera and without enabling the dedicated graphics card (referring to using the super-resolution and frame interpolation, HDR processing functions of the dedicated graphics card, the same below).

[0108] Example 6: When the foldable screen is in the unfolded state, both the main screen and the secondary screen are working, but the camera is not being used to take pictures and the dedicated graphics card is not being used.

[0109] In this example, such as Figure 6As shown, the main control chip transmits display data to the image processing chip DSI-RX4 (first DSI receiving interface) via DSI-TX4 (third DSI transmitting interface). The data then passes through Analogbypass 1 inside the image processing chip and is transmitted to the DSI-RX6 of the folding main screen via the image processing chip DSI-TX6 (first DSI transmitting interface) for display on the folding main screen.

[0110] In this example, the main control chip transmits display data to the image processing chip DSI-RX5 (second DSI receiving interface) via DSI-TX5 (fourth DSI transmitting interface). The data then passes through Analog bypass 2 inside the image processing chip and is transmitted to the folding sub-screen DSI-RX7 via the image processing chip DSI-TX7 (second DSI transmitting interface) for display on the folding sub-screen.

[0111] As can be seen from the above embodiments, in application scenarios where a camera is not used for taking pictures, and when the discrete graphics card is not used, the data passes through the simulated bypass path inside the image processing chip and is transmitted to the main screen via the first DSI transmission interface of the image processing chip. It is then displayed on the main screen. The data is then transmitted to the secondary screen via the second DSI transmission interface inside the image processing chip, where it is displayed. When the discrete graphics card is used, the data passes through the image processing chip, undergoing super-resolution and frame interpolation modules as well as an HDR processing module to process the data. The processed data is then transmitted to the main screen via the first DSI transmission interface of the image processing chip. It is then displayed on the main screen. The data is then transmitted to the secondary screen via the second DSI transmission interface inside the image processing chip, where it is displayed.

[0112] In this embodiment of the application, after sending the processed camera data to the main control chip through the first channel, the method further includes:

[0113] Data is transmitted to the image processing chip via a third target path, wherein the third target path is at least one of a fourth path and a fifth path, the fourth path corresponding to the amplified processed camera data, and the fifth path corresponding to the processed camera data.

[0114] After transmitting the corresponding data to the image quantity chip via the third target path, the method further includes at least one of the following:

[0115] The magnified camera data is transmitted to the first display screen via the first analog bypass path and the second path within the image processing chip; and / or, the processed camera data is transmitted to the second display screen via the second analog bypass path and the third path within the image processing chip.

[0116] Example 7: The foldable screen is in the folded state, and both the main and secondary screens are working at the same time, and the camera is being used to take pictures.

[0117] In this example, such as Figure 6 As shown, the data captured by the camera is transmitted to the main control chip via CSI-TX1 and CSI-RX1 of the main control chip. The main control chip transmits the camera data to the image processing chip through a first path formed by connecting the first PCIe interface and the second PCIe interface. The image processing chip processes the camera data through its internal noise reduction module, super-resolution module, frame interpolation module, and HDR processing module.

[0118] The image processing chip can also use the first path formed by the PCIe interface to send the processed data back to the main control chip. The main control chip then transmits the data to the image processing chip's DSI-RX4 (second DSI receiving interface) via DSI-TX4 (fourth DSI transmitting interface), and then through the image processing chip's internal Analog bypass 2 to the image processing chip's DSI-TX6 (fourth DSI transmitting interface) for display on the secondary screen.

[0119] And / or, the main control chip transmits data to the image processing chip DSI-RX5 (first DSI receiving interface) via DSI-TX5 (third DSI transmitting interface), and then transmits the data to the image processing chip's DSI-TX7 (third DSI transmitting interface) via Analog bypass 2 inside the image processing chip, and displays it on the main screen.

[0120] In this embodiment, as another method of transmitting camera data, the camera data after image data processing can first be transmitted back to the main control chip through the first path formed by the PCIe interface. The main control chip then transmits the data to the first display screen through the fourth path (the path formed between the third DSI transmitting interface and the first DSI receiving interface), the first analog bypass path, and the second DSI transmitting interface, and / or through the fifth path (the path formed between the fourth DSI transmitting interface and the second DSI receiving interface), the second analog bypass path, and the second DSI transmitting interface.

[0121] Optionally, such as Figure 8As shown, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can be run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various process steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0122] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0123] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0124] The electronic device 900 includes, but is not limited to, components such as: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910.

[0125] Those skilled in the art will understand that the electronic device 900 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 910 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0126] The processor 910 is configured to, when the main control chip acquires camera data to be processed, send the camera data to be processed to the image processing chip via a first path; perform image data processing on the camera data to be processed by the image processing chip to obtain processed camera data; and send the processed camera data to a corresponding display screen via a first target path, wherein the first target path is at least one of a second path and a third path, the second path corresponds to the first display screen, and the third path corresponds to the second display screen; wherein the first PCIe interface of the main control chip is connected to the second PCIe interface of the image processing chip to form the first path, the first DSI transmitting interface of the image processing chip is connected to the first display screen to form the second path, and the second DSI transmitting interface of the image processing chip is connected to the second display screen to form the third path.

[0127] Optionally, the processor 910 is further configured to interact with the image processing chip via a second target path, wherein the second target path is at least one of the fourth and fifth paths, the fourth path being for first data displayed on the first display screen, and the fifth path being for second data displayed on the second display screen; wherein the third DSI transmitting interface of the main control chip is connected to the first DSI receiving interface of the image processing chip to form the fourth path, and the fourth DSI transmitting interface of the main control chip is connected to the second DSI receiving interface of the image processing chip to form the fifth path; the first data is used for display on the first display screen, and the second data is used for display on the second display screen.

[0128] Optionally, the processor 910 is also used for at least one of the following:

[0129] When the image processing chip receives the first channel data, it performs image data processing on the first channel data and sends the processed first channel data to the first display screen through the second channel.

[0130] When the image processing chip receives the second data, it performs image data processing on the second data and sends the processed second data to the second display screen through the third channel.

[0131] Upon receiving the first data stream and the second data stream, where the first data stream and the second data stream are identical and the resolution of the first display screen is greater than that of the second display screen, the data stream is processed according to the resolution of the second display screen. The second data stream is processed using the image data processing described above, and the processed first data stream is sent to the second display screen via the third channel. The processed first data stream is then magnified to the resolution of the first display screen and sent to the first display screen via the second channel.

[0132] Optionally, the image data processing includes at least one of the following: noise reduction processing, super-resolution and frame interpolation processing modules, and HDR processing.

[0133] Optionally, the processor 910 is also used for at least one of the following:

[0134] When the image processing chip receives the first data, it sends the first data to the first display screen through the first analog bypass and the second path inside the image processing chip.

[0135] When the image processing chip receives the second data, it sends the second data to the first display screen through the second analog bypass and the third path inside the image processing chip.

[0136] Optionally, the processor 910 is also configured to transmit the processed camera data back to the main control chip via the first channel.

[0137] Optionally, the processor 910 is further configured to transmit corresponding data to the image quantity chip via a third target path, wherein the third target path is at least one of a fourth path and a fifth path, the fourth path corresponding to the magnified processed camera data, and the fifth path corresponding to the processed camera data.

[0138] Optionally, the processor 910 is also used for at least one of the following:

[0139] The magnified camera data is transmitted to the first display screen through the first analog bypass path and the second path within the image processing chip;

[0140] The processed camera data is transmitted to the second display screen via the second analog bypass path and the third path within the image processing chip.

[0141] It should be understood that, in this embodiment, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042. The GPU 9041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0142] The memory 909 can be used to store software programs and various data. The memory 909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 909 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0143] Processor 910 may include one or more processing units; optionally, processor 910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 910.

[0144] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0145] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0146] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0147] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0148] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0150] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0151] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing circuit, characterized in that, include: The system includes a main control chip and an image processing chip. The main control chip includes a first external device interconnect bus interface (PCIE) interface. The image processing chip includes a second PCIE interface, a first display serial interface (DSI) transmitting interface, and a second DSI transmitting interface. The first PCIE interface and the second PCIE interface are connected to form a first path. The first DSI transmitting interface is connected to a first display screen through a second path. The second DSI transmitting interface is connected to a second display screen through a third path. The main control chip is used to send the camera data to be processed to the image processing chip through the first channel; The image processing chip is used to receive camera data to be processed sent by the main control chip through the first channel, and to perform image data processing on the camera data to be processed to obtain processed camera data; wherein, when receiving first data and second data, the first data and the second data are the same, and the resolution of the first display screen is greater than the resolution of the second display screen, the second data is processed according to the resolution of the second display screen, and the image data processing is performed on the second data. The image processing chip is also used to send the processed second channel data to the second display screen through the third channel, and to amplify the processed second channel data to the resolution of the first display screen before sending it to the first display screen through the second channel.

2. The image processing circuit according to claim 1, characterized in that, The main control chip further includes a third DSI transmitting interface and a fourth DSI transmitting interface; the image processing chip further includes a first DSI receiving interface and a second DSI receiving interface; wherein, the third DSI transmitting interface is connected to the first DSI receiving interface to form a fourth path, and the fourth DSI transmitting interface is connected to the second DSI receiving interface to form a fifth path. The main control chip is also used to interact with the image processing chip through a second target path, wherein the second target path is at least one of the fourth path and the fifth path, the fourth path is applied to the first data displayed on the first display screen, and the fifth path is applied to the second data displayed on the second display screen.

3. The image processing circuit according to claim 2, characterized in that, The image processing chip is also used for at least one of the following: Upon receiving the first data stream, image data processing is performed on the first data stream, and the processed first data stream is sent to the first display screen through the second channel; Upon receiving the second data stream, image data processing is performed on the second data stream, and the processed second data stream is sent to the second display screen through the third channel.

4. The image processing circuit according to claim 1 or 2, characterized in that, The image data processing includes at least one of the following: noise reduction processing, super-resolution and frame interpolation processing, and high dynamic range (HDR) processing.

5. The image processing circuit according to claim 1 or 2, characterized in that, The image processing chip further includes an amplification module for amplifying the image data obtained after the image data processing.

6. The image processing circuit according to claim 2, characterized in that, The first DSI receiving interface is also connected to the first DSI transmitting interface through a first analog bypass path within the image processing chip, and the second DSI receiving interface is also connected to the second DSI transmitting interface through a second analog bypass path within the image processing chip.

7. The image processing circuit according to claim 2, characterized in that, The image processing chip is also used to transmit the processed camera data back to the main control chip through the first channel.

8. The image processing circuit according to claim 7, characterized in that, The main control chip is also used to: transmit corresponding data to the image processing chip through a third target path, wherein the third target path is at least one of the fourth path and the fifth path, the fourth path corresponds to the amplified processed camera data, and the fifth path corresponds to the processed camera data; The image processing chip is also used for at least one of the following: The magnified camera data is transmitted to the first display screen through the first analog bypass path and the second path within the image processing chip; The processed camera data is directly transmitted to the second display screen via the second analog bypass path and the third path within the image processing chip.

9. An electronic device, characterized in that, Includes the image processing circuit as described in any one of claims 1 to 8.

10. An image processing method, characterized in that, The method includes: Once the main control chip obtains the camera data to be processed, it sends the camera data to the image processing chip through the first channel. The image processing chip performs image data processing on the camera data to be processed to obtain processed camera data. Specifically, when the image processing chip receives a first channel of data and a second channel of data, where the first channel of data is identical to the second channel of data, and the resolution of the first display screen is greater than the resolution of the second display screen, the second channel of data is processed according to the resolution of the second display screen. The processed second channel of data is then sent to the second display screen via a third channel. Additionally, the processed second channel of data is magnified to the resolution of the first display screen and then sent to the first display screen via a second channel. The first PCIe interface of the main control chip is connected to the second PCIe interface of the image processing chip to form the first path, the first DSI transmission interface of the image processing chip is connected to the first display screen to form the second path, and the second DSI transmission interface of the image processing chip is connected to the second display screen to form the third path.

11. The image processing method according to claim 10, characterized in that, The method further includes: The image processing chip interacts with the image processing chip through a second target path, wherein the second target path is at least one of a fourth path and a fifth path, the fourth path is applied to the first data displayed on the first display screen, and the fifth path is applied to the second data displayed on the second display screen. The third DSI transmitting interface of the main control chip is connected to the first DSI receiving interface of the image processing chip to form the fourth channel, and the fourth DSI transmitting interface of the main control chip is connected to the second DSI receiving interface of the image processing chip to form the fifth channel; the first channel data is used to be displayed on the first display screen, and the second channel data is used to be displayed on the second display screen.

12. The image processing method according to claim 11, characterized in that, The method further includes at least one of the following: When the image processing chip receives the first channel data, it performs image data processing on the first channel data and sends the processed first channel data to the first display screen through the second channel. When the image processing chip receives the second data, it performs image data processing on the second data and sends the processed second data to the second display screen through the third channel.

13. The image processing method according to claim 10 or 11, characterized in that, The image data processing includes at least one of the following: noise reduction processing, super-resolution and frame interpolation processing, and high dynamic range (HDR) processing.

14. The image processing method according to claim 11, characterized in that, The method further includes at least one of the following: When the image processing chip receives the first data, it sends the first data to the first display screen through the first analog bypass path and the second path inside the image processing chip. When the image processing chip receives the second data, it sends the second data to the second display screen through the second analog bypass path and the third path inside the image processing chip.

15. The image processing method according to claim 11, characterized in that, After processing the camera data to be processed using the image processing chip to obtain the processed camera data, the method further includes: The processed camera data is transmitted back to the main control chip through the first channel.

16. The image processing method according to claim 15, characterized in that, After transmitting the processed camera data back to the main control chip through the first channel, the method further includes: Data is transmitted to the image processing chip via a third target path, wherein the third target path is at least one of a fourth path and a fifth path, the fourth path corresponds to the amplified processed camera data, and the fifth path corresponds to the processed camera data.

17. The image processing method according to claim 16, characterized in that, After transmitting the corresponding data to the image processing chip via the third target path, the method further includes at least one of the following: The magnified camera data is transmitted to the first display screen through the first analog bypass path and the second path within the image processing chip; The processed camera data is transmitted to the second display screen via the second analog bypass path and the third path within the image processing chip.

Images