Image Processing System and Electronic Device

Abstract

The present application discloses an image processing system and an electronic device, relating to the technical field of image processing. The system includes a first application processor, a display processing unit of a second application processor, and an image signal processor. The input end of the first application processor is connected to the output end of an image sensor to receive first image data collected by the image sensor. The first output end is connected to the first input end of the display processing unit to perform first image processing on the first image data, obtain second image data, and send the second image data to the display processing unit. The first output end of the display processing unit is connected to the first input end of the image signal processor to perform second image processing on the second image data, obtain third image data, and send the third image data to the image signal processor. The first output end of the image signal processor is connected to the first input end of the second application processor to perform third image processing on the third image data, obtain fourth image data, and send the fourth image data to the second application processor.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to an image processing system and an electronic device. Background Art

[0002] As one of the important functions of smart phones, the camera function has become one of the core tracks for major smart phone manufacturers in technological innovation and improving user experience. Currently, major smart phone manufacturers externally attach an additional chip to the application processor (AP) of the smart phone to implement personalized camera functions in order to gain a differential competitive advantage.

[0003] However, the specific implementation method of externally attaching an additional chip to the AP chip of the current smart phone is as follows: An external chip is connected between the AP and the image sensor that collects images, and the MIPI interface is used for image transmission. This results in the external chip being able to only process the RAW domain image data output by the image sensor. And most current image processing algorithms are based on the YUV domain. Therefore, the image processing algorithms that the external chip can use are limited. Also, in the case of multiple image sensors, the same number of external chips as the number of image sensors need to be connected. Summary of the Invention

[0004] This application aims to provide an image processing system and an electronic device, which can solve the problems that the image processing algorithms that the external chip can use are limited, and in the case of multiple image sensors, the same number of external chips as the number of image sensors need to be connected.

[0005] In a first aspect, an embodiment of this application proposes an image processing system, a first application processor, a second application processor, a display processing unit, and an image signal processor;

[0006] The input end of the first application processor is connected to the output end of the image sensor, and is used to receive the first image data collected by the image sensor. The first output end of the first application processor is connected to the first input end of the display processing unit, and is used to perform first image processing on the first image data to obtain second image data, and send the second image data to the display processing unit;

[0007] The first output end of the display processing unit is connected to the first input end of the image signal processor, and is used to receive the second image data, perform second image processing on the second image data to obtain third image data, and send the third image data to the image signal processor;

[0008] The first output terminal of the image signal processor is connected to the first input terminal of the second application processor, and is configured to receive the third image data, perform third image processing on the third image data to obtain fourth image data, and send the fourth image data to the second application processor.

[0009] In a second aspect, an electronic device is provided, which includes the image processing system of the first aspect.

[0010] In an embodiment of the present application, an image processing system is provided. The system includes a first application processor, a second application processor, a display processing unit, and an image signal processor. The input terminal of the first application processor is connected to the output terminal of the image sensor, and is configured to receive first image data collected by the image sensor. The first output terminal of the first application processor is connected to the first input terminal of the display processing unit, and is configured to perform first image processing on the first image data to obtain second image data, and send the second image data to the display processing unit. The first output terminal of the display processing unit is connected to the first input terminal of the image signal processor, and is configured to receive the second image data, perform second image processing on the second image data to obtain third image data, and send the third image data to the image signal processor. The first output terminal of the image signal processor is connected to the first input terminal of the second application processor, and is configured to receive the third image data, perform third image processing on the third image data to obtain fourth image data, and send the fourth image data to the second application processor. In an embodiment of the present application, a new architecture of an image processing system is provided. In this architecture, the externally connected image signal processor is located after the first application processor. Thus, on the one hand, there is no need to set a corresponding externally connected image signal processor for each image sensor, thereby reducing the hardware cost and power consumption of the image processing system. On the other hand, the first application processor performs first image processing on the first image data, and second image data in the YUV domain can be obtained. In this way, most current image processing algorithms can be customized on the externally connected image signal processor, so that the functions that can be realized by the externally connected image signal processor can be increased. Description of the Drawings

[0011] Figure 1 is a schematic structural diagram of an image processing system provided by an embodiment of the present application Figure 1 ;

[0012] Figure 2 is a schematic structural diagram of an image processing system provided by an embodiment of the present application Figure 2 ;

[0013] Figure 3 is a schematic structural diagram of an image processing system provided by an embodiment of the present application Figure 3 ;

[0014] Figure 4 It is a schematic structure diagram of an image processing system provided by an embodiment of the present application Figure 4 ;

[0015] Figure 5 It is a schematic structure diagram of an image processing system provided by an embodiment of the present application Figure 5 ;

[0016] Figure 6 It is a schematic structure diagram of an image processing system provided by an embodiment of the present application Figure 6 ;

[0017] Reference numerals:

[0018] 100 - Image sensor; 200 - First application processor; 300 - Display processing unit;

[0019] 400 - Image signal processor; 500 - Second application processor; 600 - Encoding module;

[0020] 700 - Camera application module; 800 - Display module. Detailed implementation manners

[0021] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0022] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. generally belong to the same category, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally represents an "or" relationship between the associated objects before and after.

[0023] Next, the image processing system and electronic device provided by the embodiments of the present application will be described in detail in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0024] An embodiment of the present application provides an image processing system, as Figure 1 shown. The image processing system includes a first application processor 200, a second application processor 500, a display processing unit 300, and an image signal processor 400;

[0025] The input end of the first application processor 200 is connected to the output end of the image sensor 100, and is used to receive the first image data collected by the image sensor 100. The first output end of the first application processor 200 is connected to the first input end of the display processing unit 300, and is used to perform first image processing on the first image data to obtain second image data, and send the second image data to the display processing unit 300;

[0026] The first output end of the display processing unit 300 is connected to the first input end of the image signal processor 400, and is used to receive the second image data, perform second image processing on the second image data to obtain third image data, and send the third image data to the image signal processor 400;

[0027] The first output end of the image signal processor 400 is connected to the first input end of the second application processor 500, and is used to receive the third image data, perform third image processing on the third image data to obtain fourth image data, and send the fourth image data to the second application processor 500.

[0028] In an embodiment of the present application, as Figure 2 shown, the image processing system provided by the embodiment of the present application further includes at least one image sensor 100, wherein the image sensor 100 is used to collect first image data and send the first image data to the first application processor 200.

[0029] In an embodiment of the present application, at least one image sensor 100 may be at least one of an ultra-wide-angle image sensor, a 2x focal length image sensor, a 5x focal length image sensor, a main camera image sensor, etc.

[0030] It should be noted that in Figure 2 it is shown by taking the image processing system including one image sensor 100 as an example.

[0031] In the embodiment of the present application, the image data collected by the image sensor 100 is recorded as the first image data. After the image sensor 100 collects the first image data, step S1 is executed, where S1 is to send the first image data to the application processor 200. Among them, the first image data is an unprocessed RAW domain raw image.

[0032] The first application processor 200 receives the first image data. After receiving the first image data, it executes step S2, where S2 is to perform first image processing on the first image data to obtain second image data, and send the second image data to the display processing unit 300. Among them, the first image processing is specifically image preprocessing and image domain conversion processing. The image preprocessing can be, for example, basic correction and noise reduction processing, and the image domain conversion processing is to convert the RAW domain to the YUV domain.

[0033] The display processing unit (DPU) 300 receives the second image data. After receiving the second image data, it executes step S3, where S3 is to perform second image processing on the second image data to obtain third image data, and send the third image data to the image signal processor 400. Among them, the second image processing is specifically image display effect enhancement processing.

[0034] The image signal processing (ISP) processor 400 receives the third image data. After receiving the third image data, it executes step S4, where S4 is to perform third image processing on the third image data to obtain fourth image data, and send the fourth image data to the second application processor 500. Among them, the third image processing is customized image processing, such as night scene enhancement processing. In this way, the data processing speed of the image signal processor 400 is fast and the energy consumption is low.

[0035] In an embodiment of the present application, the second application processor 500 is used to receive the fourth image data. After receiving the fourth image data, it executes step S5, and S5 is to perform fourth image processing on the fourth image data to obtain fifth image data. Among them, the fourth image processing is, for example, re - correction, etc.

[0036] In an embodiment of the present application, an image processing system is provided. The system includes a first application processor, a second application processor, a display processing unit, and an image signal processor. The input end of the first application processor is connected to the output end of the image sensor, and is configured to receive first image data collected by the image sensor. The first output end of the first application processor is connected to the first input end of the display processing unit, and is configured to perform first image processing on the first image data to obtain second image data, and send the second image data to the display processing unit. The first output end of the display processing unit is connected to the first input end of the image signal processor, and is configured to receive the second image data, perform second image processing on the second image data to obtain third image data, and send the third image data to the image signal processor. The first output end of the image signal processor is connected to the first input end of the second application processor, and is configured to receive the third image data, perform third image processing on the third image data to obtain fourth image data, and send the fourth image data to the second application processor. In an embodiment of the present application, a new architecture of an image processing system is provided. In this architecture, the externally connected image signal processor is located after the first application processor. In this way, on the one hand, there is no need to set a corresponding externally connected image signal processor for each image sensor, thereby reducing the hardware cost and power consumption of the image processing system. On the other hand, the first application processor performs first image processing on the first image data, and second image data in the YUV domain can be obtained. In this way, most current image processing algorithms can be customized on the externally connected image signal processor, so that the functions that can be implemented by the externally connected image signal processor can be increased.

[0037] In an embodiment of the present application, as Figure 3 shown, the image processing system provided by the embodiment of the present application further includes an encoding module 600;

[0038] The input end of the encoding module 600 is connected to the first output end of the second application processor 500, and is configured to obtain fifth image data from the second application processor 500 and encode the fifth image data.

[0039] In an embodiment of the present application, S5 further includes sending the fifth image data to the encoding module 600. That is to say, after obtaining the fifth image data, the second application processor 200 sends the fifth image data to the encoding module 600.

[0040] The encoding module 600 encodes the fifth image data, and thus saves it as a video file in a fixed storage path. In this way, the user can view the video file in the fixed storage path.

[0041] In an embodiment of the present application, as Figure 4As shown in the figure, the image processing system provided by the embodiment of the present application further includes a camera application module 700 and a display module 800;

[0042] The input end of the camera application module 700 is connected to the second output end of the second application processor 500, and is used to obtain the fifth image data from the second application processor 500. The output end of the camera application module 700 is connected to the second input end of the display processing unit 300, and is used to send the fifth image data and the preview display interface data to the display processing unit 300;

[0043] The second output end of the display processing unit 300 is connected to the second input end of the image signal processor 400, and is used to receive the fifth image data and the preview display interface data, synthesize the fifth image data and the preview display interface data to obtain the first synthesized image data, and send the first synthesized image data to the image signal processor 400;

[0044] The second output end of the image signal processor 400 is connected to the display module 800, and is used to send the first synthesized image data to the display module 800;

[0045] The display module 800 is used to receive the first synthesized image data and perform display processing on the first synthesized image data.

[0046] In the embodiment of the present application, the display module 800 is specifically a screen display module or a projection display module. In addition, the camera application module 700 is further used to provide preview display interface data. The preview display interface is specifically the template data displayed on the preview interface, such as the shooting button, the flash icon, etc.

[0047] In the embodiment of the present application, the second application processor 500 further includes sending the fifth image data to the camera application module 700. That is to say, the second application processor 500 is further used to send the fifth image data to the camera application module 700.

[0048] After the camera application module 700 obtains the fifth image data, it executes step S6, where S6 is: sending the fifth image data and the preview display interface data to the display processing unit 300.

[0049] After the display processing unit 300 receives the fifth image data and the preview display interface data, it executes step S7, where S7 is to synthesize the fifth image data and the preview display interface data to obtain the first synthesized image data, and send the first synthesized image data to the image signal processor 400.

[0050] After the image signal processor 400 receives the first synthesized image data, it executes S8, where S8 is to send the first synthesized image data to the display module 800.

[0051] After the display module 800 obtains the first synthesized image data, it performs display processing on the first image data. Based on this, the user can view the preview screen through the interface of the electronic device.

[0052] In an embodiment of the present application, the image signal processor 400 includes multiple different pipelines. On this basis, the image signal processor 400 sends the first synthesized image data to the display module 800 in a bypass manner, where the bypass path is the channel pipeline in the image signal processor that connects to the display module 800. In this way, a physical transmission is achieved, thereby reducing the power consumption of the image processing system provided in the embodiment of the present application and providing the transmission speed of the first image data.

[0053] In an embodiment of the present application, as Figure 5 shown, the image processing system provided in the embodiment of the present application further includes a camera application module 700 and a display module 800;

[0054] The output end of the camera application module 700 is connected to the second input end of the display processing unit 300, and is used to send preview display interface data to the display processing unit 300;

[0055] Performing second image processing on the second image data to obtain third image data, including:

[0056] Receiving preview display interface data;

[0057] Synthesizing the preview display interface data and the second image data to obtain second synthesized image data;

[0058] Performing second image processing on the second synthesized image data to obtain third image data;

[0059] Performing third image processing on the third image data to obtain fourth image data, including:

[0060] Splitting the preview display interface data and the sixth image data from the third image data, and performing third image processing on the sixth image data to obtain fourth image data.

[0061] The second output end of the image signal processor 400 is connected to the display module 800, and is used to synthesize the fourth image data and the preview display interface data to obtain third synthesized image data, and send the third synthesized image data to the display module 800;

[0062] The display module 800 is used to perform display processing on the third synthesized image data.

[0063] In an embodiment of the present application, the camera application module 700 executes step S9, where S9 is to send the preview display interface data to the display processing unit 300.

[0064] In an embodiment of the present application, S3 executed by the display processing unit 300 specifically is: receiving the preview display interface data; synthesizing the preview display interface data and the second image data to obtain second synthesized image data; performing second image processing on the second synthesized image data to obtain third image data; and sending the third image data to the image signal processor 400.

[0065] Also, the image signal processor 400 executes step S4, where S4 specifically is: splitting the preview display interface data and the sixth image data from the third image data, performing third image processing on the sixth image data to obtain fourth image data, and sending the fourth image data to the second application processor 500. Here, the sixth image data is the image data corresponding to the second image data in the third image data and obtained after the third image processing.

[0066] The image signal processor 400 also executes step S10, where S10 is to synthesize the fourth image data and the preview display interface data to obtain third synthesized image data, and send the third synthesized image data to the display module 800.

[0067] In an embodiment of the present application, the second application processor 500 only needs to set one output interface, which can be applied to low-end platforms. For dual-screen electronic devices, it is easier to deploy.

[0068] In an embodiment of the present application, as Figure 6 shown, the image processing system provided by the embodiment of the present application further includes: a camera application module 700 and a display module 800;

[0069] The second output end of the first application processor 200 is connected to the input end of the camera application module 700 for sending the second image data to the camera application module 700;

[0070] The output end of the camera application module 700 is connected to the display module 800 for sending the second image data and the preview display interface data to the display module 800;

[0071] The display module 800 is used for performing display processing on the preview display interface data and the second image data.

[0072] In an embodiment of the present application, the first application processor 200 is further used to execute step S11, where S11 is: sending the second image data to the camera processing module 700.

[0073] The camera processing module is used to execute step S12, where S12 is to send the preview display interface data and the second image data to the display module 800.

[0074] After obtaining the preview display interface data and the second image data, the display module 800 directly displays them, and the display effect is that the preview display interface data is superimposed on the second image data.

[0075] In the embodiment of the present application, since the user pays more attention to the video file saved by encoding than the video file displayed in preview. Thus, in the embodiment of the present application, the image data corresponding to the video file displayed in preview does not pass through the display processing unit 300, the image signal processor 400, and the second application processor 500, which can greatly reduce the delay of preview and at the same time reduce the computing overhead of the image processing system provided by the embodiment of the present application.

[0076] Based on any of the above embodiments, the display processing unit 300 and the image signal processor 400 are connected through an MIPI interface.

[0077] Moreover, the image signal processor 400 and the second application processor 200 are connected through an MIPI interface.

[0078] In addition, the application processor 200 and the display processing unit 300 are connected through a DPU interface.

[0079] For any of the above embodiments, the image processing system provided by the embodiment of the present application provides two processing paths. The first processing path is the path shown by the thin line in Figures 4 to 6 The second processing path is the path shown by the thick line in Figures 4 to 6 In this way, the flexibility of the framework of the image processing system can be realized to flexibly customize the image processing system provided by the embodiment of the present application.

[0080] In the embodiment of the present application, the embodiment of the present application also provides an electronic device, which includes the image processing system provided by any of the above embodiments.

[0081] Exemplarily, the electronic device can be a mobile phone, a tablet computer, a handheld computer, an augmented reality (AR), a virtual reality (VR) device, a robot, a wearable device, etc., or can also be a teller machine or a self-service machine, etc. The embodiment of the present application does not make specific limitations.

[0082] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the image processing methods described in various embodiments of the present application.

[0083] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. An image processing system, characterized in that, Including: A first application processor, a second application processor, a display processing unit, and an image signal processor; The input end of the first application processor is connected to the output end of the image sensor, and is used to receive the first image data collected by the image sensor. The first output end of the first application processor is connected to the first input end of the display processing unit, and is used to perform first image processing on the first image data to obtain second image data, and send the second image data to the display processing unit; The first output end of the display processing unit is connected to the first input end of the image signal processor, and is used to receive the second image data, perform second image processing on the second image data to obtain third image data, and send the third image data to the image signal processor, wherein the second image processing is image display effect enhancement processing; The first output end of the image signal processor is connected to the first input end of the second application processor, and is used to receive the third image data, perform third image processing on the third image data to obtain fourth image data, and send the fourth image data to the second application processor, wherein the third image data is image data in the YUV domain; The second application processor is further used to receive the fourth image data and perform fourth image processing on the fourth image data to obtain fifth image data; The image processing system further includes: an encoding module; The input end of the encoding module is connected to the first output end of the second application processor, and is used to obtain the fifth image data from the second application processor and encode the fifth image data; The image processing system further includes: a camera application module and a display module; The input end of the camera application module is connected to the second output end of the second application processor, and is used to obtain the fifth image data from the second application processor. The output end of the camera application module is connected to the second input end of the display processing unit, and is used to send the fifth image data and preview display interface data to the display processing unit; The second output end of the display processing unit is connected to the second input end of the image signal processor, and is used to receive the fifth image data and the preview display interface data, synthesize the fifth image data and the preview display interface data to obtain first synthesized image data, and send the first synthesized image data to the image signal processor; The second output end of the image signal processor is connected to the display module, and is used to send the first synthesized image data to the display module; The display module is used to receive the first synthesized image data and perform display processing on the first synthesized image data.

2. The system according to claim 1, wherein The image signal processor sends the first synthesized image data to the display module in a bypass manner.

3. The system according to claim 1, characterized in that, The image processing system further includes: a camera application module and a display module; The output end of the camera application module is connected to the second input end of the display processing unit, and is used to send preview display interface data to the display processing unit; Performing second image processing on the second image data to obtain third image data includes: Receiving the preview display interface data; Combining the preview display interface data and the second image data to obtain second combined image data; Performing second image processing on the second combined image data to obtain third image data; Performing third image processing on the third image data to obtain fourth image data includes: Splitting the preview display interface data and sixth image data from the third image data, and performing third image processing on the sixth image data to obtain fourth image data; The second output end of the image signal processor is connected to the display module, and is used for combining the fourth image data and the preview display interface data to obtain third combined image data, and sending the third combined image data to the display module; The display module is used for performing display processing on the third combined image data.

4. The system according to claim 1, characterized in that, The image processing system further includes: a camera application module and a display module; The second output end of the first application processor is connected to the input end of the camera application module, and is used for sending the second image data to the camera application module; The output end of the camera application module is connected to the display module, and is used for sending the second image data and the preview display interface data to the display module; The display module is used for performing display processing on the preview display interface data and the second image data.

5. The system according to claim 1, characterized in that The image processing system further includes at least one of the image sensors; The image sensor is used for collecting the first image data and sending the first image data to the first application processor.

6. The system according to claim 1, wherein The display processing unit and the image signal processor are connected through an MIPI interface.

7. An electronic device, comprising the image processing system according to any one of claims 1-6.

Images