Image processing method and device, computer readable storage medium and electronic equipment

Abstract

The invention provides an image processing method, an image processing device, a computer readable storage medium and electronic equipment, is applied to terminal equipment comprising a plurality of hardware processing units, and relates to the technical field of image processing. The image processing method comprises the following steps: carrying out image segmentation on an original image to obtain a plurality of original image blocks; distributing the original image blocks to determine the original image blocks input into the hardware processing units; performing image enhancement processing on the received original image blocks by each hardware processing unit to obtain a plurality of processed image blocks; and splicing the plurality of processed image blocks to generate a processed image corresponding to the original image. The image enhancement processing efficiency can be improved, and the processing time consumption can be reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing method, an image processing device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the development of terminal technology, terminal devices can perform image enhancement processing on images to meet users' requirements for image display effects.

[0003] In the scenario of performing image enhancement processing on images, problems such as high processing pressure and long processing time may occur. Summary of the invention

[0004] The present disclosure provides an image processing method, an image processing device, a computer-readable storage medium and an electronic device, thereby overcoming the problems of high pressure and long time consumption of image processing at least to a certain extent.

[0005] According to a first aspect of the present disclosure, there is provided an image processing method, which is applied to a terminal device, wherein the terminal device includes a plurality of hardware processing units, and the image processing method includes: performing image segmentation on an original image to obtain a plurality of original image blocks; allocating each original image block to determine an original image block input to each hardware processing unit; performing image enhancement processing on the received original image block by each hardware processing unit to obtain a plurality of processed image blocks; and splicing the plurality of processed image blocks to generate a processed image corresponding to the original image.

[0006] According to a second aspect of the present disclosure, there is provided an image processing apparatus, which is applied to a terminal device, wherein the terminal device includes a plurality of hardware processing units, and the image processing apparatus includes: an image segmentation module, which is used to perform image segmentation on an original image to obtain a plurality of original image blocks; an image block allocation module, which is used to allocate each original image block to determine the original image block input to each hardware processing unit; an image block enhancement module, which is used to perform image enhancement processing on the received original image blocks by each hardware processing unit to obtain a plurality of processed image blocks; and an image block splicing module, which is used to splice the plurality of processed image blocks to generate a processed image corresponding to the original image.

[0007] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned image processing method is implemented.

[0008] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising a processor; and a memory for storing one or more programs, wherein when the one or more programs are executed by the processor, the processor implements the above-mentioned image processing method.

[0009] In the technical solutions provided by some embodiments of the present disclosure, an original image is divided into a plurality of original image blocks, and the original image blocks are respectively input into a plurality of hardware processing units of a terminal device, each hardware processing unit performs image enhancement processing on the received original image blocks, and splices the processed image blocks to generate a processed image corresponding to the original image. The present disclosure assigns the original image blocks of the original image to different hardware processing units for processing, and the hardware processing units work together to share the computing pressure, thereby improving the efficiency of image enhancement processing and reducing processing time.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0012] Figure 1 A schematic diagram of an image enhancement process using an image super-resolution algorithm as an example is shown in the present disclosure.

[0013] Figure 2 A schematic diagram showing the processing method of the image processing process of an embodiment of the present disclosure.

[0014] Figure 3 The flowchart schematically shows an image processing method according to an exemplary embodiment of the present disclosure.

[0015] Figure 4 A schematic diagram of image segmentation of an original image according to an embodiment of the present disclosure is shown.

[0016] Figure 5 Shown with Figure 4 Schematic diagram of the allocation of corresponding original image blocks to hardware processing units.

[0017] Figure 6 Shown with Figure 5 Schematic diagram of the corresponding multi-threaded parallel execution of the image enhancement process.

[0018] Figure 7 A block diagram schematically shows an image processing apparatus according to an exemplary embodiment of the present disclosure.

[0019] Figure 8 A block diagram schematically shows an image processing apparatus according to another exemplary embodiment of the present disclosure.

[0020] Fig. 9 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0021] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0022] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0024] The image processing scheme of the embodiment of the present disclosure can be implemented by a terminal device, that is, the terminal device can execute each step of the image processing method of the embodiment of the present disclosure, and the following image processing device can be configured in the terminal device. The present disclosure does not limit the device type of the terminal device, and the terminal device may include but is not limited to a smart phone, a tablet computer, a personal computer, a smart wearable device, and a server.

[0025] The terminal device may include multiple hardware processing units. It should be understood that the hardware processing unit referred to in the present disclosure is a processing unit capable of performing image enhancement processing on an image. Each hardware processing unit can complete the image enhancement task independently. These hardware processing units include, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural-network Processing Unit), and the like.

[0026] The present disclosure does not limit the processing tasks of the image enhancement processing involved in the scheme. For example, the image enhancement processing of the embodiment of the present disclosure includes at least one of the following processing processes: image super-resolution reconstruction, image denoising, image deblurring, image defogging, image contrast enhancement, image demosaicing, image color enhancement, image brightness enhancement, image detail enhancement, image dynamic range enhancement and image style transfer.

[0027] Taking image super-resolution reconstruction as an example, the image processing process of some embodiments of the present disclosure is explained.

[0028] Super-resolution reconstruction can be referred to as super-resolution, which refers to the process of transforming a low-resolution image to obtain a high-resolution image. For example, a video frame image with a resolution of 2K is subjected to 2 times super-resolution to obtain a video frame image with a resolution of 4K. The present disclosure can use a machine learning model to construct a super-resolution model to achieve the super-resolution processing process. The machine learning model here can be a convolutional neural network (CNN), a deep neural network (DNN), etc. The present disclosure does not limit the specific model structure, training process, etc. of the model used in the super-resolution algorithm. In addition, it should be understood that the "high" and "low" resolutions mentioned in the present disclosure are only relative concepts. For example, in some embodiments, 720P can be a high resolution compared to 270P, while in other embodiments, 720P is a low resolution compared to 1080P.

[0029] refer to Figure 1 , after the decoding process, the original image can be subjected to a super-resolution process to obtain a processed image, which is then sent to the display end for display.

[0030] For the super-resolution processing, the terminal device can input the decoded entire image into a hardware processing unit, and then run the super-resolution algorithm in the hardware processing unit.

[0031] However, this image processing process may have the problems of high processing pressure and long processing time.

[0032] For example, in the video playback scenario, in order to achieve a smooth visual experience, the video frame rate can be set to 30fps. In this case, the total processing time of image decoding, super-resolution processing, and display is required to be less than 33ms / frame. Since image enhancement processing such as super-resolution processing is a task that requires a lot of calculations, the processing pressure of the hardware processing unit is high, especially for video frames with higher resolutions, which takes a long time, and it may not be able to meet the image enhancement processing requirements and affect the user experience.

[0033] In order to solve or at least overcome to a certain extent the problem of high pressure and long time consumption of image processing that may occur in terminal devices, the present disclosure provides a new image processing solution.

[0034] Figure 2 A schematic diagram showing the processing method of the image processing process of an embodiment of the present disclosure.

[0035] refer to Figure 2 First, the terminal device can perform image segmentation on the original image to obtain multiple original image blocks. Since the original image blocks are obtained by performing segmentation operations on the original image, it can be understood that the size of the original image blocks is smaller than the size of the original image. In addition, although the segmentation operation disclosed in the present invention is an average segmentation operation, that is, the size of each original image block is the same, it should be understood that the subsequent image processing process can also be performed when the sizes of the segmented original image blocks are not completely the same or completely different. Figure 2 As shown, these original image blocks may include original image block 1, original image block 2, ..., original image block n, where n is a positive integer greater than or equal to 2. It can be understood that when n=2, original image block n is original image block 2.

[0036] Next, the terminal device may distribute the divided original image blocks to multiple hardware processing units as inputs of each hardware processing unit. It should be understood that each original image block is only distributed once, and each hardware processing unit may receive one or more original image blocks. Figure 2 As shown, the multiple hardware processing units in the terminal device may include hardware processing unit 1, hardware processing unit 2, ..., hardware processing unit m, where m is a positive integer greater than or equal to 2. It can be understood that when m=2, hardware processing unit m is hardware processing unit 2.

[0037] Then, each hardware processing unit can perform image enhancement processing on the received original image blocks to obtain multiple processed image blocks. It can be understood that the processed image blocks correspond to the original image blocks one by one. Figure 2 As shown, the processed image blocks may include processed image block 1, processed image block 2, ..., processed image block n.

[0038] Subsequently, the terminal device may splice the processed image blocks to generate a processed image corresponding to the original image, that is, to obtain an image obtained by performing image enhancement on the original image.

[0039] The image processing method according to the embodiment of the present disclosure will be described below.

[0040] Figure 3 The flowchart of the image processing method according to the exemplary embodiment of the present disclosure is schematically shown. Figure 3 , the image processing method may include the following steps:

[0041] S32. Perform image segmentation on the original image to obtain multiple original image blocks.

[0042] The original image in the embodiments of the present disclosure may be an image to be enhanced. The original image may be an image captured by a camera module equipped with a terminal device. For example, the original image may be any video frame image in a video captured by the terminal device. In addition, the original image may also be an image obtained by the terminal device from other devices. The present disclosure does not limit the source, content, or size of the original image.

[0043] After determining the original image, the terminal device may segment the original image to obtain a plurality of original image blocks. It is understandable that, since the original image block is an image segmented from the original image, the size of the original image block is smaller than the size of the original image.

[0044] According to some embodiments of the present disclosure, the terminal device can segment the original image into a predetermined number of blocks, that is, the number of the segmented original image blocks is the predetermined number of blocks. The present disclosure does not limit the specific value of the predetermined number of blocks, for example, the predetermined number of blocks is set to 16, 32, etc.

[0045] According to other embodiments of the present disclosure, the terminal device may segment the original image according to the input size requirement of the subsequent image enhancement model. For example, if the input size of the image enhancement model used in the present disclosure is 256*256, the original image may be segmented according to the requirement that the size of the original image block is 256*256.

[0046] In addition, when the size of the original image is not an integer multiple of the input size required by the model, the original image can be padded before segmentation, that is, the size of the original image can be expanded to an integer multiple of the input size required by the model, and the pixel values ​​of the expanded pixels can be configured to 0 or 255.

[0047] refer to Figure 4 After the original image is processed by image segmentation, for example, 16 original image blocks can be obtained, which are respectively recorded as original image blocks 1 to 16.

[0048] It should be understood that the above segmentation of the original image is an average segmentation, that is, the sizes of the original image blocks obtained are the same. However, in other embodiments of the present disclosure, the original image can also be segmented non-averagely, and the sizes of the segmented original image blocks are not completely the same or completely different.

[0049] S34. Allocate each original image block to determine the original image block input to each hardware processing unit.

[0050] After determining the original image blocks, the terminal device may distribute the original image blocks to the various hardware processing units included in the terminal device as inputs of the various hardware processing units.

[0051] According to some embodiments of the present disclosure, the terminal device may evenly distribute the original image blocks to each hardware processing unit. For example, if there are 16 original image blocks and 4 hardware processing units, each hardware processing unit may receive 4 original image blocks.

[0052] According to other embodiments of the present disclosure, the terminal device may allocate each original image block according to the image processing capability of each hardware processing unit to determine the original image blocks input to each hardware processing unit, and specifically, the number of original image blocks input to each hardware processing unit may be determined. The image processing capability of the hardware processing unit is positively correlated with the number of received original image blocks, that is, the stronger the image processing capability of the hardware processing unit, the more original image blocks are received; the weaker the image processing capability of the hardware processing unit, the fewer original image blocks are received.

[0053] The image processing capability of the hardware processing unit can be characterized based on the time required to perform image enhancement processing on the test image. The process of determining the image processing capability can be a process pre-performed before performing the image processing method of the embodiment of the present disclosure.

[0054] First, the terminal device may obtain the image processing time of each hardware processing unit, where the image processing time is the time required to perform image enhancement processing on the test image.

[0055] It is understandable that, on the one hand, in order to ensure the accuracy of the image processing capability results, the test image used by each hardware processing unit can be the same image; on the other hand, the test image can be a whole image or an image block segmented from an image. The size of the test image can be the same as or different from the size of the original image block. In addition, the present disclosure does not limit the content, source, etc. of the test image.

[0056] Next, the terminal device can determine the image processing capability of each hardware processing unit according to the image processing time of each hardware processing unit. The image processing time of the hardware processing unit is negatively correlated with the image processing capability, that is, the shorter the image processing time of the hardware processing unit, the stronger the image processing capability; the longer the image processing time of the hardware processing unit, the weaker the image processing capability.

[0057] Regarding the process of allocating each original image block according to the image processing capability of each hardware processing unit, the terminal device can determine the proportional relationship of the image processing time of each hardware processing unit, and then allocate the original image blocks according to the proportional relationship to determine the original image blocks input into each hardware processing unit.

[0058] Taking the example of a terminal device including three hardware processing units, namely, NPU, GPU and CPU, for a test image of the same size, if it is determined that the image processing time of NPU is 1ms, the image processing time of GPU is 2.5ms and the image processing time of CPU is 5ms, the original image blocks input to each hardware processing unit can be determined according to the proportional relationship of the image processing time. It should be understood that the image processing time is related to the size of the test image, that is, the image processing time of the hardware processing unit is often different for test images of different sizes, and the specific values ​​of the above image processing time are only exemplary descriptions and are not intended to be a limitation of the disclosed solution.

[0059] Figure 5 Shown with Figure 4 Schematic diagram of the distribution of the corresponding original image blocks to the hardware processing units. Figure 5 , the NPU is assigned to original image blocks 1 to 10, the GPU is assigned to original image blocks 11 to 14, and the CPU is assigned to original image blocks 15 and 16.

[0060] It can be seen that in the process of allocating original image blocks according to the image processing capability in the embodiment of the present disclosure, when it is impossible to perfectly allocate according to the proportional relationship of the image processing time, there may be deviations of one or more original image blocks for individual hardware processing units, such as: Figure 5 The GPU in the example is allocated one less original image block than expected. However, this deviation has no effect on the results of the disclosed solution.

[0061] In view of the fact that the image processing capability of the hardware processing unit of the embodiment of the present disclosure is positively correlated with the number of received original image blocks, for the specific instance in which the image processing time of the NPU is 1ms, the image processing time of the GPU is 2.5ms, and the image processing time of the CPU is 5ms, the allocation method of the original image blocks can also be configured as other situations, in which the NPU is allocated the largest number of original image blocks, followed by the GPU, and the CPU is allocated the least number of original image blocks. For example, the NPU is allocated 11 original image blocks, the GPU is allocated 3 original image blocks, and the CPU is allocated 2 original image blocks; for another example, the NPU is allocated 12 original image blocks, the GPU is allocated 3 original image blocks, and the CPU is allocated 1 original image block; for another example, the NPU is allocated 10 original image blocks, the GPU is allocated 5 original image blocks, and the CPU is allocated 1 original image block; and so on.

[0062] S36. Each hardware processing unit performs image enhancement processing on the received original image block to obtain a plurality of processed image blocks.

[0063] After the original image blocks are input into different hardware processing units, each hardware processing unit may perform image enhancement processing on the original image blocks received by the hardware processing unit to obtain a plurality of processed image blocks.

[0064] The present disclosure does not limit the processing process of image enhancement processing, for example, it includes at least one of the following processing processes: image super-resolution reconstruction, image denoising, image deblurring, image dehazing, image contrast enhancement, image demosaicing, image color enhancement, image brightness enhancement, image detail enhancement, image dynamic range enhancement and image style transfer.

[0065] According to some embodiments of the present disclosure, in order to reduce the image processing time as much as possible, the terminal device can control each hardware processing unit to simultaneously perform image enhancement processing on the received original image blocks to obtain multiple processed images.

[0066] Specifically, a multi-threaded parallel execution method may be adopted to control each hardware processing unit to perform image enhancement processing simultaneously.

[0067] Still taking the hardware processing unit including NPU, GPU and CPU as an example, the terminal device can create three threads, which are respectively recorded as thread 1, thread 2 and thread 3.

[0068] Figure 6 Shown with Figure 5 Schematic diagram of the corresponding multi-threaded parallel execution of the image enhancement process.

[0069] When the above image processing time is determined for a test image of the same size as the original image block, as described above, the processing time of the NPU for a single original image block is 1 ms, the processing time of the GPU for a single original image block is 2.5 ms, and the processing time of the CPU for a single original image block is 5 ms. It can be understood that in some technologies, the time required to perform image enhancement processing using only the NPU is 16 ms, the time required to perform image enhancement processing using only the GPU is 40 ms, and the time required to perform image enhancement processing using only the CPU is 80 ms.

[0070] The image processing process of the embodiment of the present disclosure is applied, referring to Figure 6 , the processing time for performing image enhancement is 10 ms. It can be seen that through the solution of the embodiment of the present disclosure, the image processing time can be significantly reduced and the processing efficiency can be improved.

[0071] In addition to controlling each hardware processing unit to perform image enhancement processing simultaneously, the terminal device can also control each hardware processing unit to perform image enhancement processing in a non-simultaneous manner. For example, if there is a time difference when each hardware processing unit starts to perform image enhancement processing, even if the embodiment of the present disclosure adopts this method, compared with the solution in which only one hardware processing unit participates in the processing, it can at least alleviate the processing pressure and reduce the processing time.

[0072] In addition, among the multiple hardware processing units in the embodiment of the present disclosure, there may be a hardware processing unit that is only assigned to one original image block, or there may be a hardware processing unit that is assigned to multiple original image blocks. In the case where a target hardware processing unit among the multiple hardware processing units in the embodiment of the present disclosure receives multiple original image blocks, the terminal device may control the target hardware processing unit to perform image enhancement processing on the received original image blocks one by one.

[0073] refer to Figure 6 , NPU, GPU and CPU all correspond to the above-mentioned target hardware processing units. Taking GPU as an example, its image enhancement processing process is performed one by one on the original image blocks, that is, the original image block 11 is processed first, and then the original image blocks 12 to 14 are processed in sequence.

[0074] S38. Splice the multiple processed image blocks to generate a processed image corresponding to the original image.

[0075] After determining a plurality of processed image blocks in step S36, given that the processed image blocks correspond one-to-one to the original image blocks, the terminal device may splice the plurality of processed image blocks to generate a processed image corresponding to the original image.

[0076] In addition, the method for determining the number of original image blocks during the image segmentation process of step S32 can also be determined in combination with the image processing capabilities of each hardware processing unit. In other words, the terminal device can also determine the number of image block segments according to the image processing capabilities of each hardware processing unit, and segment the original image based on the number of image block segments to obtain multiple original image blocks.

[0077] For example, the terminal device includes n hardware processing units, and the time required for these n hardware processing units to process the test image individually is P1, P2, ..., Pn. Assuming that the original image is divided into S original image blocks, the number of original images received by each hardware processing unit is a1, a2, ..., an, and each hardware processing unit is controlled to perform image enhancement processing at the same time, and assuming that the total time for each hardware processing unit to perform image enhancement processing is the same, the following formulas 1 and 2 can be obtained:

[0078] a1×P1=a2×P2= …=an×Pn (Formula 1)

[0079] a1+a2+…+an=S (Formula 2)

[0080] According to formula 1 and formula 2, we can further get formula 3:

[0081]

[0082] Since P1, P2, ..., Pn are known, engineers or users can select an image block segmentation number S suitable for the current image processing task based on the above formula, and then perform the image segmentation process to obtain multiple original image blocks.

[0083] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0084] Furthermore, this exemplary embodiment also provides an image processing apparatus, which is applied to a terminal device, and the terminal device includes a plurality of hardware processing units.

[0085] Figure 7 FIG. 1 schematically shows a block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure. Figure 7 According to an exemplary embodiment of the present disclosure, the image processing device 7 may include an image segmentation module 71 , an image block allocation module 73 , an image block enhancement module 75 and an image block splicing module 77 .

[0086] Specifically, the image segmentation module 71 can be used to perform image segmentation on the original image to obtain multiple original image blocks; the image block allocation module 73 can be used to allocate each original image block to determine the original image block input to each hardware processing unit; the image block enhancement module 75 can be used for each hardware processing unit to perform image enhancement processing on the received original image block to obtain multiple processed image blocks; the image block stitching module 77 can be used to stitch multiple processed image blocks to generate a processed image corresponding to the original image.

[0087] According to an exemplary embodiment of the present disclosure, the image block allocation module 73 can be configured to perform: allocating each original image block according to the image processing capability of each hardware processing unit to determine the original image blocks input to each hardware processing unit; wherein the image processing capability of the hardware processing unit is positively correlated with the number of original image blocks received.

[0088] According to an exemplary embodiment of the present disclosure, referring to Figure 8 Compared with the image processing device 7 , the image processing device 8 may further include a processing capability determination module 81 .

[0089] Specifically, the processing capability determination module 81 can be configured to perform: obtaining the image processing time of each hardware processing unit, the image processing time being the time required to perform image enhancement processing on the test image block; determining the image processing capability of each hardware processing unit based on the image processing time of each hardware processing unit; wherein the image processing time of the hardware processing unit is negatively correlated with the image processing capability.

[0090] According to an exemplary embodiment of the present disclosure, the image block allocation module 73 may be configured to execute: determining the proportional relationship of the image processing time of each hardware processing unit; and allocating each original image block according to the proportional relationship to determine the original image blocks input to each hardware processing unit.

[0091] According to an exemplary embodiment of the present disclosure, the image block enhancement module 75 may be configured to execute: controlling each hardware processing unit to simultaneously perform image enhancement processing on the received original image blocks to obtain a plurality of processed image blocks.

[0092] According to an exemplary embodiment of the present disclosure, a target hardware processing unit among the multiple hardware processing units receives multiple original image blocks. In this case, the target hardware processing unit performs image enhancement processing on the received original image blocks one by one.

[0093] According to an exemplary embodiment of the present disclosure, the image segmentation module 71 can be configured to execute: determining the number of image block segmentations according to the image processing capabilities of each hardware processing unit; performing image segmentation on the original image based on the number of image block segmentations to obtain multiple original image blocks, each of which has the same size.

[0094] According to an exemplary embodiment of the present disclosure, image enhancement processing includes at least one of the following processing processes: image super-resolution reconstruction, image denoising, image deblurring, image dehazing, image contrast enhancement, image demosaicing, image color enhancement, image brightness enhancement, image detail enhancement, image dynamic range enhancement and image style migration.

[0095] Since the functional modules of the image processing device in the embodiment of the present disclosure are the same as those in the above-mentioned method embodiment, they will not be described in detail here.

[0096] Fig. 9 Schematic diagram of an electronic device suitable for implementing the exemplary embodiment of the present disclosure is shown. The terminal device of the exemplary embodiment of the present disclosure may be configured as follows Fig. 9 It should be noted that Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] The electronic device of the present disclosure includes at least a processor and a memory, wherein the memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor can implement the image processing method of the exemplary embodiment of the present disclosure.

[0098] Specifically, Fig. 9 As shown, the electronic device 90 may include: a processor 910, an internal memory 921, an external memory interface 922, a Universal Serial Bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 1, an antenna 2, a mobile communication module 950, a wireless communication module 960, an audio module 970, a sensor module 980, a display screen 990, a camera module 991, an indicator 992, a motor 993, a button 994, and a Subscriber Identification Module (SIM) card interface 995, etc. The sensor module 980 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, etc.

[0099] It is to be understood that the structure illustrated in the embodiment of the present disclosure does not constitute a specific limitation on the electronic device 90. In other embodiments of the present disclosure, the electronic device 90 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0100] The processor 910 may include one or more processing units, for example, the processor 910 may include an application processor (AP), a modem processor, a graphics processor (also known as a graphics processing unit), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (also known as a neural network processing unit), etc. Among them, different processing units may be independent devices or integrated in one or more processors. In addition, a memory may be provided in the processor 910 for storing instructions and data.

[0101] The electronic device 90 can realize the shooting function through the ISP, the camera module 991, the video codec, the GPU, the display screen 990 and the application processor. In some embodiments, the electronic device 90 may include 1 or N camera modules 991, N is a positive integer greater than 1, and if the electronic device 90 includes N cameras, one of the N cameras is a main camera. The original image in the embodiment of the present disclosure may be an image taken by the camera module 991.

[0102] The internal memory 921 can be used to store computer executable program codes, which include instructions. The internal memory 921 can include a program storage area and a data storage area. The external memory interface 922 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 90.

[0103] The present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device.

[0104] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0105] Computer-readable storage media can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0106] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the embodiments of the present disclosure.

[0107] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0108] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0109] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0110] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0111] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0112] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered merely as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0113] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: Applied to a terminal device, the terminal device includes a plurality of hardware processing units, and the image processing method includes: Performing image segmentation on the original image to obtain multiple original image blocks; Allocating the original image blocks to determine the original image blocks input to the hardware processing units; Each of the hardware processing units performs image enhancement processing on the received original image blocks to obtain a plurality of processed image blocks; The multiple processed image blocks are spliced ​​to generate a processed image corresponding to the original image.

2. The image processing method according to claim 1, characterized in that: Allocating the original image blocks to determine the original image blocks input to the hardware processing units includes: Allocating the original image blocks according to the image processing capabilities of the hardware processing units to determine the original image blocks input to the hardware processing units; The image processing capability of the hardware processing unit is positively correlated with the number of the received original image blocks.

3. The image processing method according to claim 2, characterized in that: The image processing method further comprises: Acquire the image processing time of each of the hardware processing units, where the image processing time is the time required to perform image enhancement processing on the test image; Determining the image processing capability of each of the hardware processing units according to the image processing time of each of the hardware processing units; The image processing time of the hardware processing unit is negatively correlated with the image processing capability.

4. The image processing method according to claim 3, characterized in that: Allocating the original image blocks according to the image processing capabilities of the hardware processing units to determine the original image blocks input to the hardware processing units includes: Determining a proportional relationship between the image processing times of the hardware processing units; The original image blocks are allocated according to the proportional relationship to determine the original image blocks input to each hardware processing unit.

5. The image processing method according to claim 1, characterized in that: Each of the hardware processing units performs image enhancement processing on the received original image blocks to obtain a plurality of processed image blocks, including: Control each of the hardware processing units to simultaneously perform image enhancement processing on the received original image blocks to obtain a plurality of processed image blocks.

6. The image processing method according to claim 5, characterized in that: The target hardware processing unit among the multiple hardware processing units receives the multiple original image blocks; wherein the image processing method further comprises: The target hardware processing unit performs image enhancement processing on the received original image blocks one by one.

7. The image processing method according to claim 1, characterized in that: The original image is segmented to obtain multiple original image blocks, including: Determining the number of image block divisions according to the image processing capability of each of the hardware processing units; The original image is segmented based on the number of image block segmentations to obtain a plurality of original image blocks.

8. The image processing method according to any one of claims 1 to 7, characterized in that: The image enhancement processing includes at least one of the following processing processes: image super-resolution reconstruction, image denoising, image deblurring, image defogging, image contrast enhancement, image demosaicing, image color enhancement, image brightness enhancement, image detail enhancement, image dynamic range enhancement and image style migration.

9. An image processing device, characterized in that: Applied to a terminal device, the terminal device includes a plurality of hardware processing units, and the image processing device includes: An image segmentation module, used for performing image segmentation on the original image to obtain a plurality of original image blocks; An image block allocation module, used for allocating the original image blocks to determine the original image blocks input to each hardware processing unit; An image block enhancement module, configured to perform image enhancement processing on the received original image blocks by each of the hardware processing units to obtain a plurality of processed image blocks; The image block splicing module is used to splice the multiple processed image blocks to generate a processed image corresponding to the original image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image processing method according to any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: include: processor; A memory for storing one or more programs, which, when executed by the processor, enables the processor to implement the image processing method according to any one of claims 1 to 8.

Images